Light Field Depth Estimation via Multi-Stage Cost Volume Refinement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth estimation methods for light-field imaging provide coarse information, limiting the quality of reconstructed images in applications like image segmentation and bokeh effects.
Innovation Solution
A method involving computing a first cost volume based on cost matching between pixels of sub-images for disparity values, generating a first depth map, reconstructing a view image, computing a second cost volume, filtering it to create an aggregated cost volume, and generating a second depth map to enhance image resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing depth estimation methods are used for light-field imaging, then the processing is simpler and faster, but the depth information quality is coarse and limits reconstructed image quality
Solution Approach 1:
The patent divides the depth estimation process into multiple stages: first generating a coarse depth map from light-field sub-images, then using this coarse depth map to guide a second round of stereo matching on selected view pairs to produce a refined depth map. This multi-stage segmentation approach improves depth information quality while managing processing complexity through hierarchical refinement.
Solution Approach 2:
The patent performs preliminary depth estimation using the light-field camera's inherent depth capabilities to generate a coarse depth map before conducting the more computationally intensive stereo matching process. This preliminary action provides guidance for subsequent processing steps, improving overall efficiency and accuracy.
2Measurement precision
If multi-view stereo matching is performed on all view pairs, then depth accuracy improves, but computational time and resources increase significantly
Solution Approach 1:
The patent segments the set of all possible view pairs into a selected subset based on the coarse depth map guidance. Only these selected view pairs undergo multi-view stereo matching, reducing computational time while maintaining depth accuracy through intelligent selection of the most informative view combinations.
Solution Approach 2:
The patent performs stereo matching on a partial subset of view pairs rather than all possible pairs. The coarse depth map enables identification and selection of only those view pairs that contribute most significantly to depth accuracy, avoiding unnecessary computations on redundant view combinations.
3Manufacturing precision
If coarse depth information is used for image segmentation and bokeh effects, then processing is faster, but the quality of reconstructed images and segmentation results deteriorates
Solution Approach 1:
The patent segments the depth estimation process into two phases: an initial fast phase using light-field sub-images to obtain coarse depth information, followed by a refinement phase applying multi-view stereo matching on selected views. This produces high-quality depth maps suitable for image segmentation and bokeh effects while maintaining reasonable processing efficiency through the hierarchical approach.
Solution Approach 2:
The patent performs preliminary processing with the light-field camera to generate coarse depth maps quickly, then uses these results to guide a more detailed refinement process. This preliminary action enables fast initial processing while the subsequent refinement step ensures high-quality output for applications requiring precise depth information.
Data Source
Figure 1A~1C
Figure 2~3
Figure 4~5
AI summary
In an example embodiment, a method, apparatus and computer program product are provided. The method includes computing a first cost volume for a light-field image. A first depth map comprising depth information of the plurality of sub-images of the light-field image is computed based on the first cost volume. A first view image comprising reconstruction information is reconstructed based on the depth information of the plurality of sub-images. A second cost volume corresponding to the first cost volume is computed based on the reconstruction information. The second cost volume is filtered based on the first view image to generate an aggregated cost volume. A second depth map is generated based on the aggregated cost volume. The second depth map facilitates generation of a second view image that is associated with a resolution higher than a resolution of the first view image.