Depth Determiner for Plenoptic Camera Light Fields
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current algorithms for estimating depth maps from 4D light fields face challenges in accurately estimating depth at textureless areas and depth discontinuities, often resulting in over-smoothed and inaccurate depth maps due to high computational complexity, making real-time implementation difficult with plenoptic cameras.
Innovation Solution
An image processing apparatus and method that determines depth by combining depth estimates from multiple image subsets captured at different grid locations, using a depth determiner to compute and combine depth estimates efficiently, and optionally incorporating confidence values and structure tensor analysis to improve reliability and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current algorithmic solutions are used for depth map estimation from 4D light fields, then depth map can be obtained, but the computational complexity is high making real-time implementation difficult
Solution Approach 1:
The patent segments the 4D light field data into multiple 2D images captured at regular camera positions, processing each image individually to estimate depth maps. This segmentation approach reduces the overall computational complexity by breaking down the complex 4D light field processing into manageable 2D image processing tasks that can be executed in real-time
Solution Approach 2:
The patent performs preliminary depth map estimation from individual 2D images before combining them into a final depth map. This preliminary action allows for efficient local processing of each image separately, and the results are then merged to produce the final depth estimation, enabling real-time processing while maintaining accuracy
2Measurement precision
If conventional depth estimation algorithms are applied, then processing speed can be maintained, but depth accuracy at textureless areas and depth discontinuities deteriorates due to over-smoothing
Solution Approach 1:
The patent merges multiple depth maps estimated from different 2D images captured at regular camera positions to produce a final depth map. This merging process combines information from multiple views, improving depth estimation accuracy at textureless areas and depth discontinuities without requiring excessive computational resources, as each individual depth map can be computed efficiently
Solution Approach 2:
The patent uses confidence maps as intermediary data structures to guide the merging process. The confidence map indicates the reliability of depth estimates from each image, allowing the algorithm to weight and combine depth values appropriately. This intermediary mechanism improves accuracy at challenging regions while maintaining processing efficiency through systematic combination rules
Data Source
AI summary
The disclosure relates to an image processing apparatus for determining a depth of a pixel of a reference image of a plurality of images representing a visual scene relative to a plurality of locations, wherein the plurality of locations define a two-dimensional grid with rows and columns and wherein the location of the reference image is associated with a reference row and a reference column of the grid. The image processing apparatus comprises a depth determiner configured to determine a first depth estimate on the basis of the reference image and a first subset of the plurality of images for determining the depth of the pixel of the reference image, wherein the images of the first subset are associated with locations being associated with a row of the grid different than the reference row and with a column of the grid different than the reference column.