Depth Estimation for Fine Structures Using Edge Tangent Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional depth from defocus methods for determining depth values in images, particularly around fine structures like hair, suffer from low resolution and noise issues, leading to inaccurate segmentation and visible artefacts in processed images.
Innovation Solution
The method involves capturing two images with different focus or aperture settings, generating a gradient orientation image, and applying edge tangent convolution to increase the coherence of blur difference signals, allowing for high-accuracy depth estimation and confident depth signal isolation for fine structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional depth from defocus methods are used to determine depth values, then depth mapping can be achieved, but the resolution is low and fine structures like hair cannot be resolved accurately
Solution Approach 1:
The image is divided into multiple overlapping square regions, and depth estimation is performed independently for each region. This segmentation allows the method to process local areas with different characteristics, improving the ability to resolve fine structures while maintaining overall depth map coverage.
Solution Approach 2:
The patent transforms the problem from 2D image space to 3D depth space by introducing depth as a new dimension. By estimating depth values for each pixel and organizing them into a depth map, the method creates an additional dimensional representation that preserves fine structure information while enabling depth-based processing.
2Measurement precision
If blur difference images are generated by convolving relative blur kernels with the first image and subtracting from the second image, then depth estimation can be performed, but the blur difference images are extremely noisy
Solution Approach 1:
The patent applies preliminary smoothing to the blur difference images before performing depth estimation. By smoothing these noisy images in advance, the method reduces the impact of noise on subsequent depth calculation while preserving the essential blur difference information needed for accurate depth estimation.
Solution Approach 2:
The patent introduces smoothed blur difference images as an intermediary step between the raw blur difference images and the final depth estimation. This intermediary representation filters out noise while maintaining the critical blur information, serving as a bridge that connects the noisy measurements to accurate depth values.
3Object-affected harmful factors
If square region averaging is applied to compensate for noise in blur difference images, then noise is reduced, but the resulting depth map has lower resolution and fine structure is not resolved
Solution Approach 1:
Instead of averaging over large square regions, the patent segments the image into multiple overlapping square regions and processes each independently. This segmentation approach allows noise reduction within each small region while preserving fine structure details that would be lost in large-area averaging.
Solution Approach 2:
The patent applies different processing characteristics to different local regions of the image. By treating each square region independently with its own blur difference calculation and smoothing, the method adapts to local variations in texture and structure, maintaining high resolution in fine structure areas while still reducing noise where appropriate.
4Measurement precision
If error minimisation method with non-local means filter is used to compensate for blur difference noise, then fine structure can be traced, but the algorithm is complex and takes a long time to process
Solution Approach 1:
The patent extracts only the essential smoothing operation needed to reduce noise in blur difference images, separating it from the complex error minimisation framework. By taking out just the necessary smoothing step and applying it directly to blur difference images, the method achieves fine structure preservation without the computational overhead of iterative error minimisation and non-local means filtering.
Data Source
AI summary
A method of determining a depth value of a fine structure pixel in a first image of a scene using a second image of the scene is disclosed. A gradient orientation for each of a plurality of fine structure pixels in the first image is determined. Difference images are generated from the second image and a series of blurred images formed from the first image, each difference image corresponding to one of a plurality of depth values. Each of the difference images is smoothed, in accordance with the determined gradient orientations, to generate smoothed difference images having increased coherency of fine structure. For each of a plurality of fine structure pixels in the first image, one of the smoothed difference images is selected. The depth value of the fine structure pixel corresponding to the selected smoothed difference image is determined.