Depth-Map Estimation Using Local Structure Descriptors for Noise Robustness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth-map estimation methods are sensitive to noise and computationally intensive, particularly when using patches of pixels for similarity estimation, which is a critical issue for real-time applications and mobile devices.
Innovation Solution
The method employs local spatial structure descriptors, determined ahead of depth estimation, to reduce computational complexity by transforming patches of pixels into fewer descriptors, using auto-correlation matrices or Discrete Cosine Transform, and computes similarity measures based on these descriptors and color components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If similarity between pixels is computed using only color components (R, G, B), then computation is fast, but depth estimation becomes very sensitive to noise
Solution Approach 1:
The patch of pixels surrounding each pixel is segmented into local spatial structure descriptors using auto-correlation matrices. This segmentation extracts essential structural characteristics while filtering out noise, allowing depth estimation to be based on robust structural features rather than noisy color values alone.
Solution Approach 2:
Auto-correlation matrices serve as an intermediary between raw pixel data and depth estimation. These matrices capture the local spatial structure and relationships between pixels, providing a noise-resistant representation that mediates the comparison between corresponding pixels in stereo images.
2Reliability
If similarity between pixels is computed using a patch of pixels surrounding each pixel, then noise sensitivity is reduced, but computational complexity increases significantly
Solution Approach 1:
The invention extracts only the essential local spatial structure descriptors from the patch of pixels using auto-correlation matrices. Instead of processing all pixel values in the patch, it extracts a compact representation that captures the essential structural information needed for accurate depth estimation while dramatically reducing computational complexity.
Solution Approach 2:
The patent transforms the patch of pixels from the spatial domain to a descriptor space using auto-correlation matrices. This parameter transformation changes the representation from raw pixel intensities to structural descriptors, reducing the dimensionality and complexity of the data while preserving the essential information for depth estimation.
3Reliability
If patches of pixels are used for similarity estimation, then robustness against noise is improved, but real-time processing capability is lost
Solution Approach 1:
The local spatial structure descriptors are computed in advance using auto-correlation matrices before the actual depth estimation process. This preliminary computation prepares the data in a form that is both noise-resistant and computationally efficient for subsequent similarity comparisons, enabling real-time processing.
Solution Approach 2:
The patent replaces the computationally intensive mechanical comparison of full pixel patches with a more efficient system based on auto-correlation descriptors. This substitution maintains the noise robustness of patch-based methods while achieving the processing speed required for real-time applications on mobile devices.
Data Source
Figure 1~7
Figure 2
Figure 3~4
AI summary
A method and an apparatus for depth-map estimation between at least two images respectively captured by at least two cameras are disclosed. Extrinsic and intrinsic parameters of the at least two cameras are obtained (20). Local spatial structure descriptors for each pixel of the at least two images are obtained (21) by transforming a patch of pixels surrounding each pixel, and depth is obtained (22) for at least one pixel of a first image among the at least two images, among at least two depth candidates, each depth candidate being associated with a corresponding pixel in a second image of the at least two images, said corresponding pixel in the second image being determined according to the obtained extrinsic and intrinsic parameters. Obtaining depth comprises determining (222) a similarity measure between the at least one pixel of the first image and each corresponding pixel in the second image, the similarity measure taking into account color components and local spatial structure descriptors of the corresponding pixel and the at least one pixel of the first image, and selecting (223) a pixel among the corresponding pixels in the second image that minimizes the similarity measure.