Multi-Baseline Depth Estimation Using Confidence-Weighted Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile electronic devices face challenges in accurately estimating depth in scenes with repetitive or feature-less patterns, leading to inaccurate disparity and depth calculations, which affects various image processing operations.
Innovation Solution
The use of multiple imaging sensors arranged in a non-linear manner to capture input image frames along multiple baseline directions, combined with machine learning algorithms to generate disparity and confidence maps, which are then fused to produce a high-accuracy depth map.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple imaging sensors are used to capture images along multiple baseline directions, then depth estimation accuracy is improved, but device complexity increases
Solution Approach 1:
The patent transitions from traditional two-camera horizontal baseline arrangement to a multi-camera array with baselines in multiple directions (horizontal, vertical, diagonal). This dimensional expansion allows depth estimation along multiple axes, resolving ambiguities in repetitive patterns and significantly improving depth measurement precision while managing device complexity through systematic sensor positioning.
2Device complexity
If disparity processing is performed using traditional two-camera setup, then device complexity is kept low, but depth estimation accuracy deteriorates in scenes with repetitive or feature-less patterns
Solution Approach 1:
The patent segments the depth estimation task by generating separate disparity maps for each baseline direction (horizontal, vertical, diagonal) and then fusing them through confidence-weighted averaging. This segmentation allows each disparity map to specialize in capturing depth information along its specific direction, improving overall accuracy while maintaining manageable processing complexity through modular fusion.
Solution Approach 2:
The patent combines multiple disparity maps from different baseline directions into a composite depth map using confidence-weighted fusion. This composite approach integrates the strengths of each individual disparity map, creating a robust depth estimation that overcomes the limitations of any single baseline direction, particularly in challenging scenes with repetitive or feature-less patterns.
3Measurement precision
If confidence maps are generated and used to weight disparity maps, then depth map accuracy is improved, but processing time and computational complexity increase
Solution Approach 1:
The patent generates confidence maps in parallel with disparity map generation, preparing weight information in advance before the fusion step. This preliminary action allows the confidence weights to be ready when needed for weighted averaging, improving depth map accuracy while minimizing additional processing time by avoiding sequential operations.
Data Source
AI summary
A method includes obtaining at least three input image frames of a scene captured using at least three imaging sensors. The input image frames include a reference image frame and multiple non-reference image frames. The method also includes generating multiple disparity maps using the input image frames. Each disparity map is associated with the reference image frame and a different non-reference image frame. The method further includes generating multiple confidence maps using the input image frames. Each confidence map identifies weights associated with one of the disparity maps. In addition, the method includes generating a depth map of the scene using the disparity maps and the confidence maps. The imaging sensors are arranged to define multiple baseline directions, where each baseline direction extends between the imaging sensor used to capture the reference image frame and the imaging sensor used to capture a different non-reference image frame.