Depth Map Fusion for Occlusion and Low-Confidence Regions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robotic systems face challenges in navigating and interacting with environments due to incomplete or inaccurate depth information from different sensors, such as stereoscopic and monoscopic depth maps, which lack scale and reliability for transparent, partially-transparent, refractive, or textureless materials and are hindered by occlusions.
Innovation Solution
A method and system that fuse depth maps from multiple sensors, aligning and transforming regions with relative scales to match reference scales, generating a cohesive depth map with complete and accurate information by 'inpainting' regions with low depth information, using optimization functions to ensure piecewise smoothness and scale consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth maps from multiple sensors are used, then the completeness and accuracy of depth information is improved, but the complexity of processing and integrating these depth maps increases
Solution Approach 1:
The patent combines multiple depth maps from different sensors (stereoscopic and monoscopic) into a single integrated depth map. The system identifies regions with low depth information in the first depth map and inpaints them using corresponding regions from the second depth map, merging the strengths of multiple sensors to produce a complete and accurate final depth map.
Solution Approach 2:
The patent introduces an intermediary processing system that acts as a mediator between multiple depth maps. This system aligns depth maps with different scales, identifies regions needing enhancement, and seamlessly integrates them through inpainting operations, resolving the complexity of direct multi-sensor integration.
2Loss of information
If regions with low depth information are inpainted using optimization functions, then the completeness of the depth map is improved, but the computational time and processing resources increase
Solution Approach 1:
The patent applies partial action by selectively inpainting only the specific regions with low depth information rather than processing the entire depth map. The system identifies regions failing to meet a threshold and applies optimization functions only to those areas, reducing overall computational time while maintaining completeness.
Solution Approach 2:
The patent applies local quality by using different processing strategies for different regions of the depth map. Regions with sufficient depth information are left unchanged, while regions with low depth information undergo inpainting with optimization functions, ensuring each region receives appropriate processing based on its specific needs.
3Measurement precision
If depth maps with different scales are aligned and transformed, then the consistency and accuracy of the fused depth map is improved, but the complexity of scale transformation and alignment increases
Solution Approach 1:
The patent changes the scale parameter of depth maps to achieve consistency. The system transforms the second depth map's scale to match the first depth map's reference scale, enabling accurate fusion while maintaining measurement precision through systematic parameter adjustment.
Solution Approach 2:
The patent creates equipotentiality by aligning all depth maps to a common reference scale. This eliminates scale discrepancies between different sensors, ensuring that all depth information contributes equally and consistently to the final fused depth map without hierarchical or scale-based conflicts.
Data Source
AI summary
A method includes receiving a first depth map that includes a plurality of first pixel depths and a second depth map that includes a plurality of second pixel depths. The first depth map corresponds to a reference depth scale and the second depth map corresponds to a relative depth scale. The method includes aligning the second pixel depths with the first pixel depths. The method includes transforming the aligned region of the second pixel depths such that transformed second edge pixel depths of the aligned region are coextensive with first edge pixel depths surrounding the corresponding region of the first pixel depths. The method includes generating a third depth map. The third depth map includes a first region corresponding to the first pixel depths and a second region corresponding to the transformed and aligned region of the second pixel depths.