Multi-Resolution Depth Map Completion for Sparse Sensor Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Commodity-level depth sensors generate sparse depth maps, which cannot meet the high-level application requirements for tasks like path planning and obstacle avoidance in computer vision.
Innovation Solution
A method involving a cascade hourglass network that processes color maps and sparse depth maps in multiple resolutions to generate dense depth maps, filling in pixels in the sparse depth maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a cascade hourglass network processes all features from each level to generate dense depth maps, then the depth completion accuracy can be improved, but the processing overhead and computational complexity increase significantly
Solution Approach 1:
The patent applies local quality by using different processing strategies for different features at different resolutions. Specifically, it processes color maps and sparse depth maps differently at multiple resolution levels, with each level handling specific feature types appropriately rather than uniformly processing all features. This selective processing approach maintains depth completion accuracy while significantly reducing computational overhead compared to processing all features from every network level.
2Measurement precision
If a cascade hourglass network processes all features from each level to generate dense depth maps, then the depth completion accuracy can be improved, but the operation amount of the network increases
Solution Approach 1:
The patent implements segmentation by dividing the processing into multiple resolution levels (n different resolutions), where each level of the cascade hourglass network processes data at its specific resolution. This segmentation allows the network to handle different feature types at appropriate scales, reducing the total operation amount while maintaining comprehensive feature extraction capability across all levels for accurate depth completion.
3Ease of manufacture
If commodity-level depth sensors are used, then the system cost is reduced, but the depth map density is insufficient for high-level applications
Solution Approach 1:
The patent uses color maps as an intermediary to enhance the sparse depth maps from commodity sensors. By processing both color maps and sparse depth maps through the cascade hourglass network at multiple resolutions, the system generates dense depth maps that meet high-level application requirements. This intermediary approach allows the use of inexpensive commodity sensors while achieving the depth map density needed for path planning and obstacle avoidance.
Data Source
AI summary
A method of depth map completion is described. A color map and a sparse depth map of a target scenario can be received. Resolutions of the color map and the sparse depth map are adjusted to generate n pairs of color maps and sparse depth maps of n different resolutions. The n pairs of color maps and the sparse depth maps can be processed to generate n prediction result maps using a cascade hourglass network including n levels of hourglass networks. Each of the n pair is input to a respective one of the n levels to generate the respective one of the n prediction result maps. The n prediction result maps each include a dense depth map of the same resolution as the corresponding pair. A final dense depth map of the target scenario can be generated according to the dense depth maps.