Depth Map Hole Filling via Intensity Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth image reconstruction methods face challenges in filling holes, particularly in textureless regions and areas with insufficient feature points or occlusions, leading to incomplete and inaccurate depth maps.
Innovation Solution
A two-stage iterative method that combines intensity image segmentation with depth estimation using Markov Random Fields and normalized cuts segmentation to co-adjust segmentation results and depth images, filling holes and reducing noise in depth maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional stereo reconstruction methods are used, then processing speed is maintained, but depth map completeness deteriorates due to holes in textureless regions
Solution Approach 1:
The patent applies image segmentation to divide the depth map into multiple regions based on intensity image analysis. By segmenting the image into distinct regions, the method can identify textureless areas and apply appropriate filling strategies to each region, thereby improving depth map completeness without requiring complex global processing.
Solution Approach 2:
The patent introduces an intermediary process that uses intensity image segmentation results to guide depth map hole filling. The segmentation map serves as an intermediary between the raw stereo images and the final depth map, providing region information that helps fill holes in textureless areas while maintaining processing efficiency.
2Measurement precision
If hole filling is applied to all regions, then depth map completeness improves, but noise and inaccuracies worsen in regions with insufficient feature points
Solution Approach 1:
The patent applies local quality by treating different regions of the depth map differently based on their characteristics. Regions with sufficient feature points use standard reconstruction, while textureless regions use segmentation-guided hole filling. This localized approach ensures that hole filling operations are applied only where necessary, improving accuracy without introducing noise in regions with insufficient features.
3Measurement precision
If segmentation is applied to guide hole filling, then depth map accuracy improves, but processing time increases
Solution Approach 1:
The patent performs preliminary segmentation of the intensity image before depth map reconstruction. By pre-identifying textureless regions and creating a segmentation map in advance, the method prepares region information that can be quickly referenced during depth map processing, reducing the computational burden during the actual hole filling operation.
Data Source
AI summary
Various implementations relate to improving depth maps. This may be done, for example, by identifying bad depth values and modifying those values. The values may represent, for example, holes and/or noise. According to a general aspect, a segmentation is determined based on an intensity image. The intensity image is associated with a corresponding depth image that includes depth values for corresponding locations in the intensity image. The segmentation is applied to the depth image to segment the depth image into multiple regions. A depth value is modified in the depth image based on the segmentation. A two-stage iterative procedure may be used to improve the segmentation and then modify bad depth values in the improved segmentation, and iterating until a desired level of smoothness is achieved. Both stages may be based, for example, on average depth values in a segment.