Depth Map Generation Using PatchMatch and Poisson Refinement

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for generating depth maps from multi-view images, such as the PatchMatch method, face challenges in accurately estimating depth values in regions with limited texture information, leading to low accuracy and inconsistency in depth values.

Innovation Solution

A method that generates an initial depth map, normal map, and reliability map using PatchMatch, followed by triangulation and regularization to select additional vertices and refine depth values, incorporating vectorial total variation for improved accuracy, and re-performing PatchMatch to enhance depth map quality.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If PatchMatch method is used to generate depth map from multi-view images, then processing efficiency is improved, but depth estimation accuracy deteriorates in regions with limited texture information

Engineering Contradiction:

Improveprocessing efficiencyVSAvoiddepth estimation accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent segments the depth estimation process into two distinct phases: (1) PatchMatch-based coarse depth estimation for efficient processing, and (2) Poisson equation-based refinement for accuracy improvement. This segmentation allows each method to operate in its optimal regime - PatchMatch for speed and Poisson for precision - thereby resolving the contradiction between processing efficiency and depth estimation accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary depth estimation using PatchMatch method to generate an initial depth map before applying the Poisson equation refinement. This preliminary action provides a reasonable starting point that guides the subsequent refinement process, ensuring both efficient processing and improved accuracy in the final depth map.

Inventive Principle:

Principle #10Preliminary action

2Speed

If PatchMatch method is used for depth estimation, then computational speed is improved, but depth value consistency deteriorates across images

Engineering Contradiction:

Improvecomputational speedVSAvoiddepth value consistency

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The patent implements a feedback mechanism where the initial depth map generated by PatchMatch serves as input to the Poisson equation refinement process. The refinement process uses this feedback to adjust and correct depth values across multiple images, ensuring consistency while maintaining the computational efficiency of the original PatchMatch approach.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The Poisson equation refinement acts as an intermediary process between the PatchMatch method and the final depth maps. It mediates the depth values across multiple images, ensuring consistency while preserving the computational speed benefits of PatchMatch by only refining critical areas.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Measurement precision

If image segmentation technique is added to improve depth estimation accuracy, then depth estimation accuracy is improved, but device complexity increases

Engineering Contradiction:

Improvedepth estimation accuracyVSAvoiddevice complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent replaces complex image segmentation techniques with a mathematical physics-based approach using the Poisson equation. This substitution achieves improved depth estimation accuracy through continuous optimization rather than discrete segmentation, thereby improving accuracy while avoiding the complexity associated with traditional segmentation methods.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Measurement precision

If Pyramid structure is used for generating depth map, then depth estimation accuracy is improved in certain regions, but adaptability deteriorates

Engineering Contradiction:

Improvedepth estimation accuracyVSAvoidrange of applicable regions

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent creates a universal depth refinement method using the Poisson equation that can be applied to all regions of all input images regardless of texture characteristics. Unlike pyramid structures that are limited to specific regions, this approach provides multi-functional capability to improve depth estimation across diverse scenarios including textured regions, non-textured regions, and images with varying lighting conditions.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12073576B2Apparatus and method for generating depth map from multi-view image

Publication Date: 2024.08.27 ELECTRONICS & TELECOMM RES INST

AI summary

Disclosed herein is a method for generating a depth map from multi-view images, the method including receiving multi-view images and parameters thereof; generating an initial depth map, an initial normal map, and an initial reliability map for the multi-view images; performing selection of multiple initial vertices and triangulation based on the initial depth map and the initial normal map and generating depth values and normal values for the result of triangulation; performing regularization based on triangulation, selecting additional vertices in the initial depth map, and generating additional depth values and additional normal values based on the additional vertices and regularization; and refining the initial depth map into a refined depth map by replacing depth values and normal values having reliability below a preset level in the initial depth map and the initial normal map with the additional depth values and the additional normal values.