Multi-view Depth Map Refinement for 3D Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth map technologies struggle to maintain consistency between views, leading to inaccurate 3D object reconstruction due to inconsistent depth information.
Innovation Solution
A processor-implemented method that generates points for a point cloud by unprojecting multi-view depth maps into a 3D space, extracts feature embedding vectors, generates a 2D feature map, and uses a refinement network to produce a residual depth map, which is then combined with an initial depth map to generate an improved depth map.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing depth map technologies are used to generate depth maps from multiple views, then depth maps can be obtained for each view, but consistency between views is poor leading to inaccurate 3D reconstruction
Solution Approach 1:
The patent merges depth information from multiple views by unprojecting depth maps from different camera viewpoints into a common 3D space to form a point cloud. This combines multiple depth maps into a unified representation, improving both inter-view consistency and 3D reconstruction accuracy by integrating information from all views rather than processing them independently
Solution Approach 2:
The patent employs an iterative refinement process where the point cloud and feature maps are repeatedly updated through multiple iterations. In each iteration, the refinement network processes the current point cloud and generates improved depth maps, which are then unprojected again to update the point cloud. This feedback loop progressively enhances depth map consistency and reconstruction accuracy
2Measurement precision
If simple depth map generation is used, then processing speed is fast, but depth information accuracy is insufficient for reliable 3D reconstruction
Solution Approach 1:
The patent performs preliminary feature extraction from RGB images using a pre-trained network before depth map generation. This preliminary action creates rich feature embeddings that guide the subsequent depth refinement process, enabling more accurate depth estimation without requiring excessive processing time during the main reconstruction phase
Solution Approach 2:
The patent uses a refinement network that processes only the necessary portions of the data iteratively. Rather than processing all depth maps to full resolution simultaneously, the system refines depth information progressively through multiple passes, applying computational resources where most needed while maintaining overall processing efficiency
Data Source
AI summary
A method and apparatus with depth map generation. The method may include generating points for a point cloud by unprojecting multi-view depth maps, of plural views, into a corresponding three-dimensional (3D) space using respective camera parameters corresponding to each view of the multi-view depth maps, extracting feature embedding vectors corresponding to the generated points, generating a two-dimensional (2D) feature map of a set view based on the extracted feature embedding vectors, generating a residual depth map using a refinement network with respect to the 2D feature map, generating a new depth map based on the residual depth map and an initial depth map, of the set view, among the multi-view depth maps.