Depth Image Fusion into Global Volumes for Mirror-Aware Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing spatial mapping technologies face challenges in accurately generating a global coordinate system for environments with mirrors, leading to positional drift and errors, especially when using head-mounted depth cameras, which can introduce reflections and destroy the 3D reconstruction of the scene behind mirrors.
Innovation Solution
A two-step fusion process is employed, where depth images are first fused into local volumes and then into a global volume, allowing for the correction and removal of erroneous data from local volumes, particularly those affected by mirror reflections, thereby maintaining accurate geometry without the need for extensive re-fusion of all data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If depth images are fused directly into a global volume, then the 3D reconstruction process is simplified, but errors from mirror reflections propagate throughout the entire global volume requiring extensive re-fusion
Solution Approach 1:
The patent divides the global volume into multiple local volumes, each associated with a specific coordinate system. This segmentation allows errors to be isolated to individual local volumes rather than propagating throughout the entire global reconstruction, enabling targeted correction without re-processing all data.
Solution Approach 2:
The patent extracts and identifies erroneous regions within local volumes that are caused by mirror reflections. By isolating these error regions, the system can remove or correct only the affected portions without impacting the rest of the 3D reconstruction, thereby maintaining high accuracy while preserving processing efficiency.
2Measurement precision
If mirror reflections are detected and corrected, then the accuracy of 3D reconstruction is improved, but additional processing time and computational resources are required
Solution Approach 1:
The patent performs preliminary detection of mirror reflections and identification of erroneous regions before final 3D reconstruction. By detecting and flagging problematic areas in advance, the system can apply targeted corrections only where needed, avoiding unnecessary re-processing of entire volumes and reducing overall processing time.
Solution Approach 2:
The patent applies different processing quality levels to different regions of the 3D space. Regions identified as containing mirror reflections receive enhanced processing and correction, while clean regions are processed more efficiently. This localized approach to quality control improves overall accuracy without uniformly increasing processing time across the entire scene.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method comprises receiving a plurality of depth images of an environment via an imaging device, each depth image associated with a local coordinate system. For each local coordinate system, each associated depth image is fused to generate a local volume. Each local volume is fused into a global volume having a global coordinate system. A surface mesh for the global volume is extracted. A floorplan layout for the global volume is determined based on the extracted surface mesh. One or more regions of inconsistency within a local volume that breaches the floorplan layout are determined. The one or more regions of inconsistency are removed from the local volume. The surface mesh for the global volume is re-extracted. Also provided is an imaging system which implements the method.