3D Environment Reconstruction Using Confidence-Weighted Depth Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
XR systems face challenges in efficiently creating and updating a 3D representation of the environment with limited computational resources, particularly in accurately positioning virtual objects relative to real objects and handling occlusions and environmental changes.
Innovation Solution
A method using depth sensors to maintain a map of objects in the environment, updating it frequently with low computational burden by selectively removing undetected objects based on confidence levels and using truncated signed distance functions to manage voxel weights, facilitating efficient 3D reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional 3D reconstruction methods are used to create accurate 3D representations, then the quality and accuracy of the reconstruction is improved, but the computational resources and processing time required increase significantly
Solution Approach 1:
The patent extracts and removes pixels from depth images that are unlikely to provide useful information for 3D reconstruction. By filtering out unnecessary pixels based on confidence levels and validity criteria, the system reduces the computational burden while maintaining reconstruction accuracy with the remaining essential pixels.
Solution Approach 2:
The patent applies different processing treatments to different regions of the depth image based on local characteristics. Valid pixels that provide useful information are processed with high accuracy, while invalid pixels are discarded. This localized quality approach ensures accurate reconstruction where needed while reducing overall computational complexity.
2Measurement precision
If all pixels in depth images are processed to maintain high reconstruction accuracy, then the quality of 3D representation is improved, but the processing time and computational load increase
Solution Approach 1:
The patent extracts only the essential valid pixels from depth images for processing, discarding invalid pixels that would consume computational resources without contributing to reconstruction accuracy. This extraction principle directly improves processing speed by reducing the number of pixels that need to be handled.
Solution Approach 2:
The patent applies partial action by processing only a subset of pixels (the valid ones) rather than all pixels in the depth image. This selective processing maintains sufficient reconstruction accuracy while significantly reducing computational load and improving processing throughput.
3Reliability
If the 3D representation is updated frequently to reflect environmental changes, then the realism and accuracy of virtual object interactions is improved, but the computational burden increases
Solution Approach 1:
The patent extracts only the necessary valid pixels from each depth image for updating the 3D representation, discarding invalid pixels. This approach enables frequent updates to reflect environmental changes while keeping the computational burden manageable by processing only essential data.
Solution Approach 2:
The patent implements periodic updating of the 3D representation at defined intervals or triggers, rather than continuous processing. This periodic action allows the system to maintain reliable environmental representation while reducing overall computational burden compared to continuous full-image processing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method to reconstruct an environment with depth information is provided. The method makes available to a wide variety of XR applications fresh and accurate 3D reconstruction data of environments with low processing time and low usage of computational resources and storage spaces. The method includes capturing depth information about an environment upon a change within a field-of-view of a user caused by, for example, motions of headpose, user location, and/or objects, and updating a representation of the environment based on depths associated with different confidence levels by the captured depth information. The method may include updating a geometry of the environment based on depths associated with confidence levels higher than a threshold. The method may include, upon a change of the 3D environment, inferring the removal of surfaces in the environment based on depths associated with confidence levels lower than the threshold.