3D Model Reconstruction Using Depth Maps from Real Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image editing systems struggle to accurately reconstruct 3D models from real images and synthetic images depicting dissimilar objects, due to a domain gap and the requirement for complex inputs like point clouds and 3D meshes, which limits training datasets and increases system complexity, leading to inaccurate and resource-intensive processing.
Innovation Solution
The system bridges the domain gap by using depth maps from real 2D images to train a 3D-object-reconstruction-machine-learning model, generating predicted signed distance functions, and adjusting parameters to reconstruct 3D models that better conform to real image objects, without relying on ground-truth SDF values or complex inputs like point clouds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional systems use synthetic images with ground-truth signed distance functions to train 3D reconstruction networks, then training data availability is improved, but manufacturing precision deteriorates because the systems cannot accurately reconstruct 3D models from real images or dissimilar objects
Solution Approach 1:
The patent introduces depth maps as an intermediary element that bridges the gap between 2D real images and 3D reconstruction. The depth map serves as a mediator that provides geometric information without requiring ground-truth signed distance functions, enabling the system to train on real images while maintaining reconstruction accuracy for novel objects
Solution Approach 2:
The patent changes the training parameters by replacing ground-truth signed distance functions with depth map information. This parameter substitution allows the system to work with real images that lack precise 3D annotations while still learning to reconstruct accurate 3D models through the depth-guided training approach
2Measurement precision
If conventional systems require complex inputs like point clouds, normal maps, 3D meshes or templates, then measurement precision may be improved, but device complexity increases and ease of operation deteriorates
Solution Approach 1:
The patent extracts and removes the requirement for complex inputs such as point clouds, normal maps, and 3D meshes from the system. By eliminating these cumbersome input requirements and relying solely on 2D images with depth maps, the system reduces complexity while maintaining the ability to generate accurate 3D reconstructions
Solution Approach 2:
The patent uses depth maps as simplified copies or representations of 3D information that can be derived from 2D images. Instead of requiring full 3D inputs, the system works with depth map copies that capture essential geometric information, reducing input complexity while preserving reconstruction quality
3Manufacturing precision
If conventional systems use complex inputs like 3D meshes, then manufacturing precision is improved, but loss of time increases due to heavy computational resources and slower processing speeds
Solution Approach 1:
The patent performs preliminary action by pre-computing and storing depth maps from 2D images before the 3D reconstruction process. This pre-processing step captures geometric information in advance, reducing the computational burden during actual reconstruction and thereby decreasing processing time while maintaining model quality
Data Source
AI summary
The present disclosure relates to systems, non-transitory computer-readable media, and methods that tune a 3D-object-reconstruction-machine-learning model to reconstruct 3D models of objects from real images using real images as training data. For instance, the disclosed systems can determine a depth map for a real two-dimensional (2D) image and then reconstruct a 3D model of a digital object in the real 2D image based on the depth map. By using a depth map for a real 2D image, the disclosed systems can generate reconstructed 3D models that better conform to the shape of digital objects in real images than existing systems and use such reconstructed 3D models to generate more realistic looking visual effects (e.g., shadows, relighting).