Monocular Depth Estimation With Generative Refinement of Coarse Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for depth estimation from monocular images face challenges in generating accurate and generalizable depth information due to the limitations of training data sets, which often result in coarse, noisy, and incomplete depth data, leading to models that are not adaptable across various environments.
Innovation Solution
A generative artificial intelligence model is trained using a coarse depth map aligned with a ground-truth map, masked based on patch-wise comparisons, and denoised iteratively to generate fine depth maps, allowing zero-shot training and improved fidelity across diverse scenes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are trained using training data sets with geometric prior information, then the models can generate depth information for input images, but the depth data is coarse, noisy, and incomplete due to lack of fine-grained depth data in training sets
Solution Approach 1:
The patent segments the depth estimation task into multiple refinement stages. A diffusion-based model iteratively refines coarse depth maps by progressively denoising them, dividing the estimation process into incremental improvement steps that recover fine-grained depth information lost in initial coarse estimates
Solution Approach 2:
The patent introduces an intermediary refinement model that acts as a mediator between coarse depth maps and fine-grained depth labels. This diffusion-based refinement model processes coarse depth maps and progressively denoises them to generate detailed depth information, bridging the gap between coarse training data and fine-grained depth requirements
2Measurement precision
If synthetic data sets are used to train generative models with detailed depth labels, then the models can generate detailed depth maps, but the models are not generalizable across various scenes due to limited scene variety
Solution Approach 1:
The patent applies preliminary action by first generating coarse depth maps using a pre-trained depth estimation model, then using these coarse maps as input for the diffusion-based refinement process. This preliminary coarse estimation provides a structured foundation that guides the subsequent detailed refinement, enabling the model to generalize better across different scenes
Solution Approach 2:
The patent changes the parameter representation by working in latent space rather than direct pixel space. The diffusion model operates on latent representations of depth maps, transforming the refinement process into a parameter optimization problem that improves generalizability while maintaining detailed output quality
3Measurement precision
If iterative refinement schemes are used to generate detailed depth maps, then the depth information includes granular and accurate depth information, but the computational resources and training complexity increase
Solution Approach 1:
The patent extracts the refinement function into a separate, specialized diffusion-based model that operates independently from the base depth estimation model. This modular extraction allows the refinement component to be optimized specifically for detail enhancement without complicating the overall system architecture, managing training complexity through functional separation
Data Source
AI summary
Embodiments of the present disclosure provide techniques for training a generative artificial intelligence model to generate depth estimates for an input image. An example method generally includes generating a coarse depth map from an input image in a training data set. The coarse depth map is aligned based on a ground-truth depth map corresponding to the input image in the training data set. A masked depth map is generated based on distances calculated between different portions of the aligned coarse depth map. A generative artificial intelligence model is trained to perform monocular depth estimation on an image of a scene based on the input image and the masked depth map. The trained generative artificial intelligence model is deployed.