Depth Map Prediction Using Compressed Cost Volume and Adaptive Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for generating depth images are prone to inaccuracies due to 'defected' penalty terms in cost volume functions and filters, leading to discontinuities and poor object representation, and face challenges with high computation requirements and labeling errors, making it difficult to deploy machine learning techniques for depth image generation.
Innovation Solution
A machine learning model is trained using a compressed cost volume and total loss function that includes boundary, occlusion, and transfer losses, eliminating the need for human-designed penalty terms and reducing computation resources, allowing for accurate depth map predictions and deployment on edge devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional passive methods with cost volume functions and smoothness constraints are used to generate depth images, then the process is computationally feasible, but the depth images contain inaccuracies and discontinuities due to defected penalty terms
Solution Approach 1:
The patent replaces the mechanical/optical system of conventional passive depth estimation (cost volume functions, photometric matching, smoothness constraints) with a machine learning-based system. The neural network learns optimal depth prediction directly from image data, eliminating the need for hand-crafted penalty terms and cost volume computations that cause discontinuities.
Solution Approach 2:
The patent changes the fundamental parameters of the depth estimation process by using learned parameters from training data instead of fixed mathematical constraints. The smoothness constraint parameter α in conventional methods is replaced by learned representations that adaptively handle depth discontinuities at object boundaries.
2Measurement precision
If machine learning techniques are adopted for depth image generation, then accuracy can be improved, but computation resources and training complexity increase significantly
Solution Approach 1:
The patent performs preliminary action by pre-training the machine learning model on large datasets of image-depth pairs before deployment. This offline training phase captures complex depth-image relationships, enabling the model to make accurate predictions during runtime with minimal computation, thus resolving the contradiction between accuracy and real-time performance.
Solution Approach 2:
The patent uses copying by training the model on synthetic or labeled depth data that replicates real-world scenarios. The learned model then copies the depth estimation capability to new, unseen images without requiring additional computation resources for each new prediction.
3Ease of manufacture
If uniform smoothness constraints are applied to all pixels in depth images, then the optimization process is simplified, but object boundaries and depth discontinuities are poorly preserved
Solution Approach 1:
The patent applies local quality by allowing different regions of the image to have different depth estimation characteristics. The machine learning model learns to apply appropriate smoothing or discontinuity preservation locally at each pixel based on learned features, rather than applying uniform constraints across the entire image.
Solution Approach 2:
The patent introduces dynamics by making the smoothness constraint adaptive rather than static. The model dynamically adjusts the degree of smoothing at each location based on learned patterns, allowing strong discontinuities at object boundaries while maintaining smoothness in uniform regions, thus resolving the contradiction between simplicity and precision.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a machine learning model configured to generate a predicted depth image, comprising receiving data representing training samples that include a plurality of image pairs, each image pair includes a target image and a reference image both capturing a particular scene from different orientations; for each of the plurality of image pairs, generating a compressed cost volume for the image pair; providing the compressed cost volume as an input to the machine learning model; generating, using the machine learning model, output data representing a predicted disparity map for the compressed cost volume; and generating a total loss using the predicted disparity map for the compressed cost volume, the total loss includes a boundary loss, an occlusion loss, and a transfer loss; and updating the plurality of parameters of the machine learning model by minimizing the total losses.