Monocular Depth Estimator Training With Surface-Normal Regularization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine-learning-based monocular depth estimation systems face challenges in achieving accurate depth estimates due to various factors during the training process, such as sensor limitations and noisy data, which affect the accuracy of depth predictions in robotics applications like autonomous vehicles and ADASs.
Innovation Solution
The system employs a regularization technique using self-supervised training with virtual datasets to generate ground-truth surface-normal vectors, computing a surface-normal loss between calculated and ground-truth vectors to improve the accuracy of depth predictions, leveraging densely annotated labels from computer simulators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional monocular depth estimation training is used, then the system can process images and generate depth predictions, but the accuracy of depth predictions deteriorates due to sensor limitations and noisy data
Solution Approach 1:
The patent creates virtual copies of real-world scenes through computer simulation, generating synthetic training images with perfectly known ground-truth depth maps. These virtual copies eliminate sensor noise and measurement errors present in real data, providing reliable training supervision while maintaining visual realism through rendered images from simulated camera sensors.
Solution Approach 2:
The patent introduces a computer simulator as an intermediary between the physical world and the machine learning system. The simulator acts as a mediator that translates real-world scenes into virtual representations with perfect ground-truth annotations, bridging the gap between noisy real data and the need for accurate supervision signals.
2Measurement precision
If virtual datasets with ground-truth depth maps are used for training, then depth prediction accuracy improves, but the complexity of the training system increases due to the need for simulators and surface-normal calculations
Solution Approach 1:
The training system uses the virtual environment to generate its own ground-truth data automatically. The computer simulator self-generates depth maps and surface-normal vectors without requiring manual annotation or external measurement devices, making the system self-sufficient and reducing the need for complex external infrastructure.
Solution Approach 2:
The patent moves the training process from the physical dimension to the virtual dimension, where ground-truth data can be obtained analytically from depth maps through mathematical surface-normal calculations. This dimensional shift from physical measurement to computational derivation simplifies the acquisition of precise training labels.
3Stability of the object's composition
If surface-normal loss regularization is applied during training, then depth prediction consistency improves, but the computational cost increases due to additional loss calculations
Solution Approach 1:
The patent pre-calculates surface-normal vectors from the ground-truth depth maps during data preparation, storing them for later use in loss computation. This preliminary action avoids redundant calculations during training iterations, reducing computational energy consumption while maintaining the stabilizing effect of surface-normal loss on depth predictions.
Data Source
AI summary
Systems and methods described herein relate to training a machine-learning-based monocular depth estimator. One embodiment selects a virtual image in a virtual dataset, the virtual image having an associated ground-truth depth map; generates a set of ground-truth surface-normal vectors for pixels in the virtual image based on the ground-truth depth map; processes the virtual image using the machine-learning-based monocular depth estimator to generate a predicted depth map; generates a set of calculated surface-normal vectors for the pixels in the virtual image based on the predicted depth map; and supervises training of the machine-learning-based monocular depth estimator by computing a surface-normal loss between the set of calculated surface-normal vectors and the set of ground-truth surface-normal vectors, wherein the surface-normal loss regularizes depth predictions produced by the machine-learning-based monocular depth estimator to improve accuracy of the depth predictions as the machine-learning-based monocular depth estimator is trained.