Monocular Depth Estimator Training via Camera Intrinsic Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine-learning-based monocular depth estimation systems face accuracy issues due to differences in camera intrinsics between virtual and real-world datasets, leading to reduced performance in depth estimation tasks.
Innovation Solution
A system that transforms virtual images by generating a point cloud based on ground-truth depth information and reprojecting it according to real-world camera intrinsics, ensuring consistency between virtual and real datasets for improved training of the monocular depth estimator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If virtual datasets are used for training, then training efficiency and ground-truth depth availability are improved, but camera intrinsic mismatch reduces depth estimation accuracy
Solution Approach 1:
The patent creates transformed virtual images that copy the camera intrinsic properties of real-world cameras. By reprojecting 3D point clouds using real camera intrinsics instead of virtual camera intrinsics, the virtual training data replicates the optical characteristics of real cameras, eliminating the mismatch problem while preserving the efficiency benefits of synthetic data generation.
Solution Approach 2:
The patent changes the camera intrinsic parameters used in the reprojection process. Instead of using the virtual camera's focal length, principal point, and distortion coefficients, the system transforms these parameters to match the real-world camera's characteristics. This parameter transformation aligns the virtual training data with real camera behavior, improving depth estimation accuracy without sacrificing training efficiency.
2Adaptability or versatility
If virtual datasets with different camera intrinsics are used, then diverse training data is generated, but inconsistency between virtual and real datasets degrades model performance
Solution Approach 1:
The patent performs preliminary transformation of virtual images before they are used for training. By pre-aligning the camera intrinsics of virtual images to match real-world cameras through 3D point cloud reprojection, the system ensures that all training data (both virtual and real) shares consistent camera characteristics. This preliminary alignment action prevents the inconsistency problem from affecting model training while maintaining data diversity through various virtual scene configurations.
Data Source
AI summary
Systems and methods described herein relate to training a machine-learning-based monocular depth estimator. One embodiment selects a virtual image in a virtual dataset, the virtual dataset including a plurality of computer-generated virtual images; generates, from the virtual image in accordance with virtual-camera intrinsics, a point cloud in three-dimensional space based on ground-truth depth information associated with the virtual image; reprojects the point cloud back to two-dimensional image space in accordance with real-world camera intrinsics to generate a transformed virtual image; and trains the machine-learning-based monocular depth estimator, at least in part, using the transformed virtual image.