Monocular Depth Model Trained with Sparse LiDAR Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems face challenges in accurately determining depth from monocular images due to limited resolution, image artifacts, and the expense or unavailability of training data, leading to reduced situational awareness and navigation difficulties for robotic devices.
Innovation Solution
A semi-supervised training approach is employed, comprising a self-supervised stage using monocular video and a weakly supervised stage with sparse LiDAR data, which refines the depth model to produce metrically accurate depth estimates without extensive annotated training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If monocular cameras are used for depth estimation, then cost is reduced, but measurement precision deteriorates
Solution Approach 1:
The system performs preliminary action by using a LiDAR sensor to capture depth information before the monocular camera captures the image. This pre-acquired depth data is then used to guide the training process of the neural network, enabling the camera to achieve accurate depth estimation without requiring expensive stereo cameras or extensive manual annotation.
Solution Approach 2:
The system implements self-service through self-supervised learning where the LiDAR depth data automatically supervises the training of the monocular depth estimation model without requiring manual annotation. The depth information from LiDAR serves as ground truth to train the camera-based model, making the system self-training and eliminating the need for expensive labeled datasets.
2Manufacturing precision
If extensive annotated training data is used, then manufacturing precision improves, but loss of substance increases
Solution Approach 1:
The system uses self-supervised learning where the LiDAR depth data automatically provides supervision signals for training the monocular depth estimation model. This eliminates the need for extensive manually annotated training data, as the model learns from the automatically acquired depth information, significantly reducing annotation costs while maintaining high training accuracy.
Solution Approach 2:
The LiDAR depth data acts as an intermediary that bridges the gap between monocular images and depth information. Instead of requiring manual annotation of depth data, the LiDAR provides automatic depth measurements that serve as training targets, enabling the model to learn accurate depth estimation without extensive human-labeled datasets.
3Measurement precision
If LiDAR sensors are used for depth perception, then measurement precision improves, but device complexity increases
Solution Approach 1:
The LiDAR sensor performs preliminary depth measurement to provide training data for the monocular camera model. After the model is trained using this preliminary depth information, the LiDAR sensor is no longer needed for depth estimation during operation. This allows the system to achieve high measurement precision temporarily during training, then operate with the simpler monocular camera alone.
Data Source
AI summary
System, methods, and other embodiments described herein relate to semi-supervised training of a depth model for monocular depth estimation. In one embodiment, a method includes training the depth model according to a first stage that is self-supervised and that includes using first training data that comprises pairs of training images. Respective ones of the pairs including separate frames depicting a scene of a monocular video. The method includes training the depth model according to a second stage that is weakly supervised and that includes using second training data to produce depth maps according to the depth model. The second training data comprising individual images with corresponding sparse depth data. The second training data providing for updating the depth model according to second stage loss values that are based, at least in part, on the depth maps and the depth data.