Self-Supervised Ego Vehicle Training for LiDAR-Free Scaled Depth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing self-supervised monocular depth prediction methods for autonomous vehicles fail to accurately determine the scale of a scene due to reliance on temporal image consistency, and fully-supervised methods require identical camera setups and LiDAR sensors for optimal performance, making them impractical for vehicles without LiDAR.
Innovation Solution
A system and method for training a self-supervised machine learning framework using a first trained network with LiDAR supervision and a second untrained network, where the second network is adjusted based on comparisons and loss functions to predict scaled depth from monocular images without LiDAR, utilizing knowledge distillation from a teacher network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If self-supervised monocular depth prediction methods are used, then the system can operate without LiDAR sensors, but the predicted depth lies on an arbitrary scale and cannot learn the actual scale of the scene
Solution Approach 1:
The patent introduces an intermediary scale estimation module that uses monocular cues (object size priors, perspective geometry, motion parallax) to estimate the scale of the scene. This intermediary component bridges the gap between 2D image data and 3D depth prediction, allowing the system to recover actual scale information without requiring LiDAR sensors during operation.
Solution Approach 2:
The patent performs preliminary scale estimation using monocular cues before final depth prediction. By first estimating the scale of the scene from 2D image characteristics and then using this scale information to guide the depth prediction process, the system recovers accurate scaled depth values without requiring active sensors at test time.
2Measurement precision
If fully-supervised methods trained using LiDAR data are used, then depth prediction performance is improved, but the system requires identical camera setups and LiDAR sensors for optimal performance
Solution Approach 1:
The patent segments the depth prediction task into two independent components: (1) a scale estimation module that operates independently using monocular cues, and (2) a depth prediction module that uses the estimated scale. This segmentation allows each module to be trained and optimized independently, enabling the system to adapt to different camera setups without requiring LiDAR sensors during deployment.
Solution Approach 2:
The patent introduces an intermediary scale estimation module that decouples the depth prediction task from LiDAR dependency. This intermediary component estimates scene scale from monocular cues and provides it as input to the depth prediction module, allowing the system to achieve LiDAR-level accuracy without requiring LiDAR sensors at test time or identical camera setups.
3Measurement precision
If stereo cameras are used for depth estimation, then more accurate depth prediction is achieved, but the stereo rig requires continuous auto-calibration which is computationally expensive and incurs high maintenance costs
Solution Approach 1:
The patent extracts the scale estimation function from the depth prediction process and implements it as a separate monocular module. By taking out the scale estimation task and solving it independently using 2D image cues, the system eliminates the need for complex stereo calibration hardware and continuous auto-calibration processes, achieving accurate depth prediction with a single camera.
Data Source
AI summary
A system for training a machine learning framework to estimate depths of objects captured in 2-D images includes a first trained machine learning network and a second untrained or minimally trained machine learning framework. The first trained machine learning network is configured to analyze 2-D images of target spaces including target objects and to provide output indicative of 3-D positions of the target objects in the target spaces. The second machine learning network can be configured to provide an output responsive to receiving a 2-D input image. A comparator receives the outputs from the first and second machine learning networks based on a particular 2-D image. The comparator compares the output of the first trained machine learning network with the output of the second machine learning network. A feedback mechanism is operative to alter the second machine learning network based at least in part on the output of the comparator.