Depth Feature Warping Before Decoding for Monocular Depth Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth estimation methods using monocular images suffer from aberrations such as missing edges and duplicated features due to direct warping of depth values, leading to noise in the training signal.
Innovation Solution
Warping depth features prior to decoding using a pose model to transform depth features into a target image frame, avoiding direct warping of depth values and improving the accuracy of the resulting depth map.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth values are directly warped from source image to target image frame, then depth estimation can be performed, but aberrations such as missing edges and duplicated features occur leading to noise in training signal
Solution Approach 1:
The patent applies preliminary action by warping depth features before decoding them into depth values. The encoder first extracts depth features from the source image, these features are then warped to the target image frame using the transformation model, and only after warping are the features decoded into depth values. This preliminary warping of features rather than final depth values prevents the aberrations that would otherwise occur during the decoding stage, thereby maintaining both accuracy and training signal quality.
2Ease of manufacture
If monocular cameras are used for cost-effective depth acquisition, then device cost is reduced, but depth information must be derived through processing routines rather than direct sensing
Solution Approach 1:
The patent introduces an intermediary approach by using a transformation model that learns the mapping between source and target image frames. Instead of directly deriving depth from single monocular images, the system uses the transformation model to warp depth features from a source frame to a target frame, where the warped features serve as an intermediary representation that captures depth information more accurately than direct monocular estimation.
3Adaptability or versatility
If self-supervised training is performed by minimizing photometric loss with warped depth estimates, then depth model training can proceed without ground truth, but warping causes loss of border areas and duplication of features
Solution Approach 1:
The patent applies preliminary action by performing the warping operation on depth features before they are decoded into final depth values. The encoder extracts depth features from the source image, these features are warped to the target image frame using the transformation model, and only then are they decoded. This preliminary warping of features rather than final depth values prevents the border loss and feature duplication that would otherwise occur during the decoding stage, thereby maintaining both training flexibility and depth map quality.
Data Source
AI summary
System, methods, and other embodiments described herein relate to an improved approach to training a depth model for monocular depth estimation by warping depth features prior to decoding. In one embodiment, a method includes encoding, using an encoder of a depth model, a source image into depth features of a scene depicted by the source image. The method includes warping the depth features into warped features of a target frame of a target image associated with the source image. The method includes decoding, using a decoder of the depth model, the warped features into a depth map. The method includes training the depth model according to a loss derived from the depth map.