Monocular Depth Estimation via Flipped Image Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Monocular cameras face challenges in estimating depth due to limited resolution, image artifacts, and the need for expensive or limited training data, which can hinder a robot's ability to navigate and develop comprehensive awareness of its environment.
Innovation Solution
A self-supervised, super-resolved, differentiable flip network architecture is implemented, incorporating a disparity model with a differentiable flip augmentation layer and subpixel convolutional layers to process monocular images twice, generating a fused disparity map that avoids artifacts and improves resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If monocular cameras are used for depth estimation, then cost is reduced, but measurement precision deteriorates due to depth ambiguities and limited resolution
Solution Approach 1:
The patent introduces a temporal dimension by processing the same monocular image through multiple processing paths (original and flipped versions) and fusing their disparity maps. This transforms a single 2D image processing task into a multi-dimensional process that combines spatial information from both the original and horizontally flipped images, thereby improving depth estimation precision without changing the monocular camera hardware.
Solution Approach 2:
The patent creates a flipped copy of the original monocular image and processes both copies through the disparity model. By generating and fusing disparity maps from both the original image and its flipped counterpart, the system effectively uses redundant information to improve depth estimation accuracy while maintaining cost-effectiveness of monocular cameras.
2Device complexity
If monocular images are processed to derive depth information, then device complexity is reduced, but reliability deteriorates due to image artifacts and depth ambiguities
Solution Approach 1:
The patent implements a feedback mechanism where the disparity model processes both the original image and its flipped version, then fuses the resulting disparity maps. This fusion process provides feedback that helps eliminate artifacts and resolve depth ambiguities, thereby improving the reliability of depth estimation while keeping the sensor system simple (monocular camera only).
Solution Approach 2:
The patent combines disparity maps from two different processing paths (original and flipped images) into a fused disparity map. This composite approach integrates multiple sources of depth information, creating a more reliable and artifact-free depth estimation result while maintaining device simplicity.
3Loss of time
If traditional disparity models process monocular images, then processing time is reduced, but measurement precision deteriorates due to artifacts along boundaries
Solution Approach 1:
The patent segments the processing task into two parallel paths: one processing the original monocular image and another processing the flipped image. Each path generates its own disparity map, and these segmented results are then fused together. This segmentation allows the system to maintain efficient processing while improving precision by combining multiple perspectives.
Solution Approach 2:
By adding the flipped image processing path, the system introduces an additional processing dimension without significantly increasing overall processing time. The fusion of disparity maps from both paths improves boundary quality and reduces artifacts, achieving better measurement precision while maintaining reasonable processing efficiency.
Data Source
AI summary
System, methods, and other embodiments described herein relate to generating depth estimates from a monocular image. In one embodiment, a method includes, in response to receiving the monocular image, flipping, by a disparity model, the monocular image to generate a flipped image. The disparity model is a machine learning algorithm. The method includes analyzing, using the disparity model, the monocular image and the flipped image to generate disparity maps including a monocular disparity map corresponding to the monocular image and a flipped disparity map corresponding with the flipped image. The method includes generating, in the disparity model, a fused disparity map from the monocular disparity map and the flipped disparity map. The method includes providing the fused disparity map as the depth estimates of objects represented in the monocular image.