Unsupervised Stereo Matching Using Surface Normal Assistance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing stereo matching algorithms for indoor applications face challenges in accurately predicting disparity in textureless regions due to the lack of ground truth disparity data and the difficulty in adapting outdoor-trained methods to indoor environments.
Innovation Solution
A deep neural network with a feature extraction module, a normal branch, and a disparity branch is proposed, where the feature extraction and normal branch are trained supervisedly, and the disparity branch is trained unsupervisedly, utilizing surface normal predictions to improve disparity estimation accuracy in indoor scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If learning-based stereo matching methods are used to achieve high accuracy, then disparity estimation accuracy is improved, but the need for large amounts of ground truth disparity data increases
Solution Approach 1:
The patent introduces surface normal information as an intermediary to bridge the gap between stereo images and disparity maps. The surface normal prediction network provides geometric constraints that guide the disparity estimation process, enabling accurate depth prediction without requiring large amounts of ground truth disparity data for training
Solution Approach 2:
The system segments the stereo matching problem into two independent components: surface normal prediction and disparity estimation. This segmentation allows each component to be trained separately with appropriate supervision signals, reducing the overall dependency on ground truth disparity data while maintaining high accuracy
2Adaptability or versatility
If outdoor-trained stereo matching methods are applied to indoor scenarios, then general-purpose depth perception is achieved, but accuracy deteriorates in textureless indoor regions
Solution Approach 1:
The patent applies local quality by making the stereo matching system adapt to local characteristics of indoor environments. The surface normal prediction network learns environment-specific geometric patterns that are particularly effective in textureless indoor regions, allowing the system to optimize performance for specific应用场景 rather than relying on generic outdoor training data
3Measurement precision
If supervised learning is used for disparity estimation, then accuracy is improved, but the difficulty of collecting ground truth data increases
Solution Approach 1:
The system employs self-service through unsupervised learning for disparity estimation, where the model learns to predict disparity without human-annotated ground truth data. The surface normal predictions serve as self-generated guidance that enables the system to train and improve automatically, eliminating the need for manual data collection and annotation
Data Source
AI summary
A system and method for unsupervised stereo matching with surface normal assistance for indoor applications. According to the disclosure, a deep neural network with a feature extraction module, a normal branch, and a disparity branch is disclosed. The extraction module and the normal branch are trained first in a supervised manner for surface normal prediction. The predicted surface normal is then incorporated into the disparity branch, which is trained later in an unsupervised manner for disparity estimation. The latter unsupervised learning approach can reduce our method's dependence on a large amount of ground truth data that is difficult to collect. Experimental results indicate that our proposed method can predict accurate surface normal at textureless regions. With the help of the surface normal, the predicted disparity at these challenging areas is more accurate, which leads to improved quality of stereo matching in indoor scenarios.