Neural Depth Estimation With Sparse Disparity Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth estimation methods using neural networks are less effective in environments different from their training environments and are computationally onerous, requiring extensive training for each iteration.
Innovation Solution
A method and system that utilizes a self-adaptive convolutional neural network trained with sparse depth data from a stereovision algorithm, allowing it to adapt to new environments and reduce computational load by updating only specific portions of the network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a neural network is trained extensively for each iteration to improve depth estimation accuracy, then measurement precision is improved, but productivity deteriorates due to extensive training requirements
Solution Approach 1:
The patent segments the neural network training process by identifying and updating only specific portions (layers or parameters) of the network that are most relevant to the current depth estimation task, rather than retraining the entire network. This segmentation allows for faster iteration while maintaining accuracy improvements.
Solution Approach 2:
The patent applies partial action by performing incomplete or selective training updates - specifically updating only certain network parameters or layers based on sparse depth data, rather than performing full network training. This partial update approach achieves sufficient accuracy improvement without the computational cost of complete retraining.
2Adaptability or versatility
If a neural network is trained with extensive data for each iteration to adapt to new environments, then adaptability is improved, but loss of time increases due to extensive training
Solution Approach 1:
The patent performs preliminary action by pre-training the neural network on a diverse dataset covering multiple environments before deployment. This preliminary training provides a robust foundation that reduces the need for extensive retraining when adapting to new environments, allowing faster adaptation with fewer iterations.
Solution Approach 2:
The patent segments the adaptation process by identifying which specific network components need updating for new environments and only retraining those segments, rather than retraining the entire network. This enables environmental adaptation with significantly reduced training time.
3Reliability
If traditional stereomatching algorithms are used to generate disparity maps, then reliability is maintained, but measurement precision deteriorates compared to neural network approaches
Solution Approach 1:
The patent merges traditional stereomatching algorithms with neural network approaches in a hybrid system. The traditional algorithm provides reliable initial disparity estimates, while the neural network refines these estimates to achieve higher precision. This combination leverages the strengths of both approaches.
Solution Approach 2:
The patent uses the traditional stereomatching algorithm as an intermediary to generate initial depth information that serves as training data or guidance for the neural network. This intermediary step provides reliable starting points that improve the neural network's learning efficiency and final precision.
Data Source
Figure 1~2
Figure 3~5
AI summary
The present invention relates to a Method to determine the depth of a scene (I) from at least one digital image (R, T) of said scene (I), comprising the following steps: A. acquiring said at least one digital image (R, T) of said scene (I); B. calculating a first disparity map (DM1) from said at least one digital image (R, T), wherein said first disparity map (DM1) consists of a matrix of pixels p ij with i=1,..,M and j=1 where i and j indicate respectively the line and column index of said first disparity map (DM1), and M and N are positive integers; C. calculating a second disparity map (DM2), by a neural network, from said at least one digital image (R, T); D. selecting a plurality of sparse depth data S ij relative to the respective pixel p ij of said first disparity map (DM1); E. extracting said plurality of sparse depth data S ij from said first disparity map (DM1); and F. optimizing said second disparity map (DM2) by said plurality of sparse depth data S ij , training at least one portion of said neural network with the information relative to said depth of said scene (I) associated with said sparse depth data S ij . The present invention also relates to a system (S) for determining the depth of a scene (I) starting from at least one digital image (R, T) of said scene (I).