Focus Convolutional Neural Network for Depth Map Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for estimating depth of field information from images, such as those used in mobile phone cameras, often produce poor quality depth maps, especially in boundary regions, and require complex camera systems or lengthy capturing times, with optimization processes being slow and computationally costly.
Innovation Solution
The development of focus and stereo convolutional neural networks (Focus-Net and Stereo-Net) to process focal stacks and stereo images, respectively, for efficient and accurate depth information extraction, combining depth from focus and disparity cues, and integrating these networks into a unified solution (BDfF-Net) to generate high-quality depth maps quickly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If optimization algorithms (graph-cut, belief propagation) are used to extract depth from focal stack, then measurement precision of depth information is improved, but productivity is deteriorated due to slow processing speed
Solution Approach 1:
The patent replaces traditional optimization algorithms (graph-cut, belief propagation) with a deep learning-based convolutional neural network (Focus-Net). This substitution transitions from iterative mechanical optimization to direct neural network inference, achieving both high precision depth estimation and real-time processing speed required for mobile applications.
2Measurement precision
If dual-camera system is used to capture binocular focal stack, then measurement precision of depth information is improved, but device complexity is increased
Solution Approach 1:
The patent introduces an intermediary processing system (Focus-Net convolutional neural network) that takes binocular focal stack as input and produces high-quality depth maps. This intermediary layer enables the system to leverage dual-camera depth cues while maintaining a unified, manageable processing architecture that doesn't linearly increase complexity with additional sensors.
3Measurement precision
If traditional stereo matching algorithms are used, then measurement precision of depth from disparity is improved, but loss of time is increased due to computational cost
Solution Approach 1:
The patent merges depth from focus (single-camera focal stack) and depth from disparity (binocular stereo) into a unified deep learning model (BDfF-Net). This combination allows the system to simultaneously leverage both depth cues through a single processing pipeline, achieving high precision while reducing total computational time compared to sequential processing of separate algorithms.
Data Source
AI summary
A method and apparatus for extracting depth information from a focal stack is disclosed. The method may include processing the focal stack through a focus convolutional neural network (Focus-Net) to generate a plurality of feature maps, stacking the plurality of feature maps together, and fusing the plurality of feature maps by a plurality of first convolutional layers to obtain a depth image. The Focus-Net includes a plurality of branches, and each branch includes a downsampling convolutional layer having a different stride for downsampling the focal stack and a deconvolutional layer for upsampling the focal stack.