Depth Image Generation Using Multi-Scale Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth image generation methods rely on separate depth sensors, which are costly, limited in measurement distance, prone to errors, and vulnerable to external light, and often produce low-quality depth images with noise, affecting the accuracy of depth information.
Innovation Solution
A multi-scale-based depth image generation method using deep learning models processes color or infrared images to generate high-resolution depth images without relying on separate depth sensors, by decomposing images into frequency components and refining depth information through neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a depth camera is used to acquire depth information, then depth measurement capability is obtained, but the device cost increases and measurement distance is limited
Solution Approach 1:
The patent applies multi-functionality by enabling ordinary image sensors (designed for 2D imaging) to perform depth measurement functions through computational photography techniques. The system uses multiple ordinary cameras or image sensors to achieve depth mapping capabilities traditionally requiring specialized depth cameras, thereby reducing device cost while maintaining depth measurement capability.
Solution Approach 2:
The patent replaces mechanical/optical depth measurement systems (such as time-of-flight sensors or structured light cameras) with a computational approach using standard image sensors and algorithms. Instead of relying on specialized hardware mechanisms for depth acquisition, the system uses image processing and computational methods to infer depth information from ordinary images.
2Reliability
If a depth camera is used to acquire depth information, then depth measurement capability is obtained, but the system becomes vulnerable to external light and prone to errors
Solution Approach 1:
The patent replaces optical depth measurement systems that are sensitive to external light with a computational photography approach. By using multiple ordinary image sensors and processing techniques such as focus variation or stereo matching, the system achieves depth measurement without relying on optical mechanisms that are vulnerable to external light interference.
Solution Approach 2:
The patent introduces computational algorithms as an intermediary between the image sensors and depth output. Instead of directly measuring depth through optical means that are susceptible to external light, the system uses intermediate computational processing of images from multiple sensors to derive depth information, thereby isolating the measurement process from external light interference.
3Measurement precision
If depth information is acquired using traditional methods, then depth data is obtained, but the quality is low with noise affecting accuracy
Solution Approach 1:
The patent combines multiple images or image data sources through merging techniques to produce a single high-quality depth map. By integrating information from multiple ordinary image sensors or multiple images taken under different conditions, the system achieves higher depth measurement precision and reduces noise compared to single-sensor approaches.
Solution Approach 2:
The patent employs feedback mechanisms in the form of iterative optimization algorithms that refine depth estimates by comparing multiple images or measurements. The system uses feedback from image matching errors, focus metrics, or disparity calculations to continuously improve depth accuracy and reduce noise in the final depth output.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A method for depth image generation may include: receiving an input image; generating a first low-resolution image having a resolution lower than a resolution of the input image; acquiring a first depth residual image corresponding to the input image by using a first generation model based on a first neural network; generating a first low-resolution depth image corresponding to the first low-resolution image by using a second generation model based on a second neural network; and generating a target depth image corresponding to the input image, based on the first depth residual image and the first low-resolution depth image.