Depth Map Generation Using Previous-Frame Features for Faster Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing monocular/stereo depth map extraction methods fail to utilize previous frame information, leading to inefficient processing and increased calculation time and cost, especially in real-time applications.
Innovation Solution
A method that generates a depth map of a current frame by using a depth map of a previous frame based on continuity between frames, involving feature map extraction, warping, and up-scaling, utilizing optical flow and neural networks for efficient depth estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth map is generated by extracting feature pyramids of various resolutions from current frame image only, then depth extraction can be performed independently for each frame, but processing time and calculation cost increase significantly
Solution Approach 1:
The patent performs preliminary actions by extracting and storing feature pyramids from previous frames before they are needed for depth estimation. When processing the current frame, these pre-extracted features from previous frames are readily available, eliminating the need to re-extract them and significantly reducing processing time while maintaining independent frame processing capability
Solution Approach 2:
The patent creates copies of feature pyramids from previous frames and uses these copied features in conjunction with current frame features for depth estimation. This copying approach allows the system to leverage historical feature information without重新 processing, reducing calculation cost while maintaining processing independence
2Measurement precision
If all processing steps are performed from the first step when a current frame comes in, then depth map can be generated without relying on previous frame information, but calculation cost and processing time increase
Solution Approach 1:
The patent merges feature pyramids from previous frames with feature pyramids from the current frame to create a combined feature representation. This merging allows the system to utilize both historical context and current information, improving depth map accuracy while reducing the computational burden of processing only current frame data from scratch
Solution Approach 2:
The patent maintains continuity of useful action by continuously utilizing feature information from previous frames in the depth estimation process. Instead of restarting processing for each frame, the system continuously incorporates historical features, improving both accuracy and processing efficiency through sustained utilization of available information
3Manufacturing precision
If feature pyramids of various resolutions are extracted for each frame, then depth map can be generated with sufficient detail, but calculation cost increases
Solution Approach 1:
The patent performs preliminary extraction of feature pyramids at multiple resolutions from previous frames and stores them for later use. This preliminary action eliminates the need to re-extract these computationally expensive multi-resolution features for each frame, significantly reducing calculation energy while maintaining the ability to generate high-resolution depth maps
Solution Approach 2:
The patent creates copies of multi-resolution feature pyramids from previous frames and uses these copies instead of re-extracting them. This copying approach preserves the detailed feature information needed for high-resolution depth maps while avoiding the energy-intensive extraction process, reducing calculation energy consumption
Data Source
AI summary
Provided is a depth map generation method using previous frame information. A depth map generation method according to an embodiment of the disclosure extracts a feature map of a 1/n scale resolution on a stereo image of a current frame, and generates a depth map of a 1/n scale resolution of the current frame by using a depth map of a 1/n scale resolution on a stereo image of a previous frame, and the extracted feature map. Accordingly, a depth map of a current frame is generated by using a depth map of a previous frame based on continuity of information between continuous frames of a moving image, so that efficient use of resources, reduction of a calculation response time, enhancement of accuracy are possible.