2D Image Depth Estimation With Temporal Frame Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for obtaining depth information from 2D images, such as using multiple cameras or lidar sensors, are costly and require significant data preparation, while monocular depth estimation using deep neural networks suffer from accuracy issues.
Innovation Solution
An electronic apparatus and method that processes 2D images using a processor to calculate image difference values and apply weights based on these values and thresholds to generate accurate depth information, employing IIR and HR filtering to refine the results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple 2D cameras or lidar sensors are used to obtain depth information, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent combines temporal information from multiple image frames with spatial information from a single 2D image. By merging depth information from previous frames with current frame analysis, the system achieves multi-camera level accuracy using only one camera, thus resolving the contradiction between measurement precision and device complexity
Solution Approach 2:
The system performs preliminary depth estimation on previous image frames and stores this depth information for later use. This preliminary action allows the current frame processing to leverage pre-computed depth data, improving overall measurement precision while avoiding the need for multiple simultaneous cameras
2Device complexity
If monocular depth estimation using deep neural networks is used, then device complexity is reduced, but measurement precision deteriorates
Solution Approach 1:
The patent maintains continuous depth estimation across multiple consecutive image frames by leveraging temporal continuity. The system continuously refines depth information by comparing sequential frames and applying temporal filtering, which compensates for the inherent limitations of monocular estimation while keeping the system simple
Solution Approach 2:
The system implements feedback mechanisms where depth information from previous frames is used to guide and refine current frame depth estimation. The temporal filtering and weight adjustment based on image differences create a feedback loop that continuously improves measurement precision without adding hardware complexity
3Measurement precision
If depth information from multiple frames is combined, then measurement precision is improved, but loss of information increases due to temporal variations
Solution Approach 1:
The patent dynamically adjusts the weighting of depth information from different time points based on image difference values. When temporal variations are detected (high image difference), the system reduces the weight of historical depth data. This dynamic adaptation prevents information loss from temporal variations while maintaining precision when conditions are stable
Solution Approach 2:
The system changes the parameter of weight allocation based on temporal conditions. By adjusting weights according to image difference thresholds and temporal filtering parameters, the system optimizes the balance between leveraging historical data for precision and avoiding information loss from temporal changes
Data Source
AI summary
An electronic apparatus includes a memory and a processor that obtains second depth information of a second image frame subsequent to the first image frame, obtains an image difference value between the first image frame and the second image frame, obtains final depth information corresponding to a second image frame by applying a first weight and a second weight to a first depth information and a second depth information, respectively, and generates an image related to the second image frame based on the obtained final depth information.