Monocular Depth Estimation via Polygon Mesh Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for estimating depth values in mobile devices using monocular cameras require additional hardware like depth cameras or two cameras, and face challenges with limited computational resources, making real-time processing difficult, especially in small mobile devices.
Innovation Solution
A processor-implemented image processing method that estimates depth values by generating a polygon mesh using feature points with depth values, predicting 3D candidate coordinates, and setting a search range for corresponding pixels based on these coordinates, allowing for depth estimation without additional hardware and efficient real-time processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If additional hardware devices such as depth cameras or two cameras are used for depth estimation, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The monocular camera performs multiple functions: capturing 2D images and providing depth information through computational algorithms. The system uses a single camera that can operate in both 2D imaging mode and depth estimation mode by processing image sequences with SFM and polygon mesh techniques, eliminating the need for dedicated depth sensing hardware.
Solution Approach 2:
The system creates a virtual depth map by computationally reconstructing 3D information from 2D image sequences. Instead of directly measuring depth with specialized hardware, the patent copies depth information from multiple 2D views through structure-from-motion algorithms and polygon mesh generation, achieving depth estimation without physical depth sensors.
2Measurement precision
If SLAM scheme is used for depth estimation, then measurement precision is improved, but productivity decreases due to limited computational resources
Solution Approach 1:
The depth estimation process is divided into distinct stages: feature point detection, polygon mesh generation, and depth map creation. Each stage processes specific aspects of the image data independently, allowing for optimized computation at each step and enabling real-time performance on mobile devices with limited resources.
Solution Approach 2:
The system performs preliminary structure-from-motion processing to generate a polygon mesh before creating the final depth map. This preliminary 3D reconstruction provides a framework that accelerates subsequent depth estimation, as the mesh structure pre-organizes spatial relationships and reduces computational complexity during real-time depth map generation.
3Measurement precision
If two cameras are used for stereo vision, then measurement precision is improved, but device complexity increases due to baseline distance requirements
Solution Approach 1:
Instead of using spatial separation (baseline distance) between two cameras to achieve depth, the patent transitions to temporal separation by capturing image sequences over time with a single camera. The system exploits motion parallax and temporal changes in viewpoint to infer depth, replacing the spatial dimension with temporal dimension for depth acquisition.
Solution Approach 2:
The patent introduces computational algorithms as an intermediary between the single camera and depth estimation. Rather than directly measuring depth through physical camera separation, the system uses SFM algorithms, polygon mesh generation, and epipolar geometry computations as mediators to derive depth information from monocular image sequences.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
An image processing method includes: selecting frames from an input image sequence, wherein the selected frames include a first frame and a second frame; generating a polygon mesh including polygons for each of the selected frames using feature points having depth values, among feature points of each of the selected frames; setting a search range of a corresponding pixel of the second frame corresponding to a pixel of the first frame based on the polygon mesh; and estimating a depth value of the pixel of the first frame by matching the pixel of the first frame and the corresponding pixel based on the search range.