Monocular Depth Meshing for Dense Mobile 3D Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for generating 3D images and measuring depth require stereoscopic cameras or lidar sensors, which are not available on all devices, leading to sparse depth outputs and high computational complexity, making them unsuitable for mobile devices with limited compute power.
Innovation Solution
A depth estimation system using a monocular camera that includes image feature identification, pose determination, and a machine-learned model to generate dense and accurate depth estimates, refining the depth using a point cloud and SLAM, and creating a mesh model for real-time display.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional systems use stereoscopic cameras or lidar sensors to generate 3D images and measure depth, then measurement precision is improved, but device complexity increases and computational power requirements increase
Solution Approach 1:
The patent uses a monocular camera to capture 2D images and employs machine learning models to generate depth information that copies or simulates the functionality of stereoscopic cameras and lidar sensors. The system creates a virtual representation of depth data from a single image, avoiding the need for complex hardware while achieving comparable measurement precision through computational methods
2Measurement precision
If conventional systems use stereoscopic cameras or lidar sensors to generate 3D images, then measurement precision is improved, but computational power requirements increase
Solution Approach 1:
The patent replaces complex hardware systems (stereoscopic cameras, lidar sensors) with a monocular camera combined with machine learning models. The computational tasks are performed through software algorithms that process single images to generate depth information, substituting mechanical/optical complexity with computational intelligence that can run on mobile devices with limited processing power
3Device complexity
If monocular cameras are used with machine learning models to generate depth estimation, then device complexity is reduced, but measurement precision may deteriorate due to sparse depth outputs
Solution Approach 1:
The patent changes the approach to depth estimation by using machine learning models that have been trained on large datasets to predict depth values from monocular images. The system transforms the problem from geometric computation to pattern recognition, where the model learns to map 2D image features to 3D depth information, achieving dense and accurate depth maps without complex hardware
Data Source
AI summary
A depth estimation system to perform operations that include: receiving image data generated by a client device, the image data comprising a depiction of an environment; identifying a set of image features based on the image data; determining a pose of the client device based on the set of features; generating a depth estimation based on the image data and the pose of the client device; and generating a mesh model of the environment based on the depth estimation.