Camera Motion Estimation for Dense 3D Model Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for estimating camera motion and determining a three-dimensional model of an environment in real-time, especially in unknown scenes, require manual measurements, known objects, or constrained camera motions, leading to sparse online feature maps and limitations in occlusion handling and AR applications.
Innovation Solution
A method that estimates camera motion and builds a three-dimensional model by using image registration and similarity measures to integrate current intensity images and depth samples into a reference set, allowing simultaneous tracking and reconstruction without the need for manual measurements or known objects, and enabling occlusion handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual measurements or known objects are used to estimate camera motion, then correctly scaled 3D coordinates can be obtained, but the system complexity and ease of operation deteriorate due to requiring pre-measurement or known objects in the environment
Solution Approach 1:
The system performs self-calibration by automatically determining scaling factors from visual features in the environment without requiring manual measurements or pre-placed known objects. The camera motion estimation and 3D reconstruction process itself generates the scaling information needed, making the system self-sufficient and eliminating the need for external calibration artifacts.
Solution Approach 2:
The invention changes the approach from using fixed known objects to dynamically computing scaling parameters from the visual data itself. By deriving scale information from the structure-from-motion reconstruction and comparing it with the known camera baseline, the system adapts the scaling parameter automatically based on the captured images and feature correspondences.
2Measurement precision
If visual features are tracked over multiple frames for point reconstruction, then 3D model accuracy improves, but the response time and productivity deteriorate due to delayed participation of newly visible points in camera motion estimation
Solution Approach 1:
The system performs preliminary camera motion estimation using available visual features before complete point tracking is finished. By estimating camera pose from partial feature correspondences and then using this pose to reconstruct points, the system enables newly visible points to participate in motion estimation sooner, reducing the delay while maintaining accuracy through iterative refinement.
3Measurement precision
If constrained camera motion between two known frames is used, then correct scale estimation is achieved, but the adaptability and ease of operation worsen due to requiring user experience and constrained motion patterns
Solution Approach 1:
The system eliminates the need for constrained camera motion by automatically determining the scaling factor from the visual features and camera baseline. The camera can move freely while the system computes the appropriate scale from the geometric relationships in the captured images, making the method adaptable to various camera trajectories without requiring user expertise in constrained motion patterns.
4Reliability
If dense representation is achieved using stereo-dense matching and GPU, then occlusion handling capability improves, but the device complexity and cost increase due to requiring stereo cameras and GPU hardware
Solution Approach 1:
The invention replaces the need for complex stereo camera hardware and GPU-based dense matching with a computational approach using standard camera footage. By leveraging structure-from-motion algorithms and visual features from monocular or simple stereo cameras, the system achieves dense 3D reconstruction and occlusion handling through software processing rather than requiring specialized hardware components.
Data Source
Figure 1a~1b
Figure 2a
Figure 2b
AI summary
A method for estimating a camera motion and for determining a three-dimensional model of an environment, comprising the steps of: a. providing intrinsic parameters of a camera; b. providing a set of reference two-dimensional imaged points captured by the camera at a first camera pose and reference depth samples associated to the set of reference two-dimensional imaged points; c. determining a three-dimensional model of the environment using the reference depth samples and the intrinsic parameters; d. providing a set of current two-dimensional imaged points captured by the camera at a second camera pose and current depth samples associated to the set of current two-dimensional imaged points and determining a current three-dimensional model using the current depth samples and the intrinsic parameters; e. estimating a camera motion between the first camera pose and the second camera pose by means of an image registration method; f. determining based on the estimated camera motion a similarity measure between the three-dimensional model and the current three-dimensional model, and if it is determined that the similarity measure meets a first condition, updating the three-dimensional model of the environment using at least one point of the current three-dimensional model and adding the set of current two-dimensional imaged points to the set of reference two-dimensional imaged points.