Photometric 3D Modeling with Joint Camera and Structure Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for reconstructing 3D object geometry from 2D images are limited by assumptions such as fixed camera parameters, constant brightness, and high resource consumption, leading to inaccurate and inefficient modeling, especially under varying lighting conditions.
Innovation Solution
The approach jointly optimizes structure, camera, and lens distortion parameters using a photometric method that accounts for variable lighting conditions, reducing memory requirements and improving accuracy by solving an optimization problem that relates pixels between source and target images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex algorithms such as multi-view stereo (MVS) are used to construct dense geometry, then measurement precision of 3D geometry is improved, but device complexity and resource consumption increase
Solution Approach 1:
The patent changes the fundamental parameters of the reconstruction approach by using photometric error minimization instead of traditional geometric feature matching. This involves optimizing pixel intensity values across multiple views under the assumption of constant brightness, thereby simplifying the algorithm while maintaining reconstruction accuracy through direct photometric constraints rather than complex geometric computations
2Device complexity
If traditional SfM methods with fixed camera parameters are used, then device complexity is reduced, but manufacturing precision of 3D model deteriorates under varying lighting conditions
Solution Approach 1:
The patent applies dynamics by allowing camera parameters to be optimized rather than fixed, enabling the system to adapt to varying lighting conditions. The photometric optimization process dynamically adjusts camera extrinsic parameters to minimize pixel intensity differences across views, thereby maintaining 3D model accuracy in non-stationary lighting environments
3Adaptability or versatility
If photometric method with variable lighting conditions is used, then adaptability to different environments is improved, but measurement precision may deteriorate due to lighting variations
Solution Approach 1:
The patent converts the harmful effect of varying lighting conditions into a beneficial optimization target. By formulating the constant brightness assumption as a photometric error minimization problem, the method uses lighting variations as constraints to optimize camera parameters and achieve accurate pixel correspondences, thereby transforming an adverse factor into a useful computational guide
Data Source
AI summary
Aspects of the present disclosure involve a system and a method for performing operations comprising: accessing a source image depicting a target structure; accessing one or more target images depicting at least a portion of the target structure; computing correspondence between a first set of pixels in the source image of a first portion of the target structure and a second set of pixels in the one or more target images of the first portion of the target structure, the correspondence being computed as a function of camera parameters that vary between the source image and the one or more target images; and generating a three-dimensional (3D) model of the target structure based on the correspondence between the first set of pixels in the source image and the second set of pixels in the one or more target images based on a joint optimization of target structure and camera parameters.