Joint Camera Calibration With Bundle Adjustment for 3D Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented and mixed reality devices face inaccuracies in location data due to miscalibration of multiple sensors, such as cameras, which affect the accuracy of environment mapping and content presentation.
Innovation Solution
A method for concurrent camera calibration and bundle adjustment that determines calibration data during the bundle adjustment process, using image data clusters and proximity operators to improve the accuracy of spatial relationships between cameras, thereby enhancing the precision of 3D model updates and device trajectory determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If bundle adjustment is performed separately from camera calibration, then the processing steps are simpler and more modular, but the location data accuracy deteriorates due to miscalibration of multiple sensors
Solution Approach 1:
The patent combines bundle adjustment and camera calibration into a single joint optimization process. The system simultaneously estimates 3D point coordinates, camera positions, orientations, and calibration parameters by minimizing a combined error function that includes both bundle adjustment residuals and calibration constraints. This merging eliminates the need for separate calibration steps and improves location data accuracy by accounting for sensor miscalibration during the same optimization process.
2Measurement precision
If calibration data is determined during bundle adjustment, then the accuracy of spatial relationships between cameras improves, but the computational time and processing load increase
Solution Approach 1:
The system performs preliminary initialization of calibration parameters using manufacturer specifications or pre-calibration data before the joint optimization process. This preliminary action provides reasonable starting values that reduce the number of iterations needed for convergence, thereby reducing computational time while still achieving accurate spatial relationships between cameras.
Solution Approach 2:
The joint optimization process uses iterative feedback where the error from bundle adjustment residuals and calibration constraints is continuously evaluated and used to update the calibration parameters. This feedback mechanism allows the system to progressively refine the spatial relationships between cameras, improving accuracy with each iteration while converging to an optimal solution.
3Adaptability or versatility
If multiple sensors are used in AR devices, then the functionality and data richness improve, but the location data accuracy deteriorates due to sensor miscalibration
Solution Approach 1:
The joint optimization framework is designed to handle multiple sensor types (cameras, IMUs, depth sensors) with different calibration requirements within a single unified process. The system can simultaneously estimate calibration parameters for various sensors while performing bundle adjustment, making the system universally applicable to multi-sensor AR devices and improving location data accuracy without sacrificing functionality.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for camera calibration during bundle adjustment. One of the methods includes maintaining a three-dimensional model of an environment and a plurality of image data clusters that each include data generated from images captured by two or more cameras included in a device. The method includes jointly determining, for a three-dimensional point represented by an image data cluster (i) the newly estimated coordinates for the three-dimensional point for an update to the three-dimensional model or a trajectory of the device, and (ii) the newly estimated calibration data that represents the spatial relationship between the two or more cameras.