Stereo Camera Bundle Adjustment Under Headset Deformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality devices experience deformations due to movements, leading to inaccuracies in camera calibration and positioning, which are not effectively addressed by existing methods.
Innovation Solution
Implementing bundle adjustment using epipolar constraints to refine camera models and account for deformations in real-time, optimizing reprojection and epipolar errors to improve camera calibration accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If bundle adjustment is performed using traditional reprojection error minimization, then camera pose and 3D point positions can be refined, but accuracy deteriorates when the device undergoes deformations during use
Solution Approach 1:
The patent applies epipolar constraints as a feedback mechanism to detect and correct deformations. By monitoring epipolar errors across multiple images and using them to update camera extrinsic parameters, the system continuously compensates for device deformations, maintaining calibration accuracy despite physical changes to the device structure
Solution Approach 2:
The patent dynamically adjusts camera extrinsic parameters based on detected deformations. By allowing these parameters to change in response to measured epipolar errors, the system adapts to structural changes in the device while maintaining accurate camera calibration and positioning
2Measurement precision
If online calibration is performed as part of bundle adjustment, then camera extrinsics can be corrected, but computational efficiency deteriorates due to susceptibility to weighting schemes
Solution Approach 1:
The patent applies partial correction by using epipolar constraints only for correcting camera extrinsics rather than performing full bundle adjustment. This selective application of correction methods maintains accuracy where needed while reducing overall computational burden and improving update rates
3Measurement precision
If multiple cameras are used for triangulation-based SLAM, then positioning accuracy can be improved, but device complexity increases due to need for deformation estimation
Solution Approach 1:
The patent makes the epipolar constraint method universal by applying it to all camera pairs in the system. The same constraint-based approach works for any number of cameras, providing a scalable solution that maintains positioning accuracy without proportionally increasing system complexity
Data Source
AI summary
Methods, systems, and apparatus for performing bundling adjustment using epipolar constraints. A method includes receiving image data from a headset for a particular pose. The image data includes a first image from a first camera of the headset and a second image from a second camera of the headset. The method includes identifying at least one key point in a three-dimensional model of an environment at least partly represented in the first image and the second image and performing bundle adjustment. Bundle adjustment is performed by jointly optimizing a reprojection error for the at least one key point and an epipolar error for the at least one key point. Results of the bundle adjustment are used to perform at least one of (i) updating the three-dimensional model, (ii) determining a position of the headset at the particular pose, or (iii) determining extrinsic parameters of the first camera and second camera.