Epipolar-Constrained Bundle Adjustment for AR Headset Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality systems face challenges in accurately accounting for deformations in wearable devices due to movements and deformations, leading to inaccuracies in camera calibration and environmental mapping.
Innovation Solution
Implementing bundle adjustment using epipolar constraints to refine camera positions and extrinsic parameters in real-time, accounting for deformations caused by movements and rotations in wearable augmented reality devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If bundle adjustment is performed using traditional reprojection error minimization, then computational efficiency is maintained, but accuracy of camera calibration deteriorates due to unaccounted deformations in wearable devices
Solution Approach 1:
The patent changes the optimization parameters by introducing epipolar geometry constraints as additional error terms in the bundle adjustment process. Instead of only minimizing reprojection error, the system minimizes a combined error function that includes epipolar constraints, which account for camera deformations. This parameter change enables accurate calibration despite wearable device movements without requiring complex deformation modeling.
Solution Approach 2:
The patent replaces direct mechanical deformation measurement (which would require complex sensors and models) with a computational substitution using epipolar geometry constraints. By substituting the mechanical deformation problem with a geometric constraint-based mathematical model, the system achieves accurate calibration while maintaining computational efficiency suitable for real-time operation.
2Measurement precision
If epipolar constraints are added to bundle adjustment to account for deformations, then calibration accuracy improves, but computational load increases
Solution Approach 1:
The patent applies partial action by selectively enforcing epipolar constraints only on corresponding feature points that are visible in multiple camera views, rather than attempting to model all possible deformations. This partial application of constraints provides sufficient accuracy improvement while avoiding the excessive computational burden of comprehensive deformation modeling, enabling real-time operation on wearable devices.
3Reliability
If traditional bundle adjustment is used without deformation compensation, then processing speed is maintained, but reliability of environmental mapping deteriorates
Solution Approach 1:
The patent introduces epipolar geometry constraints as an intermediary mathematical model that mediates between the observed image data and the underlying camera deformation. This intermediary constraint system translates deformation effects into measurable geometric relationships, enabling reliable environmental mapping without requiring direct measurement or complex modeling of the physical deformation processes.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and apparatus for performing bundling adjustment using epipolar constraints. A method includes receiving image data from a headset for a particular pose. The image data includes a first image from a first camera of the headset and a second image from a second camera of the headset. The method includes identifying at least one key point in a three-dimensional model of an environment at least partly represented in the first image and the second image and performing bundle adjustment. Bundle adjustment is performed by jointly optimizing a reprojection error for the at least one key point and an epipolar error for the at least one key point. Results of the bundle adjustment are used to perform at least one of (i) updating the three-dimensional model, (ii) determining a position of the headset at the particular pose, or (iii) determining extrinsic parameters of the first camera and second camera.