Stereo Camera Pair Online Rectification for XR Image Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video see-through extended reality (XR) systems face challenges in rendering accurate stereoscopic image pairs due to non-parallel optical axes of imaging sensors, requiring inconvenient and time-consuming offline calibration and rectification at specialized platforms.
Innovation Solution
Implement online rectification techniques to calibrate and rectify a stereo pair of image frames using extrinsic parameters, allowing on-the-fly processing without the need for physical shipment to a specialized platform.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If offline calibration and rectification are performed at specialized platforms, then manufacturing precision of stereoscopic image pairs is improved, but device complexity and loss of time increase
Solution Approach 1:
The system performs self-calibration by automatically detecting features in the captured image and computing extrinsic parameters without requiring external specialized equipment or manual intervention. The processing device autonomously completes the calibration process using onboard imaging sensors and computational algorithms.
Solution Approach 2:
The patent replaces the mechanical/physical calibration process (requiring specialized platforms and manual adjustment) with a computational approach using image processing algorithms, feature detection, and mathematical computations to determine extrinsic parameters and perform rectification.
2Manufacturing precision
If offline calibration and rectification are performed at specialized platforms, then manufacturing precision of stereoscopic image pairs is improved, but loss of time increases
Solution Approach 1:
The system performs calibration actions continuously in the background during normal operation, capturing images and computing extrinsic parameters beforehand. This preliminary action ensures that when stereoscopic rendering is needed, the calibration is already complete, eliminating delays.
Solution Approach 2:
The calibration process operates continuously and automatically during device usage rather than requiring discrete offline sessions. The system maintains ongoing image capture and processing to continuously refine and update extrinsic parameters, making calibration time negligible.
3Ease of operation
If online rectification is performed, then ease of operation is improved, but measurement precision may worsen
Solution Approach 1:
The system employs feedback mechanisms where captured images are processed to compute extrinsic parameters, which are then used to generate rectified stereoscopic views. The process iteratively refines parameters based on the quality of feature matching and image alignment, ensuring high precision while maintaining ease of use.
Solution Approach 2:
The patent creates a computational model (copy) of the camera system's extrinsic parameters by analyzing feature correspondences between images. This digital copy allows for precise rectification without physical adjustment, maintaining both convenience and measurement precision through algorithmic accuracy.
Data Source
AI summary
A method includes obtaining, using multiple imaging sensors of an electronic device, a left image frame and a right image frame forming a stereo pair of image frames. The method also includes identifying, using at least one processing device of the electronic device, extrinsic parameters associated with relative positions and orientations of the imaging sensors. The method further includes performing, using the at least one processing device, an online stereo rectification of the stereo pair of image frames based on the extrinsic parameters such that epipolar lines of the left and right image frames are horizontally aligned to generate a rectified stereo pair of image frames. In addition, the method includes rendering, using the at least one processing device, one or more images for display based on the rectified stereo pair of image frames.