Video See-Through AR Calibration for Low-Latency Distortion Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video see-through augmented reality systems suffer from latency and computational inefficiencies due to real-time image rendering processes that correct distortions caused by camera and display lenses, leading to inaccurate scene display and user disorientation.
Innovation Solution
A method to generate a transformation map prior to rendering, accounting for distortions introduced by the see-through camera, display lens, and virtual camera, using models to pre-calculate corrections for subsequent images, reducing the need for on-the-fly calculations and improving latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time image rendering is performed with video see-through AR systems, then virtual objects can be displayed, but distortions from cameras and display lenses cause latency and misregistration
Solution Approach 1:
The system performs preliminary actions by pre-calculating and storing distortion correction data in lookup tables before actual rendering occurs. The camera distortion model and display lens distortion model are pre-computed and stored, allowing rapid retrieval during real-time rendering without performing complex calculations in real-time, thus reducing latency while maintaining accuracy.
2Manufacturing precision
If distortion correction is performed during image rendering, then rendering accuracy improves, but computational resources and processing time increase
Solution Approach 1:
Complex distortion correction computations are performed in advance during system initialization or calibration phases. The camera distortion model and display lens distortion model are pre-computed and stored in lookup tables. During real-time rendering, the system simply retrieves pre-computed correction data from these tables, transforming the computationally intensive task from runtime to setup time, thereby maintaining high registration precision while preserving rendering speed.
Solution Approach 2:
Instead of performing complex real-time distortion calculations, the system creates a simplified representation (copy) of the distortion characteristics in the form of lookup tables. These tables store pre-computed distortion correction data that can be rapidly queried during rendering, replacing the need for complex real-time computational models while maintaining the same correction accuracy.
3Measurement precision
If frame-dependent distortion correction is applied, then accuracy is maintained, but processing latency increases
Solution Approach 1:
The system performs distortion correction in advance by pre-computing correction data for various frames or conditions and storing it in lookup tables. During actual rendering, the system retrieves the appropriate pre-computed correction data based on the current frame parameters, avoiding the need to perform complex frame-dependent calculations in real-time. This maintains correction accuracy while significantly reducing processing latency.
Data Source
AI summary
In one embodiment, a method includes capturing, by a calibration camera, a calibration pattern displayed on a display of a video see-through AR system, where the calibration camera is located at an eye position for viewing content on the video see-through AR system. The method further includes determining, based on the captured calibration pattern, one or more system parameters of the video see-through AR system that represent combined distortion caused by a display lens of the video see-through AR system and caused by a camera lens of the see-through camera; determining, based on the one or more system parameters and on one or more camera parameters that represent distortion caused by the see-through camera, one or more display-lens parameters that represent distortion caused by the display lens; and storing the one or more system parameters and the one more display-lens parameters as a calibration for the system.