Vision-Aided Inertial Navigation with Rolling-Shutter Time Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vision-aided inertial navigation systems (VINS) face challenges due to unsynchronized clocks between cameras and inertial measurement units (IMUs) and the rolling-shutter effect in consumer-grade devices, leading to inaccurate navigation.
Innovation Solution
A linear-complexity technique using an interpolation-based measurement model and Observability-Constrained Extended Kalman filter (OC-EKF) to compensate for time synchronization and rolling-shutter effects, allowing for accurate fusion of inertial and visual measurements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If offline calibration methods are used to compensate for time offset between camera and IMU, then equipment requirements are reduced, but navigation accuracy significantly degrades due to clock jitter and varying time offsets
Solution Approach 1:
The patent implements dynamic time offset compensation by continuously estimating and updating the time offset between camera and IMU clocks during operation, rather than using static offline calibration. The system models the time offset as a dynamic parameter that varies over time due to clock jitter, and applies real-time correction to rolling-shutter image timestamps based on this dynamic model, thereby maintaining navigation accuracy without complex offline calibration equipment.
2Measurement precision
If rolling-shutter camera measurements are used without compensation, then device complexity is reduced, but navigation accuracy significantly degrades due to the rolling-shutter effect
Solution Approach 1:
The patent applies preliminary compensation for the rolling-shutter effect by correcting image timestamps before they are used in navigation calculations. The system pre-calculates the time offset for each row of the rolling-shutter camera based on the dynamic time synchronization model, and adjusts the timestamps of visual measurements accordingly before fusing them with inertial data, thereby eliminating navigation accuracy degradation without adding complex processing during the main navigation loop.
3Measurement precision
If high-precision time synchronization is implemented, then navigation accuracy is improved, but computational complexity and processing time increase
Solution Approach 1:
The patent optimizes the balance between precision and speed by changing the parameter representation and update frequency of the time offset model. Instead of continuously updating all synchronization parameters at maximum frequency, the system selectively updates only the critical time offset parameters at optimized intervals, and uses efficient interpolation methods to estimate values between updates, thereby maintaining high precision while reducing computational burden and improving real-time processing performance.
Data Source
AI summary
Vision-aided inertial navigation techniques are described. In one example, a vision-aided inertial navigation system (VINS) comprises an image source to produce image data at a first set of time instances along a trajectory within a three-dimensional (3D) environment, wherein the image data captures features within the 3D environment at each of the first time instances. An inertial measurement unit (IMU) to produce IMU data for the VINS along the trajectory at a second set of time instances that is misaligned with the first set of time instances, wherein the IMU data indicates a motion of the VINS along the trajectory. A processing unit comprising an estimator that processes the IMU data and the image data to compute state estimates for 3D poses of the IMU at each of the first set of time instances and 3D poses of the image source at each of the second set of time instances along the trajectory. The estimator computes each of the poses for the image source as a linear interpolation from a subset of the poses for the IMU along the trajectory.