Visual-Inertial Odometry With Shared Image Stabilization Pipeline
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera devices face challenges in accurately determining pose and orientation due to blurriness caused by unintentional movement, which affects visual-inertial odometry (VIO) performance and image stabilization, leading to errors in localization and object detection.
Innovation Solution
Integration of image stabilization (IS) and VIO techniques through a closed-loop design using a tracking filter that shares computational resources, iteratively updating intermediate results to improve image quality and pose estimation by compensating for unintentional camera movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image stabilization is performed separately from VIO, then image quality can be improved, but computational resources and power consumption increase
Solution Approach 1:
The patent combines image stabilization and VIO into a single integrated processing pipeline where both functions share common computational resources, including feature detection algorithms, tracking filters, and processing hardware, thereby reducing overall power consumption while maintaining image quality
Solution Approach 2:
The integrated system uses multi-functional processing units that can perform both image stabilization and VIO tasks. The same feature detection and tracking components serve dual purposes: stabilizing images and estimating camera pose, eliminating the need for separate dedicated hardware for each function
2Measurement precision
If image stabilization is performed separately from VIO, then image stabilization can be achieved, but device complexity increases
Solution Approach 1:
The patent merges image stabilization and VIO into a unified processing architecture where both functions share common computational resources, including feature detection algorithms, tracking filters, and processing hardware, thereby reducing overall system complexity
Solution Approach 2:
The system employs multi-functional processing units that handle both image stabilization and VIO operations. The same feature detection and tracking components serve dual purposes: stabilizing images and estimating camera pose, eliminating the need for separate dedicated hardware for each function
3Speed
If VIO processes un-stabilized frames, then processing speed can be maintained, but pose estimation accuracy deteriorates due to blurriness
Solution Approach 1:
The system performs preliminary image stabilization processing on frames before they are fed into the VIO pipeline. By pre-stabilizing the input frames, the VIO algorithm receives clearer images with reduced motion blur, enabling more accurate feature detection and pose estimation without significantly impacting overall processing speed
Solution Approach 2:
The integrated design maintains continuous processing flow where stabilization and VIO operations overlap in time. The system processes frames through both stabilization and VIO pipelines in a coordinated manner, ensuring that useful computational actions continue without interruption while improving accuracy
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An example method for image processing includes receiving a first set of information, the first set of information being indicative of angular velocity and movement of the device based on a change in position of the device across a current frame captured by the device and a subsequent frame captured by the device, filtering the first set of information to compensate for unintentional movement of the device to generate a second set of information, the second set of information being indicative of angular velocity and movement of the device without unintentional movement, performing image stabilization on the current frame based on both the first set of information and the second set of information to generate an image stabilized current frame, and outputting, for display, the image stabilized current frame.