Hybrid Inertial-Visual Movement Estimation for Video Stabilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Embedded processors in devices like digital tablets and mobile cellular telephones face challenges in efficiently estimating movement between successive video images due to high variability in calculation time of existing algorithms, leading to quality issues such as fuzzy content and distortions caused by the rolling shutter effect.
Innovation Solution
A method that combines visual information from RANSAC-type algorithms with inertial sensor data, such as gyroscopes and accelerometers, to improve movement estimation between successive video images, using a Pre-emptive RANSAC algorithm to reduce calculation time while enhancing estimation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If RANSAC algorithm is used to estimate global movement between successive images, then measurement precision is improved, but loss of time increases due to high variability in calculation time
Solution Approach 1:
The patent applies preliminary action by using inertial sensor data to predict movement between successive images before performing visual matching. The inertial measurement unit (IMU) provides advance information about device motion, allowing the system to pre-establish a hypothesized homography matrix based on predicted movement, thereby reducing the computational search space and time for the RANSAC algorithm.
Solution Approach 2:
The patent introduces inertial sensor data as an intermediary between image capture and movement estimation. The IMU serves as a mediator that provides complementary motion information, allowing the system to combine inertial predictions with visual verification. This intermediary approach enables the RANSAC algorithm to work with pre-filtered hypotheses rather than all possible transformations.
2Productivity
If Pre-emptive RANSAC algorithm is used to reduce calculation time, then productivity is improved, but measurement precision deteriorates in extreme cases such as when objects move within the image field
Solution Approach 1:
The patent merges two independent measurement systems: inertial sensing (IMU) and visual processing (camera). By combining data from both sources, the system creates a hybrid estimation approach where inertial predictions provide initial hypotheses and visual data provides verification and correction. This merging allows the system to maintain high processing speed while improving accuracy in extreme cases where visual features alone may be insufficient.
Solution Approach 2:
The patent dynamically adjusts the weighting and influence of inertial versus visual data based on scene conditions. When visual features are abundant and stable, the system relies more on visual matching; when features are scarce or ambiguous (extreme cases), the system increases reliance on inertial predictions. This parameter adjustment allows the system to optimize between speed and accuracy based on real-time conditions.
Data Source
AI summary
The method includes for each current pair of first and second successive video images determining movement between the two images. The determining includes a phase of testing homography model hypotheses on the movement by a RANSAC type algorithm operating on a set of points in the first image and first assumed corresponding points in the second image so as to deliver one of the homography model hypothesis that defines the movement. The test phase includes a test of first homography model hypotheses of the movement obtained from a set of second points in the first image and second assumed corresponding points in the second image. At least one second homography model hypothesis is obtained from auxiliary information supplied by an inertial sensor and representative of a movement of the image sensor between the captures of the two successive images of the pair.