Motion Filter State Adjustment for Video Stabilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video stabilization systems face challenges in maintaining a smooth motion trajectory while adhering to system constraints, particularly at high zoom ratios, leading to jumpy or unstable output frames due to excessive corrective motion.
Innovation Solution
A method and system that adjusts state variables in the motion filter to account for system constraints, recalculating correction values to ensure smooth stabilization within allowed limits, employing a 'Constraint Kalman Filtering' approach to manage corrective motion and prevent output frames from exceeding defined boundaries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the amount of corrective motion displacement is increased to stabilize video frames at high zoom ratios, then the stabilization effectiveness is improved, but the output frames may exceed the input frame boundaries creating undefined regions
Solution Approach 1:
The patent applies preliminary action by adjusting state variables in advance based on predicted corrective motion. The motion filter anticipates future correction needs and pre-adjusts the state variables to ensure that when corrective motion is applied, the output frame remains within the input frame boundaries, preventing undefined regions before they occur.
Solution Approach 2:
The patent implements feedback by continuously monitoring the corrective motion displacement and using this information to adjust state variables. The system feeds back the actual corrective motion applied and uses this to refine future corrections, ensuring the output frame stays within boundaries while maintaining stabilization effectiveness.
2Object-affected harmful factors
If the motion filter adjusts state variables to respect system constraints, then the output frames remain within boundaries, but the motion trajectory may become less smooth
Solution Approach 1:
The patent applies dynamics by making the state variable adjustment adaptive rather than fixed. The system dynamically adjusts the degree of state variable modification based on the current motion conditions and constraint requirements, allowing smooth motion trajectories when possible while enforcing boundaries when necessary, thus balancing smoothness and constraint compliance.
Solution Approach 2:
The patent changes parameters by modifying state variables such as position, velocity, and acceleration estimates. By adjusting these parameters within controlled ranges and using them to predict future motion, the system maintains smooth trajectories while ensuring output frames remain within boundaries, resolving the contradiction between smoothness and constraint adherence.
3Reliability
If corrective motion is applied to align video frames with neighborhood frames, then high frequency fluctuations are cancelled, but the amount of corrective motion is limited by the difference between input and output frame sizes
Solution Approach 1:
The patent applies preliminary action by pre-calculating and adjusting state variables before applying corrective motion. This anticipation allows the system to work within the constraints of frame size differences while still achieving effective jitter cancellation, avoiding the need for complex real-time frame size management.
Solution Approach 2:
The patent replaces direct mechanical frame size manipulation with a computational approach using state variable adjustment. Instead of managing complex frame size differences directly, the system substitutes this with adjusting motion parameters and state variables, simplifying the overall system complexity while maintaining stabilization effectiveness.
Data Source
AI summary
For applying a motion filter of a video stabilization system to a sequence of video frames, an estimate of a motion in the current video frame compared to a first video frame of the sequence of video frames is received. Based on the received motion estimate and on at least one state variable of the motion filter, a correction value for the motion in the current video frame is computed. The at least one state variable is updated in the computation. In case the computed correction value exceeds a system constraint of the video stabilization system, the at least one state variable is adjusted in accordance with an extent by which the system constraint is exceeded. The correction value is then recomputed based on the motion estimate and on the adjusted state variable.