Video Stabilization with Rolling Shutter Distortion Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video stabilization techniques fail to effectively address rolling shutter distortions, particularly wobble distortion, in CMOS sensor-equipped cameras, which are caused by both global and local camera motions, and assume constant motion between frames, leading to incomplete distortion correction.
Innovation Solution
A six-parameter affine model is used to describe the transformation between video frames, incorporating parameters for skew, vertical scaling, and rotational transformations, along with an adaptive IIR filter to estimate intentional transformations, and a method to fill invalid regions in frames for full-size output, addressing wobble distortion and stabilizing both translational and rotational jitter.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If prior art techniques for reducing rolling shutter distortion are used, then skew and vertical scaling artifacts can be compensated, but wobble distortion caused by camera shake cannot be effectively addressed
Solution Approach 1:
The patent transitions from static motion compensation to dynamic motion handling by separating intentional translational motion from unintentional shake motion. The system dynamically adapts to different motion types (panning, zooming, walking toward subject) while independently compensating for wobble distortion caused by camera shake, allowing the distortion correction to respond dynamically to varying camera motion patterns throughout the video sequence.
Solution Approach 2:
The patent segments the motion compensation process into two independent components: intentional translational motion compensation and unintentional shake-induced wobble distortion compensation. This segmentation allows each component to be handled separately with appropriate techniques, enabling the system to address both skew/vertical scaling artifacts and wobble distortion simultaneously without interference between the two compensation mechanisms.
2Device complexity
If constant motion between adjacent frames is assumed, then computational complexity is reduced, but distortion correction completeness deteriorates
Solution Approach 1:
The patent employs dynamic motion estimation that adapts to actual camera motion between frames rather than assuming constant motion. The system estimates motion parameters dynamically for each frame pair, allowing it to capture varying motion patterns (acceleration, deceleration, direction changes) while maintaining computational efficiency through optimized estimation algorithms that focus on the essential motion parameters needed for distortion correction.
3Manufacturing precision
If rolling shutter distortion compensation is applied, then skew and vertical scaling are corrected, but video stabilization against camera shake is not achieved
Solution Approach 1:
The patent segments the overall image processing pipeline into two distinct stages: rolling shutter distortion compensation and video stabilization. The first stage corrects geometric distortions (skew, vertical scaling) to restore proper frame geometry, while the second stage independently addresses camera shake through stabilization techniques. This segmentation allows each function to be optimized independently, ensuring both geometric accuracy and video stability are achieved without compromising either objective.
Data Source
AI summary
A method of processing a digital video sequence is provided that includes estimating compensated motion parameters and compensated distortion parameters (compensated M/D parameters) of a compensated motion/distortion (M/D) affine transformation for a block of pixels in the digital video sequence, and applying the compensated M/D affine transformation to the block of pixels using the estimated compensated M/D parameters to generate an output block of pixels, wherein translational and rotational jitter in the block of pixels is stabilized in the output block of pixels and distortion due to skew, horizontal scaling, vertical scaling, and wobble in the block of pixels is reduced in the output block of pixels.