Rolling Shutter Video Stabilization Motion Vector Rectification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for combining rolling shutter rectification and video stabilization in image processing suffer from a dramatic loss of accuracy when sharing the motion estimation step, leading to suboptimal performance compared to sequential solutions, as they perform motion estimation between unrectified original images rather than rectified frames.
Innovation Solution
An apparatus and method that perform motion estimation once on a pair of images, then alter the motion vectors to account for rolling shutter distortions and image jitter, using a rolling-shutter parameter extraction unit and video stabilization parameter extraction unit to generate correction parameters for rectifying and stabilizing the video sequence, ensuring both algorithms operate on rectified images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If motion estimation is shared between rolling shutter rectification and video stabilization algorithms, then computational cost is reduced by performing motion estimation only once, but measurement precision of video stabilization deteriorates dramatically because motion estimation is performed on unrectified images
Solution Approach 1:
The patent applies preliminary action by performing rolling shutter rectification before video stabilization processing. The system first corrects the geometric distortion caused by rolling shutter effects, then performs motion estimation and video stabilization on the already-rectified images. This ensures that motion estimation operates on geometrically accurate data, maintaining high measurement precision while still achieving computational efficiency through the sequential sharing of motion estimation results.
2Measurement precision
If motion estimation is performed sequentially for both rolling shutter rectification and video stabilization, then measurement precision is maintained by performing motion estimation on rectified frames, but computational cost increases due to performing motion estimation twice
Solution Approach 1:
The patent merges the two processing pipelines by performing rolling shutter rectification first, then using the rectified images for both rolling shutter correction application and video stabilization processing. The motion estimation performed on rectified images serves both purposes simultaneously, combining what would otherwise be separate motion estimation operations into a single unified process that maintains precision while improving computational efficiency.
Data Source
AI summary
This invention generally relates to rolling shutter effects rectification and stabilisation of video sequences. The proposed solution simulates motion estimation on rolling shutter corrected images by altering the motion vector field based on a rolling-shutter correction parameter. By doing so it is taken advantage of the fact that the motion vector field obtained from motion estimation of images rectified in rolling shutter is equivalent to the rectified motion vector field of the subject application. This way the proposed apparatus is able to perform both rolling shutter correction and video stabilization algorithms based on original images while in the sequential implementation of prior art, original images are used for the rolling shutter algorithm and rolling shutter corrected images are used in video stabilisation.