Motion Estimation Device for Rolling Shutter Distortion Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital cameras using a rolling shutter scheme suffer from image distortion when the imaging target or the camera moves, which complicates motion estimation between image frames.
Innovation Solution
A motion estimation method that groups row blocks of image frames into banks, calculates normalized cross-correlation or sum of absolute difference, detects local maxima/minima, estimates relative displacement, and calculates motion coefficients such as velocity, acceleration, and jolt to accurately determine motion between frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a rolling shutter scheme is used to capture image frames, then the imaging speed and frame rate are improved, but image distortion occurs when the imaging target or camera moves
Solution Approach 1:
The image frame is divided into multiple row blocks that are processed independently. Each row block's motion is estimated separately using NCC or SAD calculation, allowing the system to handle local variations in motion and distortion caused by rolling shutter effects without compromising overall image quality.
Solution Approach 2:
The patent changes the parameter of motion estimation by using normalized cross-correlation or sum of absolute difference calculations on segmented row blocks. This allows adaptive handling of different motion scenarios (constant velocity, acceleration, jolt) and compensates for rolling shutter distortion by adjusting motion parameters for each row block.
2Loss of information
If motion estimation is performed on the entire image frame, then the motion information is comprehensive, but the computational complexity and processing time increase
Solution Approach 1:
The image frame is divided into multiple row blocks organized in banks, where each bank contains a reference row block and other row blocks. Motion estimation is performed independently on each bank, significantly reducing computational complexity while maintaining comprehensive motion information through the collective results of all banks.
Solution Approach 2:
Instead of performing full-frame motion estimation, the patent applies motion estimation selectively to representative row blocks in each bank. The motion coefficients derived from these partial estimations are then applied to reconstruct motion information for the entire frame, achieving a balance between computational efficiency and information completeness.
3Measurement precision
If motion coefficients are calculated for each row block individually, then the motion estimation precision is improved, but the processing time and computational load increase
Solution Approach 1:
The patent segments the image into banks of row blocks, where motion coefficients are calculated for representative blocks in each bank rather than every single row block. This segmentation maintains precision within each bank while reducing overall processing time through parallel processing of multiple banks.
Solution Approach 2:
The motion coefficients calculated from reference row blocks serve multiple purposes: they characterize the motion of the entire bank, enable reconstruction of motion for all row blocks in the bank, and can be used for both motion compensation and motion analysis, reducing redundant calculations.
Data Source
AI summary
A motion estimation method includes grouping row blocks of a first frame into a first plurality of banks and grouping row blocks of a second frame into a corresponding second plurality of banks, respectively; calculating the normalized cross correlation (NCC) or the sum of absolute difference (SAD) between the banks of the first frame and the banks of the second frame; detecting the local maxima from the NCC or the local minima from the SAD; estimating a first relative displacement between corresponding reference row blocks of the first frame and of the second frame; calculating motion coefficients using first relative displacements; and estimating second relative displacements using the motion coefficients.