Real-time Image Stabilization via Centroid Alignment and Tile-based Affine Transform
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image data captured by shaky video cameras often introduces artifacts, which degrade image fidelity, and existing methods fail to effectively address motion involving translation and rotation in multiple directions.
Innovation Solution
The method calculates the centroid of a reference video frame using weighted averages of pixel intensity or hue and translates subsequent frames to align with this centroid, and for more complex motion, breaks the image into tiles, using curve fitting to determine an affine transform that accounts for translation, rotation, and other movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If simple centroid translation is used for stabilization, then processing speed is improved, but effectiveness deteriorates when dealing with complex camera motion involving rotation and translation in multiple directions
Solution Approach 1:
The patent divides the video frame into multiple tiles or regions, calculating centroids for each tile separately. This segmentation allows the system to capture complex camera motions more accurately while maintaining real-time processing capabilities through efficient parallel computation.
Solution Approach 2:
The patent dynamically adjusts the stabilization approach based on motion complexity. For simple motions, it uses direct centroid translation; for complex motions involving rotation and multi-directional translation, it employs affine transforms determined through curve fitting techniques, optimizing both speed and effectiveness.
2Reliability
If affine transform with curve fitting is used to handle complex motion, then stabilization effectiveness is improved, but computational complexity increases
Solution Approach 1:
The patent applies affine transforms and curve fitting only when necessary for complex motions, rather than for every frame. This selective application reduces overall computational complexity while maintaining high stabilization effectiveness when needed.
Solution Approach 2:
By dividing frames into tiles and processing them in parallel, the patent reduces the computational burden of affine transforms through distributed computation, making complex stabilization more computationally manageable.
3Device complexity
If the entire frame is processed as a single unit, then processing simplicity is maintained, but ability to handle complex multi-directional camera motion deteriorates
Solution Approach 1:
The patent segments the video frame into multiple tiles, calculating centroids for each tile independently. This segmentation enables accurate capture of complex multi-directional camera motions while maintaining processing efficiency through parallel computation of individual tile centroids.
Solution Approach 2:
The patent transitions from processing the entire frame as a single 2D unit to processing multiple smaller tile regions, adding a spatial dimension to the analysis. This allows independent motion analysis of different frame regions, improving accuracy for complex motions.
Data Source
AI summary
An apparatus and method for stabilizing image frames in a video data stream. A weighted average or centroid of the intensity or hue associated with pixels vs. the horizontal and vertical position of each pixel is calculated for a reference frame in the video data stream. A corresponding centroid is calculated for a subsequent frame in the stream. This image frame is then translated so that the centroid of the subsequent frame and the centroid of the reference frame coincide, reducing artifacts from shaking of the video capture device. Alternatively, the video stream frames may be divided into tiles and centroids calculated for each tile. The centroids of the tiles of a subsequent frame are curve fit to the centroids of tiles in a reference frame. An affine transform is then performed on the subsequent frame to reduce artifacts in the image from movements of the video capture device.