Patch-Based Alignment Data Determination for Video Stabilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera stabilization methods fail to effectively handle high and low frequency instabilities in image capture at high frame rates and high resolution, particularly in scenarios with limited hardware, leading to latency issues and increased storage and computation costs.
Innovation Solution
A method for determining alignment data between target and reference frames by receiving patch data, calculating shifts between corresponding patches, and using these shifts to determine alignment data, which can be implemented in a system with a processor and memory to process video sequences and stabilize images in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If digital image stabilisation methods are used to process high definition images at high frame rate, then image quality is improved, but processing time and computation cost increase significantly
Solution Approach 1:
The image is divided into multiple patches, and alignment data is determined for each patch independently rather than processing the entire high definition image at once. This segmentation allows parallel processing of patches, reducing overall computation time while maintaining image quality stabilization
Solution Approach 2:
The patent determines alignment data for only a selected plurality of patches rather than all pixels in the high definition image. This partial action approach provides sufficient stabilization效果 with significantly reduced computation cost and processing time compared to full-image processing
2Device complexity
If post-processing of video sequences is performed to avoid processing limitations, then computation cost is reduced, but latency increases making it unsuitable for live broadcast
Solution Approach 1:
The patent performs alignment data determination for patches in real-time as video frames are captured, rather than waiting to complete post-processing of entire video sequences. This preliminary action enables live broadcast applications by providing stabilization results with minimal latency while keeping computation costs manageable through patch-based processing
3Object-affected harmful factors
If mechanical stabilisation systems are used to damp high frequency instability, then camera shake is reduced, but the system cannot handle low frequency motion such as drift
Solution Approach 1:
The patent determines alignment data that captures both high frequency motion (camera shake) and low frequency motion (drift) by analyzing patch positions across multiple frames. The alignment data parameters are designed to represent various types of camera motion, enabling the system to handle a broad range of instabilities including both high and low frequency components through digital processing rather than mechanical means
Data Source
AI summary
A method of determining alignment data for a target frame and a reference frame. Patch data is received for each of a plurality of reference patches of the reference frame. The patch data comprises a reference direction of a feature in an associated reference patch and a reference signal corresponding to a projection of reference patch image data in a direction substantially perpendicular to the reference direction. A shift between at least one of said reference patches and a corresponding target patch in the target frame is determined using the reference signal associated with the reference patch and a target signal corresponding to a projection of the target patch image data in the direction substantially perpendicular to the reference direction associated with said reference patch. The alignment data for the target frame is determined using the determined shift for the reference patch.