Real-time Video Stabilization via Motion Sensing and Crop Polygon Warping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile devices suffer from camera shake and motion distortions due to rolling shutter effects, making videos unwatchable, and existing stabilization methods are either ineffective or require post-processing, which can degrade image quality and increase processing power.
Innovation Solution
Implementing real-time video stabilization using on-board motion sensing systems that access movement information to warp image frames and define a virtual camera, maintaining stability when camera movement is below a threshold, and integrating motion data to correct rolling shutter distortions within the video capture pipeline.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If real-time video stabilization is implemented using on-board motion sensing, then video stability is improved, but device complexity increases
Solution Approach 1:
The patent applies preliminary action by integrating motion sensing data during the video capture phase rather than performing stabilization as a post-processing step. The motion sensing system continuously tracks camera movement, and the image processor uses this pre-captured motion information to stabilize video frames in real-time, avoiding the need for complex retrospective analysis and reducing overall processing complexity.
Solution Approach 2:
The patent segments the video stabilization process into distinct functional components: motion sensing for capturing camera movement, crop polygon identification for defining stable regions, and selective warping for distortion correction. This segmentation allows each component to be optimized independently and processed efficiently in the video pipeline without requiring complex integrated processing.
2Stability of the object's composition
If video stabilization is applied to all frames, then video stability is improved, but processing power consumption increases
Solution Approach 1:
The patent applies local quality by selectively stabilizing only the crop polygon region containing scene content common to multiple frames, rather than applying stabilization transformations to the entire image frame. This localized approach concentrates processing energy on the essential stable region while leaving peripheral areas unchanged, significantly reducing computational load and energy consumption.
Solution Approach 2:
The patent implements partial action by applying stabilization only when camera movement exceeds a threshold and only to the necessary crop polygon region, rather than universally stabilizing all frames. This selective application reduces processing energy by avoiding unnecessary stabilization operations on frames that are already sufficiently stable.
3Manufacturing precision
If rolling shutter distortions are corrected, then image quality is improved, but processing complexity increases
Solution Approach 1:
The patent introduces an intermediary approach by using the crop polygon as a mediator between the distorted rolling shutter image and the final corrected output. The crop polygon defines a stable reference region that guides the warping transformation, simplifying the correction process by providing a clear geometric framework rather than requiring complex pixel-level analysis of the entire distorted image.
Solution Approach 2:
The patent applies parameter changes by transforming the image coordinates according to the motion sensing data and crop polygon geometry to correct rolling shutter distortions. By changing the coordinate transformation parameters based on measured camera motion, the system efficiently corrects distortions without requiring complex iterative optimization or heavy computational resources.
Data Source
AI summary
Real-time video stabilization for mobile devices based on on-board motion sensing. In accordance with a method embodiment of the present invention, a first image frame from a camera at a first time is accessed. A second image frame from the camera at a subsequent time is accessed. A crop polygon around scene content common to the first image frame and the second image frame is identified. Movement information describing movement of the camera in an interval between the first time and the second time is accessed. The crop polygon is warped to remove motion distortions of the second image frame is warped using the movement information. The warping may include defining a virtual camera that remains static when the movement of the camera is below a movement threshold. The movement information may describe the movement of the camera at each scan line of the second image frame.