Mobile Video Stabilization Using Gyroscope and Focal-Length Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video stabilization techniques for mobile devices struggle with accurately modeling camera motion to correct for undesired movements, rolling shutter distortion, and focus breathing, leading to inconsistent results between preview and final video composition, and are often limited by computing resources.
Innovation Solution
A method that correlates gyroscope and focal length sensor data with hand motion to estimate real camera orientation, using a stabilization mesh to transform frames and correct for undesired movements, while maintaining a constant virtual camera orientation to stabilize video in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image-based analysis is used to estimate camera motion, then motion correction can be applied, but noise, blur, and dominant objects cause unreliable estimates
Solution Approach 1:
The patent introduces an intermediary approach by combining multiple data sources (gyroscope data, focal length sensor data, and image-based analysis) rather than relying solely on image-based analysis. This mediator approach filters out unreliable estimates caused by noise, blur, and dominant objects by cross-validating with sensor data
Solution Approach 2:
The system dynamically changes parameters by adjusting the weighting and contribution of different data sources (gyroscope, focal length sensor, image analysis) based on current video conditions. When image-based analysis becomes unreliable due to noise or blur, the system shifts reliance toward sensor data
2Manufacturing precision
If complex stabilization techniques are used to correct all distortions, then video quality improves, but computing resources on mobile devices are exceeded
Solution Approach 1:
The patent segments the stabilization process into distinct components: gyroscope-based orientation estimation, focal length sensor-based focus breathing correction, and selective image-based refinement. Each component handles specific aspects of stabilization, reducing overall computational complexity while maintaining quality
Solution Approach 2:
The system extracts and corrects specific distortion components separately - focusing on focus breathing correction using focal length sensor data and orientation correction using gyroscope data, rather than attempting to correct all distortions simultaneously through computationally intensive methods
3Adaptability or versatility
If focus distance is changed to adjust focus, then focus breathing occurs causing image boundary shift, but maintaining fixed focus limits depth of field control
Solution Approach 1:
The patent implements feedback by continuously monitoring focal length sensor data to detect focus breathing effects. When focus distance changes cause image boundary shifts, the system uses this feedback information to apply corrective transformations that compensate for the breathing effect, maintaining image boundary stability while preserving focus control flexibility
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosed technology includes techniques for providing improved video stabilization on a mobile device. Using gyroscope data of the mobile device, the physical camera orientation of the mobile device may be estimated over time. Using the physical camera orientation and historical data, corresponding virtual camera orientations representing a camera orientation with undesired rotational movement removed may be modeled using a non-linear filter to provide for mapping of a real image to a stabilized virtual image. The virtual camera orientation may be modified to prevent undefined pixels from appearing in the output image.