Stabilizing Spherical Video via Path Smoothing and Image Warping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video playback speed increases jerky or shaky motion due to image sensor movement during capture, which is not effectively stabilized in existing video applications.
Innovation Solution
A system and method that generates stabilized visual content by determining a capture path of image sensors, smoothing it to reduce jitter, and warping the image space based on the difference between the capture and smoothed paths, allowing for smooth playback even at increased speeds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If playback speed is increased, then productivity is improved, but stability of visual content deteriorates causing jerky/shaky appearance
Solution Approach 1:
The system performs preliminary stabilization processing on the visual content before playback. It determines capture paths from motion data, generates smoothed paths to reduce jitter, and pre-warps image spaces based on the differences between capture and smoothed paths. This preliminary processing ensures that when playback speed is increased, the stabilized content remains smooth and free from jerky artifacts.
Solution Approach 2:
The system changes spatial parameters of the visual content by warping image spaces. It calculates transformation parameters based on the difference between capture paths and smoothed paths, then applies these transformations to stabilize the visual content. This parameter change approach allows the content to maintain stability regardless of playback speed increases.
2Adaptability or versatility
If image sensor movement is captured, then adaptability is improved, but stability of visual content deteriorates
Solution Approach 1:
The system segments the stabilization process into distinct components: motion data acquisition, capture path determination, smoothed path generation, and image space warping. By separating the capture of actual motion (for adaptability) from the generation of smoothed playback paths (for stability), the system can maintain both motion capture capability and visual content stability simultaneously.
Solution Approach 2:
The system introduces smoothed paths as an intermediary between the actual capture paths and the final visual content. The smoothed paths act as a mediator that preserves the essential motion information while eliminating jitter and instability, allowing the system to maintain both adaptability to motion and stability of the output content.
3Measurement precision
If capture path follows actual sensor motion, then measurement precision is improved, but stability of visual content deteriorates
Solution Approach 1:
The system applies different quality characteristics to different aspects of motion processing. It maintains high measurement precision in the capture path (accurately tracking actual sensor motion) while applying a different quality characteristic to the smoothed path (reducing high-frequency jitter). This local differentiation allows precise motion tracking to coexist with stable visual content.
Solution Approach 2:
The system dynamically adjusts the path characteristics by generating smoothed paths that adapt to the captured motion. The smoothing process dynamically reduces jitter while preserving essential motion patterns, allowing the system to maintain both measurement precision and visual content stability through dynamic path adjustment.
Data Source
AI summary
Spherical visual content represented in an image space may be obtained. The spherical visual content may have been captured by image sensor(s) during a time duration. The spherical visual content may include phenomena caused by motion of the image sensor(s) and/or optical components that guide light onto the image sensor(s). A capture path taken by the image sensor(s) during the time duration may be determined. The capture path may reflect positions and orientations of the image sensor(s) during the time duration. A smoothed path may be determined based on the capture path. The smoothed path may have smoother changes in positions and/or orientations than the capture path. The image space may be warped based on a difference between the capture path and the smoothed path. The stabilized visual content may be determined by projecting the spherical visual content represented in the warped image space to a spherical projection space.