Time-lapse Video Stabilization Using Spatiotemporal Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video generation technologies struggle to create stable time-lapse videos from footage captured by a moving image capture device, as the motion introduces jerky or shaky footage.
Innovation Solution
A system that generates time-lapse videos by determining time-lapse video frames based on a spatiotemporal metric characterizing spatial smoothness and temporal regularity, using rotational position information from a motion sensor to stabilize the video frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If video frames are captured continuously during device motion, then temporal resolution is improved, but spatial stability deteriorates causing jerky footage
Solution Approach 1:
The patent segments the continuous video frames into discrete keyframes selected based on motion thresholds. Instead of using all captured frames, it divides them into meaningful segments where each keyframe represents a stable moment in the sequence, resolving the conflict between temporal density and spatial stability.
Solution Approach 2:
The patent introduces motion analysis and spatiotemporal metrics as intermediary processes between raw video capture and final time-lapse generation. These intermediaries evaluate frame stability and select appropriate frames, acting as a mediator that filters out unstable frames while preserving temporal progression.
2Loss of information
If all video frames are retained in time-lapse output, then temporal completeness is improved, but output size and processing complexity increase
Solution Approach 1:
The patent extracts only the essential keyframes from the complete video sequence based on motion criteria. By taking out only the frames that contribute meaningfully to temporal progression while removing redundant or unstable frames, it maintains temporal completeness of significant events while dramatically reducing output size and processing requirements.
Solution Approach 2:
The patent changes the selection parameter from including all frames to including only frames that meet specific spatiotemporal criteria. This parameter change in frame selection strategy reduces output complexity while preserving the essential temporal narrative of the video content.
3Productivity
If frames are selected without motion compensation, then processing speed is improved, but visual smoothness deteriorates
Solution Approach 1:
The patent performs preliminary motion analysis and frame stability evaluation during the keyframe selection process itself, rather than requiring separate post-processing stabilization steps. By anticipating and compensating for motion issues in advance during frame selection, it achieves visual smoothness without sacrificing processing efficiency.
Data Source
AI summary
Video content may be captured by an image capture device during a capture duration. The video content may include video frames that define visual content viewable as a function of progress through a progress length of the video content. Rotational position information may characterize rotational positions of the image capture device during the capture duration. Time-lapse video frames may be determined from the video frames of the video content based on a spatiotemporal metric. The spatiotemporal metric may characterize spatial smoothness and temporal regularity of the time-lapse video frames. The spatial smoothness may be determined based on the rotational positions of the image capture device corresponding to the time-lapse video frames, and the temporal regularity may be determined based on moments corresponding to the time-lapse video frames. Time-lapse video content may be generated based on the time-lapse video frames.