Multi-source Video Stabilization via Pixel Interpolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile devices struggle to capture stable video due to user-handshake-induced camera wobble, resulting in shaky footage as frames are captured from slightly different perspectives.
Innovation Solution
A method to create a stabilized video by accessing two videos with different fields of view, aligning and interpolating pixels between them to create frames with a common stabilized field of view, and equalizing frame rates by dropping or interpolating frames, using processors to perform these operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If a single camera is used to record video, then the device complexity is low, but the video stability is poor due to hand-held shaking
Solution Approach 1:
The patent combines multiple video streams from different cameras (e.g., wide-angle and telephoto cameras) into a single stabilized video output. By merging the data from multiple sources, the system achieves better stability compensation than a single camera could provide alone, resolving the contradiction between video stability and device complexity.
Solution Approach 2:
The patent introduces an image processing system as an intermediary that takes raw video frames from multiple cameras, performs stabilization calculations, and generates the final stabilized video. This intermediary processing layer enables stability improvement without requiring complex mechanical stabilization mechanisms in the cameras themselves.
2Stability of the object's composition
If multiple videos with different fields of view are processed to create stabilized frames, then the video stability is improved, but the processing complexity increases
Solution Approach 1:
The patent divides the video stabilization task into separate processing stages: frame alignment, pixel selection, interpolation, and composition. By segmenting the processing workflow, the system manages the complexity of handling multiple video streams with different fields of view through modular, manageable steps rather than a monolithic complex process.
Solution Approach 2:
The patent applies different processing techniques to different regions of the video frames. For example, it selectively interpolates pixels from different source frames based on the local field of view requirements, applying appropriate stabilization methods to specific areas rather than uniformly processing the entire frame, thus optimizing processing efficiency.
3Reliability
If frames are interpolated from multiple videos to equalize frame rates, then the video quality is improved, but the processing time increases
Solution Approach 1:
The patent performs preliminary frame rate equalization by pre-selecting and interpolating frames before the main stabilization processing. By preparing the frame sequences in advance and establishing the timing structure early in the processing pipeline, the system reduces the computational burden during subsequent stabilization steps, thereby reducing overall processing time while maintaining quality.
Data Source
AI summary
Producing stabilized video can include: (a) accessing a first video including a sequence of first frames, (b) accessing a second video including a sequence of second frames, and (c) building the stabilized video based on the accessed first and second videos. The stabilized video can include a sequence of stabilized frames. At least one stabilized frame can include pixels derived from at least one of the first frames and at least one of the second frames.