Dual-Rate Video Stitching for Smartphone Parallax Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image stitching techniques for consumer devices, such as smartphones, are resource-intensive and struggle to achieve high accuracy due to limited processing resources, making them unsuitable for efficient video stitching.
Innovation Solution
The proposed method combines image-based registration and calibration-based registration techniques, where calibration-based registration is performed at a higher rate than image-based registration, with the latter being refined by the former to generate stitching transforms, reducing resource consumption and improving alignment accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image-based registration is performed at high rate, then alignment accuracy is improved, but processing resource consumption increases
Solution Approach 1:
The patent implements a dual-rate registration system where image-based registration is performed periodically at selected frames (keyframes) rather than continuously at every frame. This periodic execution of the computationally intensive image-based registration reduces overall processing resource consumption while maintaining alignment accuracy through regular updates at strategic intervals.
Solution Approach 2:
The patent performs calibration-based registration in advance for all frames, creating a preliminary alignment foundation. This preliminary calibration data is then refined selectively using image-based registration only when needed, avoiding redundant high-cost processing while maintaining final alignment accuracy.
2Productivity
If calibration-based registration is performed at high rate, then processing efficiency is improved, but alignment accuracy deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where image-based registration results from keyframes are used to evaluate and correct the calibration-based registration. The system continuously monitors alignment quality through periodic image-based registration and uses this feedback to adjust calibration parameters, ensuring that high-rate calibration processing does not compromise final alignment accuracy.
Solution Approach 2:
The patent merges two registration approaches by combining calibration-based registration (for continuous frames) with image-based registration (for keyframes). The calibration-based registration provides continuous processing efficiency, while image-based registration periodically validates and refines the alignment, creating a hybrid system that achieves both efficiency and accuracy.
3Measurement precision
If image-based registration is performed for all frames, then alignment accuracy is improved, but device complexity increases
Solution Approach 1:
The patent segments the frame processing into two distinct categories: keyframes that undergo full image-based registration, and intermediate frames that use only calibration-based registration. This segmentation allows the system to apply different processing complexities to different frames, reducing overall device complexity while maintaining alignment accuracy for the complete video sequence.
Data Source
AI summary
Systems and methods for stitching videos are disclosed. Image-based registration between frames from a first video source and frames from a second video source is performed at a first rate. Calibration-based registration between frames from the first video source and frames from the second video source are performed at a second rate higher than the first rate. Then, for a first frame from the first video source for which calibration-based registration data and image-based registration data have been generated, a stitching transform that maps the first frame to a counterpart frame from the second video source based on image-based registration data is generated. A delta transform from the image-based registration data and the calibration-based registration data at the first frame is also derived. For a subsequent frame from the first video source for which calibration-based registration data have been generated, but no image-based registration data have been generated, a stitching transform that maps the subsequent frame to a counterpart frame from the second video source based on the calibration-based registration data and the delta transform is generated. Frames from the first video source and frames from the second video source are stitched according to their respective generated stitching transforms.