Multi-camera Video Stabilization via Pose-Matched Trajectory Corrections
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-camera electronic image capture devices face challenges in stabilizing video image streams due to differences in camera configurations and poses, leading to unsatisfactory results when switching between streams, causing user experience issues and inefficiencies in computational and power utilization.
Innovation Solution
The techniques involve calculating stabilized trajectories and corrections for each camera stream, accounting for existing optical image stabilization and rolling shutter corrections, to synchronize and match the poses of multiple cameras, allowing seamless combination and switching of stabilized video streams, with options for users to choose between or combine streams.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the same video image stabilization corrections are applied to multiple camera streams, then the stabilization process is simple and computationally efficient, but the results are unsatisfactory when cameras have different configurations and poses
Solution Approach 1:
The patent applies local quality by customizing stabilization corrections for each camera based on its specific properties (focal length, field of view, optical stabilization characteristics, pose). Instead of using a uniform correction approach, the system calculates camera-specific correction factors that account for individual camera characteristics, ensuring each stream is stabilized appropriately for its unique configuration while maintaining overall system efficiency.
2Reliability
If separate stabilization corrections are calculated for each camera stream, then the stabilization quality is improved for each camera, but the computational complexity and power consumption increase
Solution Approach 1:
The patent implements preliminary action by pre-calculating and storing stabilization correction factors for each camera based on their known properties (focal length, field of view, optical stabilization parameters, pose). These pre-computed corrections are then applied during video processing without requiring complex real-time calculations, thereby maintaining high stabilization quality while reducing computational complexity and power consumption during actual operation.
3Reliability
If optical image stabilization is used for each camera, then camera shake is effectively neutralized, but the lens elements must move to different positions during capture
Solution Approach 1:
The patent uses an intermediary approach by introducing software-based correction layers that compensate for the different lens positions caused by optical image stabilization. The system calculates trajectory corrections that account for each camera's OIS movements and applies these corrections to align the video streams. This intermediary correction layer reconciles the physical lens position differences with the need for compositional consistency across multiple camera views.
Data Source
AI summary
Various techniques are disclosed herein, which comprise obtaining a first stream of images from a first camera and a second stream of images from a second camera, wherein the first camera and second camera have different poses, and at least a portion of the image streams are captured concurrently. Next, a first stabilized trajectory and corresponding set of corrections used to stabilize the first stream of images are calculated. Based on determined differences between the poses of the first camera and second camera during the capture of the first and second streams of images, a second set of corrections to be applied to the second stream of images are derived to match the first stabilized trajectory. If desired, e.g., in response to determining that the stabilized first stream of images does not meet a stabilization criterion, the second stream of images may be stabilized using the second set of corrections.