Multi-Camera Zoom Transition Alignment Using Image Feature Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-camera systems in computing devices experience perceptible image distortions, such as binocular disparity, during camera transitions due to changes in field of view, which are not adequately addressed by existing geometry-based approaches that fail to correct errors from Voice Coil Motors (VCM), optical image stabilization (OIS) adjustments, and thermal effects.
Innovation Solution
An image-based approach that utilizes image features to correct geometric metadata errors by performing bundle adjustment and recalibration, combining geometric and image-based warping transformations to reduce viewing artifacts during camera switches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If geometry-based warping transformation is used for camera transition, then the transition speed is fast, but image alignment accuracy deteriorates due to errors from VCM, OIS, and thermal effects
Solution Approach 1:
The patent implements a feedback mechanism where image features from the captured frames are used to detect misalignment caused by VCM, OIS, and thermal effects. The system then adjusts the warping transformation parameters based on this feedback to compensate for the geometric metadata errors, achieving accurate alignment despite the fast transition speed.
Solution Approach 2:
The patent dynamically changes the warping transformation parameters by combining geometry-based transformations with image-based corrections. It adjusts the transformation matrix elements based on detected feature mismatches, allowing the system to adapt to geometric metadata errors while maintaining fast transition performance.
2Measurement precision
If image-based visual features are used to correct geometric metadata, then image alignment accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent applies partial image-based correction by using it only to adjust specific parameters of the warping transformation rather than performing a complete image-based registration. This selective approach reduces computational complexity while still achieving improved alignment accuracy for the most critical error sources.
Solution Approach 2:
The patent performs preliminary geometry-based warping transformation to achieve rough alignment before applying image-based feature matching for fine-tuning. This two-stage approach reduces the computational burden of image-based methods by limiting their scope to correcting residual errors rather than performing full alignment.
3Reliability
If bundle adjustment is performed to recalibrate geometric metadata, then viewing artifacts are reduced, but processing time increases
Solution Approach 1:
The patent performs bundle adjustment as a preliminary calibration step to establish accurate geometric metadata before camera transitions occur. This pre-calibration approach reduces viewing artifacts during transitions without adding processing time during the actual transition, as the computational heavy lifting is done in advance.
Solution Approach 2:
The patent applies partial bundle adjustment by focusing on correcting only the specific geometric parameters that cause viewing artifacts (such as focal length and principal point offsets) rather than performing a complete recalibration of all camera parameters. This selective correction reduces processing time while maintaining effectiveness in reducing viewing artifacts.
Data Source
AI summary
An example method includes displaying an initial preview of a scene being captured by a first camera operating within a first range of focal lengths. The method includes detecting a zoom operation predicted to cause the first camera to reach a limit of the first range. The method includes activating a second camera, operating within a second range of focal lengths, to capture a zoomed preview of the scene. The method includes updating a geometry-based warping transformation based on a comparison of respective image features from the initial preview and the zoomed preview. The method includes aligning the zoomed preview with the initial preview by applying the updated warping transformation. The method includes displaying the aligned zoomed preview of the image captured by the second camera while operating within the second range.