Video Rate Image Stitching via Feedback Loop
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image stitching methods for forming composite images from multiple sub-images are time-consuming and unsuitable for high-pixel-count video imagery due to environmental perturbations, focus variations, and parallax issues, which require extensive analysis and long processing times.
Innovation Solution
A sub-image stitching process that exploits the coherence between successive frames by embedding the stitching process in a feedback loop, using affine transformations and masking functions to continuously adjust geometric relations and maintain registration, allowing for rapid formation of seamless composite images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image stitching methods are used to combine multiple sub-images into a composite image, then the registration between sub-images can be achieved, but the processing time becomes excessively long (tens of minutes to hours)
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing the geometric transformation parameters (intrinsic and extrinsic parameters) for each microcamera in the array during a calibration phase. This pre-computed information is then reused during video stitching operations, eliminating the need for time-consuming feature matching and homography calculation during real-time processing, thus reducing stitching time from hours to seconds while maintaining registration accuracy
Solution Approach 2:
The patent implements dynamics by using a feedback loop that continuously adjusts the geometric relations between sub-images based on coherence information from successive video frames. The system dynamically updates the transformation parameters to maintain accurate registration as cameras move or environmental conditions change, enabling real-time stitching at video rates while preserving precision
2Measurement precision
If extensive analysis is performed to determine relative positions and deghost double images caused by parallax, then registration accuracy is improved, but processing complexity and time increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-determining and storing the extrinsic parameters (relative positions and orientations) of all microcameras during an initial calibration process. This pre-computed geometric model allows the system to directly compute sub-image transformations without performing complex real-time analysis of parallax effects or feature matching, thereby reducing processing complexity while maintaining accurate relative position determination
Solution Approach 2:
The patent implements feedback by using a closed-loop system that continuously monitors the coherence between successive video frames and adjusts the geometric transformation parameters accordingly. This feedback mechanism automatically compensates for small deviations in camera positions or orientations, maintaining registration accuracy without requiring complex manual analysis or intervention
Data Source
AI summary
Methods for stitching multiple sub-images together to form a substantially seamless composite image are disclosed. Overlap regions formed by each pair of neighboring sub-images are periodically examined and key features common to the overlap regions in each sub-image of the pair are identified. A transformation is determined for each sub-image pair based on the positions of these key features. The transformation is split between the sub-images and applied to distort the overlap regions in each sub-image pair such that they are substantially aligned. Applying the transformations to each overlap region in the overall image enables creation of a substantially seamless composite image. In some embodiments, the process wherein the transformations are determined is run as a feedback loop to enable continuing refinement of the transformations.