Mobile Camera Panorama Stitching with Object-Aware Frame Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image stitching processes often select base frames containing low-quality representations of objects of interest, leading to reduced image fidelity and noticeable distortions, particularly when seams are placed over these objects.
Innovation Solution
The process intelligently selects base frames by considering the quality of objects of interest within candidate frames and penalizes seams placed on these regions during the blending phase, ensuring each region of interest is fully contained within a selected base frame and applying a computational bias to minimize seam placement on these areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional stitching algorithms select base frames from overlapping regions, then the stitching process is simple and fast, but the quality of objects of interest is degraded due to low-quality representations and seam artifacts
Solution Approach 1:
The patent applies preliminary action by performing frame quality assessment and region of interest detection before the stitching process. The system pre-identifies high-quality frames containing complete regions of interest and pre-calculates seam placement strategies to avoid these regions, thereby ensuring high image fidelity from the outset without compromising stitching simplicity
Solution Approach 2:
The patent implements local quality by differentiating between different regions within frames during selection. Instead of evaluating frames uniformly, the system specifically identifies frames where regions of interest are fully contained and of high quality, giving priority to local quality metrics in critical areas while maintaining overall stitching efficiency
2Reliability
If seams are placed in overlapping regions during blending, then the stitching process is efficient and fast, but objects of interest suffer from distortions and artifacts
Solution Approach 1:
The patent applies local quality by implementing differential seam placement strategies. The system identifies regions of interest within the panorama and specifically avoids placing seams through these regions, even though this requires more complex calculations. This ensures that objects of interest maintain high quality without severe distortions or artifacts
Solution Approach 2:
The patent uses an intermediary approach by introducing a seam placement optimization layer between the blending process and the final composite image. This intermediary step calculates optimal seam locations that avoid regions of interest, acting as a mediator that preserves object quality while still achieving efficient blending in non-critical areas
3Reliability
If all captured frames are used for stitching, then the coverage area is maximized, but the quality of the composite image is reduced due to inclusion of low-quality frames
Solution Approach 1:
The patent implements local quality by evaluating frames based on the quality of specific regions within them rather than overall frame quality. Frames are selected if they contain fully-contained regions of interest at high quality, even if this reduces the total number of frames used, thereby prioritizing local quality over maximum coverage
Solution Approach 2:
The patent applies taking out by extracting and prioritizing high-quality regions of interest from the captured frames. The system identifies and extracts frames that completely contain regions of interest at high quality, separating these from lower-quality frames that would degrade the composite image, even if it means excluding some coverage areas
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Devices and methods for selecting and stitching image frames are provided. A method includes obtaining a plurality of image frames. The method also includes identifying one or more regions of interest within one or more image frames in the plurality of image frames. The method further includes selecting, based on a respective quality measure associated with each image frame of the plurality of image frames, a set of base frames, where each identified region of interest of the one or more identified regions of interest is fully contained within at least one base frame in the selected set of base frames. The method additionally includes stitching together the selected set of base frames to create a composite image.