Panorama Seam Selection Using Saliency-Based Object Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image stitching algorithms struggle with moving subjects, leading to alignment issues, ghosting artifacts, and compromised image quality, particularly in panoramic images, due to the lack of consideration for objects of interest during seam selection and blending.
Innovation Solution
A computing device uses saliency heat maps to identify objects of interest and applies weighted penalties or boosts to seam selection, ensuring seamless integration and high-quality blending of image frames, particularly in panoramic images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional image stitching algorithms are used to combine multiple image frames, then the stitching process can be performed, but moving subjects cause alignment issues and ghosting artifacts
Solution Approach 1:
The patent applies different processing strategies to different regions of the image based on saliency heat maps. Objects with high saliency (likely moving subjects) are handled differently from background regions - specifically, the algorithm penalizes seam placements that pass through high-saliency regions, allowing local optimization of stitching quality in different areas of the image
Solution Approach 2:
The patent introduces saliency heat maps as an intermediary component that mediates between the image frames and the seam selection process. These heat maps provide additional information about likely moving subjects, which then influences the seam selection and blending operations to avoid ghosting artifacts
2Productivity
If seams are selected based on traditional overlapping region features, then stitching can proceed, but objects of interest suffer from distortions and misalignment
Solution Approach 1:
The patent performs preliminary computation of saliency heat maps for each image frame before the seam selection process. This preliminary action identifies objects of interest in advance, allowing the subsequent seam selection to avoid these regions and preserve object integrity, rather than correcting alignment issues after they occur
3Device complexity
If all image frames are processed equally during stitching, then the process is simple, but moving subjects compromise overall image quality
Solution Approach 1:
The patent modifies the uniform processing approach by introducing weightings based on saliency heat maps. Different regions of the image are processed with different weights - high-saliency regions (likely containing moving subjects) are given lower weights or special handling during seam selection and blending, while low-saliency background regions maintain standard processing
Solution Approach 2:
The patent changes the parameters of the stitching process dynamically based on saliency information. The seam selection algorithm uses penalty weights derived from saliency heat maps to modify the traditional seam scoring, effectively changing the optimization criteria to avoid placing seams through high-saliency regions
Data Source
AI summary
An example method includes receiving, by a computing device, a plurality of image frames. The method also includes determining, by the computing device, one or more objects of interest within the plurality of image frames, wherein the determining is based on saliency heat maps indicative of the one or more objects of interest. The method further includes determining at least one seam corresponding to at least one pair of successive image frames of the plurality of image frames, wherein the determining of the at least one seam is based on respective weights associated with the one or more objects of interest. The method additionally includes stitching together, by the computing device and based on the at least one seam, the at least one pair of successive image frames to generate a panorama image.