Content-Aware Image Resizing via Anchor Point Sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image and video resizing techniques fail to effectively preserve important content when resizing images or videos for different display devices and viewing conditions, as they are often content-neutral and result in artifacts, high computational costs, and misordering of pixels.
Innovation Solution
The method involves determining pixel saliency, selecting anchor points through saliency-based sampling, and using efficient pixel mapping with iterative adaptation to constrain pixels within boundaries, allowing for global optimization of image or video resizing while preserving spatial and temporal ordering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If standard image scaling is used, then the resizing process is simple and fast, but the important content in the image is not preserved and artifacts appear
Solution Approach 1:
The image is segmented into important and unimportant regions based on saliency detection. Only the important regions are preserved during resizing, while unimportant regions are allowed to be distorted or removed. This segmentation approach enables content-aware resizing that preserves important content without requiring complex global optimization.
Solution Approach 2:
Different regions of the image are treated differently during the resizing process. Important regions identified by high saliency values are preserved with higher fidelity, while unimportant regions are allowed to be more aggressively transformed. This local quality approach ensures that content preservation is focused where it matters most.
2Measurement precision
If pixel mapping is performed for all pixels, then the mapping accuracy is high, but the computational cost is extremely high
Solution Approach 1:
The patent extracts and processes only the most important pixels (those with high saliency values) rather than all pixels. By taking out the critical pixels and using them as anchor points for mapping, the system achieves acceptable mapping accuracy with significantly reduced computational cost compared to processing every pixel.
Solution Approach 2:
Instead of performing complete pixel mapping for all pixels, the patent applies partial action by mapping only the essential anchor pixels first, then using interpolation to estimate the positions of remaining pixels. This partial approach provides sufficient accuracy for content preservation while dramatically improving processing speed.
3Duration of action of moving object
If fast-forwarding or slow-down schemes are used for video resizing, then the runtime can be adjusted, but the important events in the video are altered
Solution Approach 1:
The patent performs preliminary saliency detection and important event identification before performing temporal resizing. By预先 identifying which regions and events are important, the system can then apply intelligent sampling and duplication strategies that preserve these events during runtime adjustment, rather than blindly applying uniform fast-forwarding or slow-down.
Solution Approach 2:
The patent changes the sampling rate parameter dynamically based on the importance of different video regions and events. Rather than using a fixed sampling rate for all frames, the system adjusts the sampling parameter to maintain important events at their original temporal density while allowing less important regions to be compressed or expanded.
4Area of stationary object
If cropping is used to remove pixels from the image periphery, then the image size is reduced, but the important content may be removed and the image quality deteriorates
Solution Approach 1:
The patent applies local quality by differentiating between important and unimportant regions during the resizing process. Rather than uniformly cropping from all regions, the system identifies important content locations and ensures they are preserved, while allowing non-critical peripheral regions to be cropped or transformed more aggressively to achieve the target size.
Data Source
AI summary
Aspects of the present invention include systems and methods for resizing a set of images, which may comprises one or more images, while preserving the important content. In embodiments, the saliency of pixels in the set of images is determined using one or more image features. A small number of pixels, called anchor points, are selected from the set of images by saliency-based sampling. The corresponding positions of these anchor points in the set of target images are obtained using pixel mapping. In embodiments, to prevent mis-ordering of pixel mapping, an iterative approach is used to constrain the mapped pixels to be within the boundaries of the target image/video. In embodiments, based on the mapping of neighboring anchor points, other pixels in the target are inpainted by back-projection and interpolation. The combination of sampling and mapping greatly reduces the computational cost yet leads to a global solution to content-aware image/video resizing.