Attention-driven Image Salience Adjustment via Global Parametric Edits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for adjusting image salience are either time-consuming and prone to errors due to manual adjustments or result in unrealistic artifacts when using automated approaches like GANs, and require a target saliency map, making them impractical for interactive applications or high-resolution images.
Innovation Solution
An automated deep neural network-based method that predicts global parametric image edits for the foreground and background regions of an image, optimizing salience without a target saliency map, using a novel loss function with a salience and adversarial component to ensure realism and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual adjustment of pixel brightness or color is used to increase salience, then salience adjustment can be achieved, but the process becomes time-consuming and burdensome
Solution Approach 1:
The patent replaces manual mechanical adjustment of pixel parameters with an automated system that uses machine learning models and optimization algorithms to automatically adjust image parameters for salience enhancement, eliminating the time-consuming manual process while maintaining precision
Solution Approach 2:
The system enables self-service automation where the computer automatically performs salience adjustment without human intervention by analyzing the image, identifying target objects, and applying optimized parameter adjustments based on pre-trained models and algorithms
2Extent of automation
If GAN with encoder-decoder generator is used for automated saliency adjustment, then automation is achieved, but artifacts reduce the realism of generated images
Solution Approach 1:
The patent changes the approach from generating entire images to adjusting specific image parameters (brightness, contrast, saturation, etc.) of identified regions, which maintains realism by applying subtle, physically-based adjustments rather than generating new pixel data that can introduce artifacts
Solution Approach 2:
The system applies different parameter adjustments to different regions of the image based on their salience importance, with the target object receiving enhancement parameters and background regions receiving suppression parameters, maintaining local realism while achieving global salience adjustment
3Extent of automation
If GAN-based automated adjustment is used, then automation is achieved, but a target saliency map is required as input making it impractical when unavailable
Solution Approach 1:
The system performs preliminary action by pre-training machine learning models to automatically identify objects and estimate their salience without requiring target saliency maps as input, enabling the system to function autonomously with only the original image as input
Solution Approach 2:
The patent extracts and removes the requirement for target saliency map input by using alternative approaches such as object detection models and saliency estimation algorithms that can infer salience directly from the image content, simplifying the input requirements while maintaining automation
4Measurement precision
If patch synthesis algorithm is used for salience adjustment, then desired salience can be achieved, but separate optimization for each image consumes significant time making it infeasible for interactive applications
Solution Approach 1:
The system performs preliminary action by pre-training machine learning models on large datasets of images and their corresponding salience annotations, so that when processing new images, the pre-trained models can quickly predict salience and optimal parameters without requiring time-consuming optimization for each individual image
Solution Approach 2:
The patent creates a universal solution by training models that can generalize across different images, objects, and scenarios, allowing the same pre-trained model to efficiently process various types of images for salience adjustment without requiring image-specific optimization, thereby achieving both precision and speed
Data Source
AI summary
Techniques of adjusting the salience of an image include generating values of photographic development parameters for a foreground and background of an image to adjust the salience of the image in the foreground. These parameters are global in nature over the image rather than local. Moreover, the optimization of the salience over such sets of global parameters is provided through two sets of these parameters by an encoder: one set corresponding to the foreground, in which the salience is to be either increased or decreased, and the other set corresponding to the background. Once the set of development parameters corresponding to the foreground region and the set of development parameters corresponding to the background region have been determined, a decoder generates an adjusted image with an increased salience based on these sets of development parameters.