Image Harmonization via Low-Resolution Parameter Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image harmonization technologies using machine learning models face high calculation costs, limiting their application to low-resolution images, as the cost increases with image resolution, making it impractical for high-resolution image processing.
Innovation Solution
An image processing system that acquires a high-resolution image and a mask image, down-samples them to lower resolution, uses a parameter map estimation model to generate a low-resolution parameter map, and then up-samples it to the original resolution to calculate color-adjusted pixel values using a transformation function, reducing the computational burden and enabling high-resolution image harmonization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a machine learning model directly estimates a harmonized image from a high-resolution composite image, then the image harmonization quality is improved, but the calculation cost increases significantly
Solution Approach 1:
The patent segments the image processing into two distinct stages: (1) generating a parameter map at low resolution using a machine learning model, and (2) applying this parameter map to the high-resolution image through up-sampling and transformation functions. This segmentation allows the computationally intensive ML model to operate only on down-sampled images while still achieving high-resolution output quality.
Solution Approach 2:
The patent introduces a parameter map as an intermediary element between the low-resolution ML processing and the high-resolution output. This parameter map, generated at low resolution and then up-sampled, serves as a mediator that guides the color adjustment of the high-resolution image without requiring the ML model to directly process high-resolution data.
2Use of energy by moving object
If a machine learning model processes low-resolution images for image harmonization, then the calculation cost is reduced, but the application is limited to low-resolution images
Solution Approach 1:
The patent changes the resolution parameter of the input image to the machine learning model by down-sampling high-resolution images to low resolution before processing. This parameter change enables the ML model to operate efficiently while the subsequent up-sampling and transformation steps restore the high-resolution output, effectively decoupling the model's operational resolution from the final output resolution.
Data Source
AI summary
Provided is an image processing system with at least one processor being configured to: calculate a color-adjusted pixel value vector, which is output of a transformation function defined by a formula including a term of a product of a pixel value vector and a transformation matrix, wherein the pixel value vector indicates a color of each of one or more adjustment pixels, which are at least the one or more target pixels out of pixels of the first image, and the transformation matrix has, as elements, a plurality of transformation parameters set to each of one or more adjustment pixels in a first parameter map, which is the second parameter map increased in resolution and which has the first resolution; and acquire a color-adjusted image including a color-adjusted pixel, the color-adjusted pixel having a color indicated by the color-adjusted pixel value vector.