Composite Image Harmonization With Parametric Color Curves and Shadow Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image harmonization technologies face limitations in computational efficiency and user controllability, particularly in high-resolution image processing, as they often rely on pixel-wise image-to-image translation networks that lack parametric controls and are constrained to low-resolution images.
Innovation Solution
A two-stage parametric model for image harmonization using a color curves prediction model and a shadow map prediction model, trained with supervised and unsupervised methods, operates on lower-resolution images and applies predicted parameters to higher-resolution images, allowing user-adjustment of color curve parameters and shadow maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If pixel-wise image-to-image translation networks are used for image harmonization, then image quality can be improved, but computational cost increases significantly and user controllability is lost
Solution Approach 1:
The image harmonization process is segmented into two distinct stages: a refinement stage that operates on low-resolution images to capture global harmonization patterns, and a rendering stage that applies these patterns to high-resolution images. This segmentation allows the computationally intensive refinement operations to be performed on smaller images, reducing overall computational cost while maintaining high output quality.
Solution Approach 2:
The system creates a low-resolution copy of the input image for the refinement stage, processes this copy to generate harmonization parameters, and then applies these parameters to the original high-resolution image. This copying approach allows expensive computational operations to be performed on a simplified representation rather than the full-resolution data.
2Manufacturing precision
If pixel-wise image-to-image translation networks are used for image harmonization, then image quality can be improved, but user controllability deteriorates due to lack of parametric controls
Solution Approach 1:
The system transforms the image harmonization problem into a parametric control problem by learning parameters (such as color grading curves, lighting conditions, and texture characteristics) that describe the harmonization transformation. Users can directly manipulate these learned parameters to control the harmonization process, providing intuitive controllability while maintaining high image quality through the underlying neural network models.
3Productivity
If parametric model operates on lower-resolution images, then computational cost is reduced, but image quality may deteriorate
Solution Approach 1:
The system resolves the resolution conflict by operating in two different resolution dimensions at different processing stages. The refinement stage uses low-resolution images for efficient parameter learning, while the rendering stage transfers these parameters to the high-resolution domain for final image generation. This dimensional switching allows the system to benefit from both low-resolution computational efficiency and high-resolution output quality.
Data Source
AI summary
An image processing system employs a parametric model for image harmonization of composite images. The parametric model employs a two-stage approach to harmonize an input composite image. At a first stage, a color curve prediction model predicts color curve parameters for the composite image. At a second stage, the composite image with the color curve parameters is input to a shadow map prediction model, which predicts a shadow map. The predicted color curve parameters and shadow map are applied to the composite image to provide a harmonized composite image. In some aspects, the color curve parameters and shadow map are predicted using a lower-resolution composite image and up-sampled to apply to a higher-resolution version of the composite image. The harmonized composite image can be output with the predicted color curve parameters and/or shadow map, which can be modified by a user to further enhance the harmonized composite image.