Frequency-Map Upscaling for Sharpness and Detail Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional ML-based image super-resolution upscaling lacks direct control over the amount of sharpness and details in the output, failing to enhance images beyond the quality of the reference ground truth image.
Innovation Solution
Applying a Fourier Transform function on the brightness channel of an input image to generate a 2-D frequency map, adjusting this map with a gain factor to enhance higher frequency areas, and using the adjusted map as an additional input channel for ML-based upscaling to control the target sharpness and details in the output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional ML-based upscaling is used to match the reference ground truth image, then the output quality closely matches the reference, but the sharpness and details cannot be enhanced beyond the reference quality
Solution Approach 1:
The patent transforms the image from spatial domain to frequency domain using Fourier Transform, adding a new dimension of control. By operating in the frequency domain and manipulating the 2-D frequency map, the system can independently control sharpness and details without being constrained by the reference image quality, thus resolving the contradiction between matching reference quality and enabling enhanced sharpness control
Solution Approach 2:
The patent introduces frequency domain parameters (2-D frequency map with adjustable gain factors) to control the upscaling process. By changing the frequency domain parameters, specifically applying gain factors to different frequency regions, the system can independently adjust sharpness and details to exceed the reference image quality while maintaining the ability to match it when needed
2Manufacturing precision
If the ground truth image lacks preferred sharpness or details, then the training process cannot add additional details beyond what exists in the ground truth
Solution Approach 1:
The patent performs preliminary action by computing the 2-D frequency map of the ground truth image before the upscaling training process. This frequency map serves as a guide that preserves and emphasizes high-frequency information, allowing the training process to recover and enhance details that would otherwise be lost, enabling the system to add details beyond the original ground truth quality
Solution Approach 2:
The 2-D frequency map acts as an intermediary between the ground truth image and the upscaling process. It mediates the transfer of high-frequency information by providing a frequency-domain representation that highlights areas needing detail enhancement, enabling the system to recover lost details and add new information that exceeds the original ground truth quality
3Ease of operation
If no frequency domain adjustment is applied, then the processing is simpler, but there is no direct control over the amount of sharpness and details in the output
Solution Approach 1:
The patent applies local quality by allowing different gain factors to be applied to different regions of the 2-D frequency map. This enables selective enhancement of specific frequency components (e.g., emphasizing edges while preserving smooth areas), providing direct and flexible control over sharpness and details in different parts of the image without requiring complex global processing
Data Source
Figure 1A~1C
Figure 2A~2C
Figure 3
AI summary
Enabling adjustment of sharpness and details of an input image in upscaling, including: applying a Fourier transform function on a brightness channel of the input image to generate a 2-D frequency map; adjusting the 2-D frequency map to control a target amount of sharpness and details in an upscaled output image; and using the adjusted 2-D frequency map as an additional input channel along with standard color image data for a training and upscaling process.