Upscaling Images with Estimated Downscaling Kernels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Media content downscaled using an unknown downscaling kernel cannot be effectively upscaled without introducing significant visual artifacts, degrading the visual quality and leading to poor viewing experiences in media production pipelines.
Innovation Solution
A scaling application that estimates the downscaling kernel using a kernel mapping network, generates a degradation map, and iteratively optimizes the kernel to minimize visual artifacts through a parameter optimizer, allowing for upscaled images with minimal artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If downscaling is performed using an unknown downscaling kernel, then the media content can be processed at lower resolution, but visual artifacts are introduced during upscaling that degrade quality
Solution Approach 1:
The system performs preliminary estimation of the downscaling kernel before upscaling operations. By predicting the kernel parameters in advance using the downscaled image as input, the system prepares the necessary information to guide the upscaling process, thereby preventing artifact introduction while maintaining processing efficiency
Solution Approach 2:
The system employs feedback mechanisms where the estimated downscaling kernel is iteratively refined based on the relationship between downscaled and upscaled images. This feedback loop allows the system to adjust kernel parameters to minimize visual artifacts, resolving the contradiction between processing efficiency and image quality
2Manufacturing precision
If the downscaling kernel is known, then upscaling can be performed without visual artifacts, but the system cannot handle cases where downscaling was performed with unknown kernels
Solution Approach 1:
The system performs self-service by automatically estimating the unknown downscaling kernel from the available downscaled image without requiring external information about the original downscaling process. This self-contained approach enables the system to handle unknown kernels while maintaining image quality through accurate kernel prediction
Solution Approach 2:
The system changes the approach from requiring known kernel parameters to estimating them dynamically. By transforming the problem into one of kernel prediction using neural networks, the system adapts to unknown kernels while preserving image quality through learned parameter optimization
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A scaling application estimates a downscaling kernel used to generate a downscaled image. The scaling application upscales the downscaled image based on the estimated downscaling kernel, thereby generating a higher resolution version of the downscaled image with minimal visual artifacts. The scaling application includes various networks that perform the above operations. A kernel mapping network generates a degradation map based on the estimated downscaling kernel. A degradation-aware generator network generates a reconstructed image based on the downscaled image and the degradation map. A kernel discriminator network generates an image delta that reflects visual artifacts present in the reconstructed image. The scaling application includes a parameter optimizer that iteratively modifies the estimated downscaling kernel to reduce visual artifacts indicated in the image delta. Via one or more iterations, the scaling application generates a reasonably accurate estimation of the downscaling kernel and can then upscale the downscaled image with reduced visual artifacts.