Adaptive Sharpening for Upsampled Pixel Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Super resolution techniques face challenges in efficiently upsampling images, particularly in devices with limited compute and bandwidth resources, as they often require high-performance computing and result in high latency, power consumption, and silicon area usage, especially when using neural networks.
Innovation Solution
The method involves applying adaptive sharpening to blocks of upsampled pixels by determining range kernels and combining them with a sharpening kernel to create bilateral sharpening kernels, which are then used to enhance the upsampled image, reducing blurring and improving image quality without the need for intermediate storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural networks are used for upsampling, then image quality is improved, but power consumption and compute costs increase
Solution Approach 1:
The patent replaces expensive neural network models with inexpensive conventional processing algorithms (bilinear upsampling + adaptive sharpening) that can be executed efficiently on resource-constrained devices. The adaptive sharpening uses locally computed kernels that are discarded after use, avoiding the need for persistent high-performance computing resources.
Solution Approach 2:
The patent substitutes the complex computational mechanism of neural networks with simpler conventional image processing mechanisms (interpolation algorithms and convolutional sharpening filters). This replacement maintains acceptable image quality while dramatically reducing computational requirements and power consumption.
2Manufacturing precision
If neural networks are used for upsampling, then image quality is improved, but latency increases
Solution Approach 1:
The patent uses lightweight conventional algorithms that execute rapidly without the lengthy inference times characteristic of neural networks. The adaptive sharpening kernels are computed on-the-fly and applied immediately, enabling real-time processing with minimal latency.
Solution Approach 2:
The patent performs upsampling using efficient bilinear interpolation before applying adaptive sharpening, rather than relying on a single complex neural network pass. This two-stage approach with preliminary upsampling reduces overall processing time while maintaining quality.
3Manufacturing precision
If high-resolution images are rendered directly by GPU, then image quality is improved, but silicon area and compute costs increase
Solution Approach 1:
The patent segments the image processing task into two distinct stages: (1) low-resolution rendering by the GPU using conventional rasterization, and (2) super-resolution enhancement using adaptive sharpening algorithms. This segmentation allows the GPU to operate at reduced resolution, minimizing silicon area requirements while the subsequent sharpening stage restores image quality.
Solution Approach 2:
The patent merges conventional GPU rendering with post-processing super-resolution algorithms to achieve high-quality output. By combining the strengths of hardware acceleration (for the rendering stage) with adaptive image processing (for the enhancement stage), the system achieves high image quality without requiring the GPU to be designed for high-resolution direct rendering.
Data Source
AI summary
Methods and processing modules apply adaptive sharpening, for a block of input pixels for which upsampling is performed, to determine a block of output pixels. A block of upsampled pixels is obtained based on the block of input pixels. One or more range kernels is determined based on a plurality of upsampled pixels of the block of upsampled pixels. Each of the one or more range kernels is combined with a sharpening kernel to determine one or more bilateral sharpening kernels. The one or more bilateral sharpening kernels are used to determine the output pixels of the block of output pixels.