Diagonal Kernel Upsampling for Image Gradient Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Super resolution techniques face challenges in efficiently upsampling images, particularly in devices with limited compute and bandwidth resources, as existing methods require high-performance computing and result in high latency, power consumption, and silicon area usage.
Innovation Solution
A method involving determining image gradients to calculate weighting parameters for a diagonal kernel, which is applied to input pixels to generate upsampled pixels, including diagonal pixels not in the original image grid, thereby reducing blurring and staircasing artifacts while optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a neural network is used to upsample images, then image quality is improved, but processing time, power consumption, and silicon area increase significantly
Solution Approach 1:
The patent replaces expensive neural network models with inexpensive, simple upsampling algorithms (bilinear, bicubic, Lanczos) that can be executed quickly on resource-constrained devices. These simple algorithms sacrifice some quality compared to neural networks but provide acceptable results with minimal computational overhead, making them suitable for mobile and embedded systems.
Solution Approach 2:
The patent changes the approach from using complex learned parameters (neural network weights) to using fixed mathematical parameters of traditional upsampling algorithms. By adjusting parameters like kernel size and interpolation method in these simple algorithms, the system achieves a balance between quality and efficiency without requiring high-performance computing resources.
2Manufacturing precision
If a neural network is used to upsample images, then image quality is improved, but power consumption and silicon area increase
Solution Approach 1:
The patent substitutes power-hungry neural network computations with energy-efficient traditional upsampling algorithms. These algorithms require minimal computational resources and can be implemented with simple mathematical operations that consume significantly less power, making them ideal for battery-operated mobile devices.
Solution Approach 2:
The patent replaces the complex 'mechanical system' of neural network computation with simpler mathematical interpolation methods. This substitution eliminates the need for large-scale parallel processing hardware, reducing both silicon area and power consumption while maintaining acceptable image quality for practical applications.
3Productivity
If conventional upsampling methods are used, then processing efficiency is improved, but image quality deteriorates due to blurring and staircasing artifacts
Solution Approach 1:
The patent introduces adaptive sharpening that dynamically adjusts the sharpening strength based on local image characteristics. The system analyzes edge orientations and gradients in different regions and applies customized sharpening parameters to preserve edges while reducing artifacts, thereby improving image quality without sacrificing the efficiency of conventional upsampling methods.
Solution Approach 2:
The patent applies different processing strategies to different regions of the image based on local characteristics. In edge regions, it uses orientation-aware sharpening with reduced strength to avoid artifacts, while in flat regions, it applies stronger sharpening to enhance detail. This local adaptation improves overall image quality while maintaining processing efficiency.
Data Source
AI summary
Methods and processing modules upsample a block of input pixels to determine a block of upsampled pixels. At least one of the upsampled pixels is a diagonal pixel, wherein a diagonal pixel is at a position that is not in any of the rows nor in any of the columns of input pixels in the block of input pixels. Indications of image gradients are determined for the block of input pixels. The determined indications of image gradients are used to determine one or more weighting parameters which are indicative of weights of a diagonal kernel. The upsampled pixels of the block of upsampled pixels are determined by applying kernels to the block of input pixels, wherein the diagonal pixel in the block of upsampled pixels is determined by applying the diagonal kernel to the block of input pixels in accordance with the determined one or more weighting parameters.