Bilateral Image Upsampling Without Line Buffers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image upscaling algorithms for mobile devices are computationally inefficient and impractical due to limited hardware resources, leading to blurry and artifact-ridden images, and line buffers increase hardware costs.
Innovation Solution
A bilateral interpolation method that calculates spatial and intensity distances for each pixel location using weighted averages, eliminating the need for line buffers and reducing hardware requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If complex upscaling algorithms are used to improve image quality, then image detail preservation is improved, but computational complexity and hardware requirements increase
Solution Approach 1:
The patent segments the upscaling task into two distinct phases: a learning phase where a model is trained offline using high-resolution image pairs, and an inference phase where the trained model is deployed on mobile devices. This segmentation transfers computational complexity from the resource-constrained mobile device to a more powerful training system, allowing the device to only perform lightweight inference operations while still achieving high-quality upscaling results
Solution Approach 2:
The patent performs preliminary action by pre-training the neural network model offline using high-resolution image data before deployment. The model learns complex image patterns and relationships during this preliminary training phase, so that when deployed on mobile devices, it can perform upscaling with minimal computational resources while maintaining high image quality
2Manufacturing precision
If image processing is performed at high resolution, then image quality is improved, but processing time and power consumption increase
Solution Approach 1:
The patent replaces traditional mechanical/image-processing approaches (direct high-resolution processing using conventional algorithms) with a neural network-based system. The trained neural network model performs inference operations that are computationally efficient on mobile devices while producing high-quality upscaled images, effectively substituting complex iterative processing with optimized neural network computations
3Productivity
If line buffers are used to optimize hardware efficiency, then processing efficiency is improved, but hardware area cost increases
Solution Approach 1:
The patent extracts and removes the line buffer component from the hardware architecture by redesigning the data flow to use a pixel-in-pixel-out processing model. This extraction eliminates the need for storing entire image lines in buffer memory, reducing hardware area requirements while maintaining processing efficiency through immediate pixel-level computation and output
Data Source
Figure 1
Figure 2
Figure 3a~3b
AI summary
Method for bilateral interpolation of an image, including: Providing an input image to be upscaled by bilateral interpolation; Providing an initial image; Providing a guide image; For each pixel location in the final image or a corresponding pixel location in the initial image, select a region of interest, ROIspat, in the input image around the respective pixel location and determine a spatial distance dspat from the pixel location to each pixel of the input image within the ROIspat; For each pixel location in the initial image select a region of interest, ROIint, in the guide image around the respective pixel location and determine an intensity distance dint from the pixel location to each pixel of the guide image within the ROIint; Determine the pixel value at the pixel location on the basis of a weighted average over all pixels in the ROIspat based on the spatial distance and a weighted average over all pixels in the ROIint based on the intensity distance to generate a final image.