Adaptive Sampling for GPU Image Upscaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systems with limited graphics capabilities, such as integrated GPUs, struggle to perform high-quality upscaling of content due to resource constraints, often defaulting to lower quality algorithms like bilinear filtering, resulting in suboptimal image quality.
Innovation Solution
An adaptive filter approach is used to determine the appropriate number of texture samples needed per pixel based on the device's graphics capability, allowing for higher quality upscaling using filters like Catmull-Rom or Lanczos, even on lower-end hardware by reducing the number of sample taps from 9 to 3 or 6, and setting weights to 0 for unnecessary sample locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a higher quality upscaling algorithm with more sample taps (e.g., 9 taps) is used, then image quality is improved, but resource consumption exceeds the capabilities of integrated graphics cards with limited capacity
Solution Approach 1:
The system dynamically adjusts the number of sample taps based on the detected graphics capability of the client device. High-end devices with sufficient resources receive the full 9-tap Catmull-Rom filter for maximum quality, while integrated graphics cards receive a reduced 3-tap or 6-tap version. This dynamic adaptation resolves the contradiction by matching algorithm complexity to available resources.
Solution Approach 2:
The patent changes the parameter of sample tap count from a fixed value to a variable that adapts to device capabilities. By modifying this critical parameter based on detected hardware performance, the system achieves high image quality where resources permit while maintaining operational feasibility on resource-constrained devices.
2Use of energy by moving object
If the number of sample taps is reduced from 9 to 3 or 6 to accommodate limited graphics capabilities, then resource consumption is reduced, but image quality deteriorates
Solution Approach 1:
The system applies different levels of filtering quality to different devices based on their specific capabilities. Rather than using a uniform approach, each client device receives the appropriate level of upscaling quality matched to its graphics capability, ensuring optimal image quality within resource constraints.
Solution Approach 2:
The adaptive sampling approach dynamically selects the appropriate number of taps (3, 6, or 9) based on real-time detection of device graphics capability. This dynamic adjustment ensures that image quality is maximized within the resource budget of each specific device type.
3Use of energy by moving object
If a lower quality upscaling algorithm like bilinear filtering is used, then resource consumption is reduced and it can be supported by integrated graphics cards, but perceived quality of the displayed content is lower
Solution Approach 1:
The system dynamically determines the appropriate upscaling approach based on detected device capability. Rather than defaulting to bilinear filtering for all devices, the adaptive sampler identifies capable devices that can handle Catmull-Rom or Lanczos filtering, reserving simpler bilinear filtering only for devices with severely limited resources.
Solution Approach 2:
The system uses feedback from device capability detection to select the appropriate upscaling algorithm. By measuring the graphics capability of each client device and using that information to guide algorithm selection, the system avoids the quality loss associated with universal use of bilinear filtering.
Data Source
AI summary
Approaches presented herein provide for the generation, transmission, and upsampling of content for presentation using devices with varying graphics capabilities. Various computing devices can include integrated GPUs or other limited capacity hardware that may be unable to support higher performance graphics upscaler algorithms, and can thus default to using a process such as hardware-implemented bilinear filtering for upscaling, resulting in lower quality displayed images. An adaptive filter can be used to reduce the number of texture accesses needed per pixel, which can provide for improved perceptive quality and increased device support. A number of input samples to be taken for an output pixel location can be adapted to the capacity of the device to perform the upsampling, where a reduced number of samples or “taps” per pixel can significantly reduce the resource capacity needed to perform upscaling and interpolation using a Catnumm-Rom filter or Lanczos filter, implemented as a GPU shader, and allow higher quality upscaling on limited capacity systems.