Tile-Based Super Resolution for Lower-Load Image Upscaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cloud gaming systems require computationally intensive video upscaling at the client device, particularly for high-resolution displays, leading to inefficiencies in processing power and memory usage.
Innovation Solution
A method for selective image upscaling at a client device, using neural network-based super resolution models and metadata to determine which image portions require upscaling, combined with less demanding techniques like bilinear or bicubic interpolation, and caching of previously upscaled images to improve efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If full image upscaling is applied to maintain high display resolution, then image quality is improved, but computational load and processing time increase significantly
Solution Approach 1:
The patent divides the image into multiple tiles or regions, applying upscaling processes selectively to different segments. This allows the system to process only portions of the image that require high-quality rendering, reducing overall computational load while maintaining image quality where needed.
Solution Approach 2:
Different upscaling techniques are applied to different regions of the image based on their specific requirements. Critical areas with detailed content receive higher-quality upscaling, while less important areas use simpler techniques, optimizing the balance between quality and computational efficiency.
2Ease of operation
If upscaling is applied to all image portions uniformly, then processing simplicity is maintained, but resource utilization efficiency decreases
Solution Approach 1:
The system dynamically adjusts the upscaling approach based on image content analysis. By evaluating the characteristics of different image regions, the system adapts the upscaling technique in real-time, optimizing resource utilization while maintaining operational simplicity through automated decision-making.
Solution Approach 2:
The patent changes parameters such as upscaling resolution, technique selection, and processing intensity based on image region characteristics. This allows the system to optimize resource consumption by adjusting processing parameters dynamically rather than using uniform settings across the entire image.
3Productivity
If multiple upscaling processes are applied to different image portions, then resource efficiency is improved, but system complexity increases
Solution Approach 1:
The system incorporates feedback mechanisms that analyze image characteristics and processing results to automatically select appropriate upscaling techniques. This feedback loop simplifies system complexity by automating decision-making, allowing the system to manage multiple upscaling processes efficiently without manual intervention.
Solution Approach 2:
The patent introduces intermediary components such as image analysis modules and decision-making algorithms that mediate between the image processing tasks and the upscaling operations. These intermediaries simplify the overall system architecture by centralizing the logic for selecting and managing multiple upscaling processes.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method for image upscaling at a client device is provided. The method comprising: receiving, from a server device, an image which is one of a plurality of images forming an image stream, wherein the image comprises a plurality of image portions; determining a first group of one or more image portions from the plurality of image portions to apply a first image upscaling process to from a plurality of available image upscaling processes; selecting the first group of image portions based on the determination; and applying the first image upscaling process to the first group of image portions. The upscaling process may be an image upscaling process such as super resolution.