Low-Power Display Image Upscaling With Compact Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The power consumption and performance of compute devices are heavily influenced by display resolution, with higher resolutions requiring more power and resources, while lower resolutions compromise user experience.
Innovation Solution
Implementing a machine-learning-based algorithm using a timing controller with a compact neural network architecture that reduces frame resolution before upscaling, leveraging low-power circuitry and pipelining to minimize memory usage and bandwidth, thereby reducing power consumption and increasing frame rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If display resolution is increased, then user experience is improved, but power consumption and compute resource requirements increase
Solution Approach 1:
The upscaling process is divided into multiple stages: first downsampling the high-resolution frame to a lower resolution, then applying machine learning-based upscaling to reconstruct the high-resolution image. This segmentation allows the system to process images at lower computational cost while achieving the desired high-resolution output.
Solution Approach 2:
A machine learning-based upscaler acts as an intermediary between the low-resolution processed frame and the final high-resolution display output. This intermediary component uses neural networks to intelligently reconstruct details, achieving high-resolution output without requiring continuous high-power computation throughout the entire pipeline.
2Measurement precision
If display resolution is increased, then user experience is improved, but compute resource requirements increase
Solution Approach 1:
The compute pipeline is segmented into a rendering stage (operating at lower resolution with fewer resources) and an upscaling stage (using machine learning to enhance resolution). This division allows the system to achieve high-resolution output while keeping the computational burden manageable during the resource-intensive upscaling phase.
Solution Approach 2:
Traditional computational geometry-based upscaling methods are replaced with machine learning-based neural networks. This substitution uses trained models to perform upscaling operations, which are more efficient and require fewer compute resources compared to conventional algorithmic approaches.
3Use of energy by moving object
If frame resolution is reduced, then power consumption decreases, but image quality deteriorates
Solution Approach 1:
The system uses feedback from the machine learning upscaler to reconstruct high-frequency details and textures that are lost during downsampling. The neural network analyzes the low-resolution input and generates enhanced high-resolution output, effectively compensating for the quality loss through learned patterns and relationships.
Solution Approach 2:
The system dynamically adjusts the resolution parameter during the frame processing pipeline. By temporarily reducing resolution for processing and then using machine learning to restore it, the system achieves both low power consumption during computation and high image quality for display, effectively decoupling these two parameters.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques for upscaling display image resolution are disclosed. In an illustrative embodiment, a component of a compute device, such as a graphics processing unit (GPU), sends frames to a display module at a first resolution, and the display module upscales the frame to a second, higher resolution. To do so, the display module implements a low-power machine-learning-based algorithm, which can perform high-quality upscaling. Generating the frames at the GPU at a lower resolution can save significantly more power than the display module uses to implement the machine-learning-based algorithm, reducing the overall power of the compute device.