Image Super-Resolution Scheduling for High-Frame-Rate Terminals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network-based image super-resolution technologies require high computing power and are difficult to implement on terminal devices due to limitations in GPU and NPU resources, making it challenging to achieve both high frame rates and low power consumption.
Innovation Solution
A terminal device and method that splits the super-resolution processing of frames into sequential tasks on sub-images, using a scheduling module to manage the NPU's workload, allowing for reduced power consumption while maintaining high frame rates by optimizing the super-resolution processing algorithm complexity and incorporating block splitting and postprocessing techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network based image super-resolution technology is applied to terminal devices, then picture quality can be improved, but power consumption increases due to high computing power requirements
Solution Approach 1:
The patent divides each frame into multiple sub-images and processes them separately through the NPU in a sequential pipeline manner. This segmentation reduces the computational burden on the NPU for each individual task, allowing the processor to operate more efficiently with lower power consumption while maintaining overall picture quality through the accumulation of processed sub-images.
2Manufacturing precision
If neural network based image super-resolution technology is applied to terminal devices, then picture quality can be improved, but frame rate decreases due to computing power limitations
Solution Approach 1:
By segmenting frames into sub-images and processing them in a pipeline, the system can overlap computation with other operations, improving overall throughput and frame rate despite the computational intensity of neural network processing.
Solution Approach 2:
The patent performs low-resolution rendering on the GPU first, preparing preliminary data that reduces the computational load required for subsequent super-resolution processing on the NPU. This preliminary action enables the system to maintain higher frame rates by pre-processing data in a computationally efficient manner before applying the more intensive neural network algorithms.
3Productivity
If discrete graphics card is configured to achieve high frame rate, then productivity can be improved, but device complexity increases
Solution Approach 1:
The patent enables the terminal device's integrated GPU and NPU to work together in a coordinated pipeline for super-resolution processing, making the existing components perform multiple functions. The GPU handles low-resolution rendering while the NPU performs super-resolution on sub-images, eliminating the need for a discrete graphics card while maintaining high frame rates and reducing device complexity.
Data Source
Figure 1~2
Figure 3a~3b
Figure 4~5
AI summary
This application discloses a terminal device and an image super-resolution method. The terminal device includes: a scheduling module, configured to deliver a plurality of sub-images included in a current frame to a neural-network processing unit NPU, where the current frame is a low-resolution image obtained by performing low-resolution rendering by a graphics processing unit GPU; and the NPU, configured to sequentially perform super-resolution processing on the plurality of sub-images separately in a preset sequence. According to this application, both a high frame rate and low power consumption can be implemented on the terminal device.