Terminal Image Super-Resolution via Sub-Image NPU Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network-based image super-resolution technologies require high computing power and are challenging to implement on terminal devices due to limitations in GPU and NPU resources, making it difficult to achieve both high frame rates and low power consumption.
Innovation Solution
A terminal device and method that splits the super-resolution processing of frames into sequential tasks on sub-images, utilizing an NPU to process each sub-image separately, with optional block splitting and varying algorithm complexity, and a GPU for post-processing, to reduce power consumption while maintaining high frame rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network-based image super-resolution technology is applied to terminal devices, then picture quality is improved, but power consumption increases and frame rate decreases due to computing power limitations
Solution Approach 1:
The patent divides a frame into multiple sub-images and processes them separately through the NPU. This segmentation reduces the computational burden on the NPU per task, allowing it to operate at lower frequency and thus reducing power consumption while still achieving super-resolution enhancement for the entire frame.
Solution Approach 2:
The patent introduces a scheduling module that dynamically coordinates between GPU and NPU operations. The system can adjust the processing pipeline based on power consumption requirements and performance needs, enabling flexible adaptation between power-saving mode and high-performance mode.
2Manufacturing precision
If neural network-based image super-resolution technology is applied to terminal devices, then picture quality is improved, but frame rate decreases due to computing power limitations
Solution Approach 1:
By segmenting frames into sub-images and processing them in parallel through the pipeline architecture, the system increases processing throughput. Multiple sub-images can be processed concurrently by the NPU, reducing the total time required to process a complete frame and thus improving frame rate.
Solution Approach 2:
The patent implements a continuous processing pipeline where the GPU prepares sub-images for the NPU, the NPU performs super-resolution processing, and results are returned to the GPU for final composition. This continuous pipeline eliminates idle time between processing stages, maximizing productivity and maintaining high frame rates.
3Productivity
If the NPU processes the entire frame at once, then processing speed is maintained, but power consumption increases
Solution Approach 1:
The patent segments the frame into multiple sub-images that are processed separately by the NPU. This reduces the memory bandwidth requirements and computational load per NPU activation, allowing the NPU to operate at lower frequency and thus reducing power consumption while maintaining overall processing speed through parallel化处理 of multiple sub-images.
Solution Approach 2:
The scheduling module implements periodic processing where the NPU processes sub-images in batches rather than continuously processing the entire frame at once. This periodic action allows the NPU to enter low-power states between processing batches, reducing average power consumption while maintaining processing throughput.
Data Source
AI summary
A terminal device includes a scheduling module configured to deliver a plurality of sub-images included in a current frame to a neural processing unit (NPU), where the current frame is a low-resolution image obtained by performing low-resolution rendering by a graphics processing unit (GPU). The terminal device further includes the NPU configured to sequentially perform super-resolution processing on the plurality of sub-images separately in a preset sequence.