YUV Image Enhancement via CPU-NPU Segmentation for Real-Time Mobile Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image enhancement methods for mobile terminals fail to meet real-time video processing requirements due to high calculation loads and power consumption, especially when using deep learning and machine learning-based super-resolution algorithms, which are not optimized for the YUV color space used in smartphones.
Innovation Solution
An image enhancement method that performs super-resolution processing and high dynamic range imaging on the YUV color space, utilizing both CPU and NPU for efficient processing, where the CPU enlarges U and V channel components and the NPU performs super-resolution and high dynamic range processing on the Y channel, using a two-branch multitask neural network model to enhance image quality in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If deep learning and machine learning-based super-resolution algorithms are used for image enhancement, then image quality is improved, but calculation amount increases and real-time processing requirement cannot be met
Solution Approach 1:
The patent segments the image enhancement process into two distinct modules: a super-resolution module that performs up-sampling and detail restoration, and a tone mapping module that handles dynamic range compression. This segmentation allows each module to be optimized independently, with the super-resolution module focusing on resolution enhancement and the tone mapping module focusing on real-time processing and color space conversion, thereby achieving both high image quality and real-time processing performance.
Solution Approach 2:
The patent changes the color space parameter from RGB to YUV for the mobile platform implementation. By performing super-resolution and tone mapping in the YUV color space and then converting to RGB only for final display, the patent reduces calculation complexity while maintaining image quality. This parameter change enables real-time processing on mobile devices by avoiding complex RGB-based deep learning computations.
2Manufacturing precision
If deep learning-based super-resolution algorithms are used for image enhancement, then image quality is improved, but power consumption increases
Solution Approach 1:
The patent segments the computation-intensive deep learning tasks into a super-resolution module and a tone mapping module, allowing the mobile device to use efficient implementations for each. The super-resolution module uses optimized up-sampling algorithms, and the tone mapping module uses efficient YUV color space processing, reducing overall power consumption compared to running full RGB-based deep learning models.
Solution Approach 2:
By changing the color space parameter to YUV and performing all heavy computations in this color space, the patent reduces the computational burden and associated power consumption. The YUV color space allows for more efficient processing in mobile devices, and the final RGB conversion is performed only when necessary for display, minimizing energy-intensive operations.
3Adaptability or versatility
If current image enhancement methods are used, then they work for RGB color space, but they are not applicable to YUV color space used in smartphones
Solution Approach 1:
The patent changes the color space parameter from RGB to YUV throughout the image enhancement pipeline. The super-resolution module and tone mapping module both operate in YUV color space, which is the standard for mobile video processing. This parameter change makes the enhancement method directly applicable to smartphone workflows while maintaining compatibility with the YUV color space used in mobile devices.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This application provides an image enhancement method and an electronic device, to implement image enhancement, and resolve a problem, in the conventional technology, that a real-time requirement of a mobile terminal for video processing is not met by using an image quality algorithm. The method includes: The electronic device obtains a color image in YUV color space, a CPU of the electronic device performs image size enlargement on a first U-channel component and a first V-channel component of the color image, to obtain a processed second U-channel component and a processed second V-channel component, and an NPU of the electronic device performs super-resolution processing on a first Y-channel component of the color image, to obtain a processed second Y-channel component; the NPU of the electronic device performs high dynamic range imaging processing on the color image, to obtain component coefficients respectively corresponding to the three channel components; and finally, the CPU of the electronic device obtains an enhanced color image through combination based on the component coefficients corresponding to the three channels and the processed channel components.