Segmented Deep Learning Upscaling for Mobile Image Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image upscaling methods, whether rule-based or deep learning-based, face challenges in maintaining image quality and efficiency, especially on mobile devices with limited resources, leading to potential device slowdowns or the need for offloading processing to servers.
Innovation Solution
An electronic device efficiently processes high-resolution image upscaling using on-device deep learning models by dividing input images into multiple segments, identifying appropriate models based on image characteristics, and merging the processed segments, thereby optimizing resource utilization and reducing computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If deep learning models are used for image upscaling, then image quality is improved, but computational load and memory requirements increase significantly
Solution Approach 1:
The input image is divided into multiple divided images (e.g., quadrants or tiles) that can be processed independently. Each divided image is processed by a separate deep learning model instance or the same model sequentially, allowing the computational load to be distributed and managed within device memory constraints while maintaining overall image quality improvement.
2Manufacturing precision
If deep learning models are used for image upscaling, then image quality is improved, but device memory consumption increases
Solution Approach 1:
By segmenting the image into smaller divided images, the memory footprint required to load and process each image through the deep learning model is reduced. This allows the mobile device to accommodate the memory requirements of the deep learning model while processing high-resolution images that would otherwise exceed available memory capacity.
Solution Approach 2:
The patent processes only portions of the complete image at any given time through the deep learning model, rather than attempting to process the entire high-resolution image simultaneously. This partial processing approach enables quality enhancement within the constraints of limited device memory.
3Manufacturing precision
If the resolution of the input image is large, then image quality is improved, but the deep learning model requires large memory and slows down operation
Solution Approach 1:
Large high-resolution images are divided into smaller divided images that can be processed more quickly by the deep learning model. While multiple processing steps are required for the complete image, each individual processing step operates faster on the smaller divided images, improving overall processing throughput and preventing device shutdown.
4Use of energy by moving object
If rule-based interpolation algorithms are used, then computational load is reduced, but image quality degrades
Solution Approach 1:
The patent applies deep learning models to divided images rather than attempting to process the entire image with either rule-based methods or a single comprehensive deep learning model. This segmentation enables the use of quality-enhancing deep learning techniques while keeping computational requirements manageable through localized processing of image segments.
Data Source
AI summary
An example electronic device includes a memory; a display; and at least one processor operatively coupled to the memory and the display. The at least one processor may be configured to divide an input image into a plurality of divided images, acquire image characteristics included in each of the plurality of divided images, identify at least one deep learning model to process each of the plurality of divided images from among a plurality of deep learning models for upscaling stored in a memory on the basis of the image characteristics, acquire a plurality of upscaled segmented images corresponding respectively to the plurality of divided images through the at least one deep learning model, merge the plurality of upscaled divided images to obtain an upscaled image, and display the upscaled image on the display.