AI Image Zoom Using Pixel-Frequency Joint Refinement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image zooming techniques suffer from artifacts such as jaggy edges, blurred details, and washed-out appearances, and deep neural networks (DNNs) require excessive computational resources, making them impractical for imaging systems with limited resources.
Innovation Solution
Employing Edge Synthesis Blocks (ESBs) with One Dimensional (1D) Convolution Neural Networks (CNNs) for image processing, analyzing features in multiple domains, and using a joint refinement neural network to generate high-resolution images with reduced computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional image processing techniques (bilinear upscaling, bicubic upscaling) are used for digital zooming, then the imaging system can zoom in and out without physically focusing using lenses, but artifacts such as jaggy edges, blurred details, overly smoothened edges, and washed-out appearances occur in the zoomed image
Solution Approach 1:
The patent replaces conventional mechanical/image processing zooming techniques (bilinear, bicubic upscaling) with an AI-based neural network approach. The neural network is trained to recognize and preserve edge patterns, replacing traditional interpolation methods that produce artifacts with intelligent pattern recognition that maintains image quality during digital zooming.
Solution Approach 2:
The patent changes the approach from fixed interpolation parameters to adaptive neural network parameters. The neural network learns optimal parameter values during training by analyzing edge patterns and gradient information, allowing dynamic adjustment of zooming parameters based on local image characteristics rather than applying uniform interpolation across the entire image.
2Manufacturing precision
If Deep Neural Network (DNN) is used for producing high quality zoomed images, then image quality is improved, but computational complexity becomes very high (more than 1 million operations per pixel), which is not feasible for imaging systems with limited computational resources
Solution Approach 1:
The patent segments the image processing task by focusing the neural network's attention on specific critical features (edges and gradients) rather than processing all pixel information equally. By identifying and prioritizing edge regions through gradient calculation and edge pattern recognition, the network processes only the most important image features, significantly reducing computational complexity while maintaining quality.
Solution Approach 2:
The patent extracts and isolates the most critical image features (edges and gradients) before applying the neural network. By calculating gradients and identifying edge patterns in advance, the network receives pre-processed input that contains only the essential information needed for quality zooming, eliminating the need to process redundant pixel data and reducing overall computational burden.
3Manufacturing precision
If the neural network processes all image data uniformly, then comprehensive image features are considered, but computational resources are wasted on regions with less critical details
Solution Approach 1:
The patent applies different processing strategies to different regions of the image based on their local characteristics. Edge regions identified through gradient calculation receive focused neural network processing with higher computational attention, while smooth regions with less critical details use simpler processing methods. This localized quality approach ensures comprehensive feature analysis where needed while improving overall processing efficiency.
Data Source
AI summary
A method and an electronic device for performing an AI based zoom of an image in an electronic device are provided. The method includes receiving the image; obtaining, through a pixel domain neural network (NN) block, a first set of feature maps of the image based on pixels of the image; obtaining, through a frequency domain NN block, a second set of feature maps of the image based on frequencies of the image; and obtaining, through a joint refinement NN block, a final image with a resolution higher than a resolution of the image, based on the first set of feature maps and the second set of feature maps.