AI Image Upscaling Codec for Low-Bitrate High-Resolution Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-resolution image encoding and decoding requires significant bitrate, and existing methods struggle to efficiently manage this process.
Innovation Solution
A method and apparatus using artificial intelligence (AI) for image down-scaling and up-scaling, employing deep neural networks (DNNs) trained jointly to reduce bitrate by first encoding and decoding images at a lower resolution, followed by AI up-scaling to the original resolution, utilizing AI data to maintain the joint relationship between down-scaling and up-scaling processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-resolution images are encoded and decoded using conventional codecs, then image quality is maintained, but bitrate requirements increase significantly
Solution Approach 1:
The encoding process is segmented into two stages: first encoding the down-scaled low-resolution image, then separately processing up-scaling parameters. The decoding process similarly separates reconstruction of the low-resolution image from the AI-based up-scaling operation. This segmentation allows bitrate to be spent on essential image content rather than redundant high-frequency details that AI can reconstruct.
Solution Approach 2:
The image is pre-processed by down-scaling to a lower resolution before encoding. This preliminary action reduces the amount of data that needs to be encoded and transmitted, while the AI up-scaling model is prepared in advance to reconstruct the high-resolution version during decoding, maintaining quality without the original bitrate burden.
2Quantity of substance
If images are down-scaled before encoding, then bitrate is reduced, but image quality may deteriorate
Solution Approach 1:
An AI-based up-scaling model acts as an intermediary between the encoded low-resolution image and the final high-resolution output. This intermediary uses learned patterns and priors from training data to reconstruct realistic high-frequency details that were removed during down-scaling, effectively bridging the quality gap without requiring the original high-resolution data.
Solution Approach 2:
The system changes the resolution parameter during encoding to reduce bitrate, then uses AI to transform the resolution parameter back during decoding. The AI model learns the mapping between low and high resolution spaces, allowing parameter transformation that preserves perceptual quality while achieving compression.
3Adaptability or versatility
If AI down-scaling and up-scaling are performed separately, then processing flexibility is improved, but the joint relationship between encoding and decoding is lost
Solution Approach 1:
The AI up-scaling model is trained using feedback from the down-scaling process. During training, the model receives down-scaled images as input and is optimized to produce outputs that match the original high-resolution images. This feedback mechanism ensures that the up-scaling operation is specifically adapted to the down-scaling transformations applied during encoding, maintaining their joint relationship while allowing independent processing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided is an artificial intelligence decoding apparatus includes: a memory; and a processor configured to execute the one or more instructions stored in the memory, the processor is configured to: obtain AI data related to AI down-scaling an original image to a first image; obtain image data corresponding to an encoding result on the first image; obtain a second image corresponding to the first image by performing a decoding on the image data; obtain deep neural network (DNN) setting information among a plurality of DNN setting information from the AI data; and obtain, by an up-scaling DNN, a third image by performing the AI up-scaling on the second image, the up-scaling DNN being configured with the obtained DNN setting information, wherein the plurality of DNN setting information comprises a parameter used in the up-scaling DNN, the parameter being obtained through joint training of the up-scaling DNN and a down-scaling DNN.