Multi-Scale Neural Image Compression with Intra-Prediction Residuals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural image compression methods are inefficient in encoding and decoding images due to direct encoding of entire images, which can be improved by using block-based intra-prediction and multi-scale processing to encode residuals rather than original pixels.
Innovation Solution
A method and apparatus for multi-scale neural image compression using block-based intra-prediction and residual coding, where an input image is downscaled, predicted, and residuals are encoded and decoded, with different coding strategies applied at various scales to enhance efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If direct encoding of entire images is used, then the encoding process is simple, but the coding efficiency is poor and bit consumption is high
Solution Approach 1:
The image is divided into multiple blocks of different sizes (e.g., 16x16, 8x8, 4x4 pixels). Each block is independently processed through intra-prediction and residual encoding, allowing the encoder to work on smaller segments rather than the entire image at once, thereby improving coding efficiency while managing complexity through modular processing
Solution Approach 2:
The patent introduces a multi-scale dimension by processing images at different resolutions. The image is downsampled to create multiple scale versions, and intra-prediction is applied at each scale. This dimensional transformation from single-scale to multi-scale processing enables better compression by capturing both coarse and fine details efficiently
2Manufacturing precision
If block-based intra-prediction with multi-scale processing is used, then coding performance is improved, but the processing complexity increases
Solution Approach 1:
The image processing is segmented into multiple stages: downsampling to create scale versions, block partitioning at each scale, intra-prediction for each block, and residual encoding. This segmentation allows complex multi-scale processing to be broken down into manageable, repeatable operations that improve reconstruction accuracy while controlling processing complexity through systematic organization
Solution Approach 2:
The image is pre-processed by downsampling to create multiple scale versions before the main encoding process. This preliminary action prepares the data in a form that facilitates more efficient subsequent processing, allowing the intra-prediction and residual encoding to work with pre-scaled versions rather than requiring complex real-time scaling during the main encoding phase
3Loss of substance
If residuals are encoded instead of original pixels, then bit consumption is reduced, but the encoding process becomes more complex
Solution Approach 1:
The patent extracts the prediction component from the original image data through intra-prediction, leaving only the residual (difference) to be encoded. By separating the predictable portion (captured by the prediction model using boundary pixels) from the unpredictable residual, the method reduces bit consumption since residuals contain less information than original pixels, while the extraction process itself manages complexity through established prediction algorithms
Data Source
AI summary
A method of multi-scale neural image compression with intra-prediction residuals is performed by at least one processor and includes downsampling an input image, generating a current predicted image, based on a previously-recovered predicted image, and generating a prediction residual based on a difference between the downsampled input image and the generated current predicted image. The method further includes encoding the generated prediction residual, decoding the encoded prediction residual, and generating a currently-recovered predicted image based on an addition of the current predicted image and the decoded prediction residual. The method further includes upsampling the currently-recovered predicted image, generating a scale residual based on a difference between the input image and the upsampled currently-recovered predicted image, and encoding the scale residual.