Latent Feature-Domain Intra-Prediction for Neural Image Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural image compression methods using Deep Neural Networks (DNNs) for video coding lack efficiency in encoding and decoding processes, particularly in utilizing the latent feature domain for intra-prediction and residual coding, which limits their ability to effectively compress and reconstruct images.
Innovation Solution
Implementing a block-based intra-prediction mechanism in the latent feature domain, where residuals between prediction blocks and original blocks are encoded instead of the original blocks, and using neural networks to generate compact representations and context parameters for efficient decoding and reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If block-based intra-prediction and residual coding are implemented in the latent feature domain, then coding efficiency and compression performance are improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent divides the latent feature representation into multiple blocks and processes each block separately through intra-prediction and residual coding operations. This segmentation enables efficient parallel processing while maintaining the benefits of block-based compression, resolving the contradiction by organizing complexity into manageable units that improve overall coding efficiency.
Solution Approach 2:
The patent performs intra-prediction in the latent feature domain before final decoding, preparing prediction blocks and residuals in advance. This preliminary action in the feature domain leverages the compressed representation to reduce the amount of data requiring full processing, thereby improving coding efficiency without proportionally increasing device complexity.
2Productivity
If neural networks are used to generate compact representations and context parameters, then compression performance is improved, but use of energy and computational resources increase
Solution Approach 1:
The patent extracts only the most essential features and parameters from the latent representation for compression, rather than processing the entire feature set. By taking out only the critical context parameters and residual information needed for reconstruction, the system achieves improved compression performance while reducing the energy required for neural network operations.
Solution Approach 2:
The patent applies different processing strategies to different regions and features based on their importance. Context parameters requiring high fidelity are processed with more computational resources, while less critical features use simpler compression methods. This local quality approach optimizes the balance between compression performance and energy consumption.
Data Source
AI summary
A method of decoding an image with latent feature-domain intra-prediction is performed by at least one processor and includes receiving a set of latent blocks and for each of the blocks in the set of latent blocks: predicting a block, based on a set of previously recovered blocks; receiving a selection signal indicating a currently recovered block, based on the selection signal performing one of (1) and (2): (1) generating a compact residual, a set of residual context parameters, a decoded residual, and generating a first decoded block; (2) generating a second decoded block, based on a compact representation block and a set of context parameters. The method further includes generating a set of recovered blocks comprising each of the currently recovered blocks; generating a recovered latent image by merging all the blocks in the set of recovered blocks; and decoding the recovered latent image, to obtain a reconstructed image.