Block-Based Long-Range Context Modeling for Neural Image Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional hybrid video codecs are difficult to optimize as a whole, and autoregressive context models in neural image compression suffer from high computational complexity due to consecutive prediction of latents and limited receptive fields, which restricts their performance.
Innovation Solution
Implementing a transformer-based context model, such as the Corner-to-Center Context Model (C3M) that predicts context features progressively from corner to center positions, and uses a block-based splitting strategy for high-resolution images to decode latents using a corner-to-center or edge-to-center latent decoding strategy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If autoregressive context models are used to decode latents, then decoding can be performed serially, but computational complexity increases to O(n2)
Solution Approach 1:
The patent divides the image into multiple non-overlapping blocks and processes each block independently using separate context models. This segmentation allows parallel processing of blocks while reducing the computational burden on each individual context model, as each only needs to predict latents within its own block rather than considering the entire image sequentially.
Solution Approach 2:
The patent introduces a block-based dimensional structure to the decoding process, transitioning from a single sequential autoregressive model to multiple parallel context models operating on different spatial blocks. This dimensional change enables parallel computation across blocks while maintaining contextual accuracy within each block.
2Measurement precision
If traditional context models with fixed kernel sizes are used, then local dependencies are captured, but global latent features cannot be effectively examined
Solution Approach 1:
The patent segments the image into multiple blocks, each processed by its own context model. This allows each model to focus on local dependencies within its block while the collection of blocks collectively covers the global image structure, effectively capturing both local and global features without requiring a single large complex model.
Solution Approach 2:
The patent creates a universal framework where multiple identical context model structures can be applied to different blocks. Each model serves the same function of capturing local dependencies, but their collective application across multiple blocks provides global coverage, making the system multi-functional in terms of spatial coverage.
3Measurement precision
If high resolution images are processed as a single block, then global context is available, but computational load becomes excessive
Solution Approach 1:
The patent divides high-resolution images into multiple smaller non-overlapping blocks that can be processed in parallel. Each block is handled by its own context model, reducing the computational load per model while maintaining overall image quality. The parallel processing of multiple blocks significantly speeds up the overall decoding process compared to processing the entire high-resolution image as a single block.
Data Source
AI summary
Methods and apparatuses for decoding a compressed image using a neural image compression network may be provided. The method may include generating long-range context model parameters associated with a high resolution compressed image, the long-range context model parameters corresponding to a first area. The method may also include splitting the generated long-range context model parameters into a first number of context parameter blocks. The method may also include for each block in the first number of context parameter blocks, predicting respective context features using a long-range context model and respective context parameter blocks, wherein the long-range context model uses a corner-to-center latent decoding strategy or an edge-to-center latent decoding strategy to decode latents associated with the high resolution compressed image. Then, the high resolution compressed image may be reconstructed based on predicted context features.