Learned Image Compression With Soft Frequency Entropy Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing end-to-end learnable image compression methods face challenges in training complexity and require additional training to update the entropy model when a temporal context is available, due to the non-differentiability of the quantization step and reliance on neural networks for estimating the latent space rate.
Innovation Solution
A non-parametric soft frequency counter model is introduced to estimate the latent space entropy distribution, allowing for differentiable optimization during training and eliminating the need for additional training when a temporal context is available.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a neural network is used to estimate the latent space rate, then the rate estimation accuracy is improved, but the training complexity increases
Solution Approach 1:
The patent extracts the rate estimation function from a complex neural network and implements it through a simplified entropy model with explicit mathematical formulas. The entropy model computes rate estimation using direct calculations based on latent space statistics rather than requiring a trained neural network, thereby maintaining accuracy while reducing training complexity.
Solution Approach 2:
The patent replaces the mechanical training process of a neural network with a mathematical computation system. The entropy model uses closed-form expressions and statistical computations that do not require gradient descent or backpropagation, substituting the iterative mechanical training process with direct mathematical evaluation.
2Speed
If a neural network is used to estimate the latent space rate, then the rate convergence is improved, but the training time increases
Solution Approach 1:
The patent extracts the rate estimation computation from the iterative neural network training process and formulates it as a separate, efficient entropy model that can be computed independently. This allows rate convergence to be achieved through direct mathematical computation rather than iterative neural network training, reducing training time while maintaining convergence speed.
Solution Approach 2:
The entropy model performs preliminary computations of latent space statistics and rate estimation that can be done in advance or in parallel with the main training process. By precomputing entropy-related quantities using explicit formulas, the system avoids repeated iterative computations during training, thereby reducing overall training time.
3Adaptability or versatility
If the entropy model is retrained for temporal contexts, then the adaptation accuracy is improved, but the additional training time increases
Solution Approach 1:
The entropy model is designed to be self-adapting to temporal contexts through its mathematical formulation. Rather than requiring retraining on temporal data, the model automatically adjusts to different temporal contexts through its statistical computations on the latent space, which inherently capture temporal dependencies. This self-service capability provides adaptation without additional training time.
Solution Approach 2:
The entropy model is designed as a universal framework that can handle multiple contexts (spatial and temporal) with the same mathematical formulation. The model's explicit formulas work across different data distributions and contexts without requiring context-specific training, providing multi-functionality that adapts to temporal contexts while avoiding additional training requirements.
Data Source
AI summary
A method for learned image compression implemented in an autoencoder includes: a) extracting from an image a latent space by the learnable encoder; b) quantizing the latent space by a quantizer to obtain a quantized latent space; c) entropy coding the quantized latent space by an entropy encoder to obtain a bitstream, wherein an entropy model used to encode the latent space is represented by a probability distribution; d) entropy decoding the bitstream by an entropy decoder to obtain an entropy decoded bitstream; e) feeding the entropy decoded bitstream to the decoder; f) recover a reconstructed image by the decoder; g) training the autoencoder via standard gradient descent of the backpropagated error gradient by finding learnable parameters of the learnable encoder and of the decoder that minimize a rate distortion cost function, wherein the entropy encoder is based on a differentiable formulation of a soft frequency counter.