Deep Generative Network Entropy Estimation for Low Bitrate Image Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GAN-based image compression approaches lack the use of advanced GAN losses like Wasserstein GAN with gradient penalty loss (WGAN-GP) to enhance image quality, and require extra convolution layers for entropy estimation, increasing model complexity and bit requirements.
Innovation Solution
The proposed solution involves a neural network system with a generator that includes an encoder, entropy estimator, and decoder, along with a discriminator using novel loss functions and a simple entropy estimation approach that does not require additional convolution layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If extra convolution layers are added to estimate and constrain entropy during training, then entropy control is improved, but device complexity increases and additional bits are required for transmission
Solution Approach 1:
The patent extracts the entropy estimation function from the traditional convolutional neural network architecture by removing extra convolution layers. Instead, it uses a simplified entropy estimator that operates on quantized feature entries, thereby maintaining entropy control capability while reducing network complexity and eliminating the need for additional transmission bits.
Solution Approach 2:
The patent changes the parameters and structure of the entropy estimation mechanism from complex convolutional operations to a simplified calculation based on quantized feature entries. This parameter change allows entropy to be estimated and constrained effectively without requiring additional network layers or increasing device complexity.
2Manufacturing precision
If advanced GAN losses such as WGAN-GP are utilized, then image quality is improved, but computational complexity increases
Solution Approach 1:
The patent segments the loss function into distinct components: a discriminator loss based on WGAN-GP for image quality enhancement, and a separate simplified entropy loss for rate control. This segmentation allows advanced GAN losses to be applied where they improve image quality without unnecessarily increasing overall computational complexity, as each component can be optimized independently.
3Productivity
If low bit rates are used for image compression, then transmission efficiency is improved, but image quality deteriorates
Solution Approach 1:
The patent changes the optimization parameters by using WGAN-GP loss instead of traditional MSE loss in the discriminator, and employs a simplified entropy estimator with modified calculation parameters. These parameter changes enable the network to achieve better image quality at low bit rates by focusing on perceptually important features while maintaining efficient compression.
Solution Approach 2:
The patent implements a feedback mechanism where the discriminator provides quality assessment signals back to the generator during training, using WGAN-GP loss to guide the generation process. This feedback loop enables the system to learn to produce high-quality images even at low bit rates by continuously adjusting based on quality metrics rather than simple pixel-wise errors.
Data Source
AI summary
A neural network system, a method and an apparatus for image compression are provided. The neural network may include a generator including an encoder, an entropy estimator, and a decoder, where the encoder receives an input image and generates an encoder output, a plurality of quantized feature entries are obtained based on the encoder output outputted at a last encoder block, the entropy estimator receives the plurality of quantized feature entries and calculates an entropy loss based on the plurality of quantized feature entries, and the decoder receives the plurality of quantized feature entries and generates a reconstructed image. Furthermore, the neural network may include a discriminator that determines whether the reconstructed image different from the input image based on a discriminator loss. Moreover, the generator may determine whether content of the reconstructed image matches content of the input image based on a generator loss including the entropy loss.