Neural Network Entropy Coding Quantization Optimization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Neural network variables related to entropy coding in video/image compression schemes face issues when quantized, leading to degradation in compression efficiency, especially in common use cases, and these losses cannot be recovered by re-training the network, due to the lack of consideration for specific properties of entropy coding variables in general optimization tools.

Innovation Solution

Optimizing the definition of entropy coding variables by determining a probability distribution function parameter based on a distribution optimized for quantization, and using this parameter to determine a code vector for entropy coding, which can be empirically defined or determined through algorithms evaluating coding redundancy or solving ordinary differential equations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If neural network variables are quantized to low-precision integers to improve speed and power consumption, then processing efficiency is improved, but compression efficiency deteriorates due to information loss in entropy coding variables

Engineering Contradiction:

Improveprocessing speedVSAvoidcompression efficiency

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The patent transforms the probability distribution function parameter from its original scale to a new scale optimized for quantization. Specifically, it applies a monotonic transformation function to map the parameter values to a range that minimizes quantization error impact on entropy coding performance, thereby resolving the contradiction between low-precision representation and compression efficiency

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent performs preliminary optimization of the probability distribution function parameter definition before quantization occurs. By designing the parameter transformation function in advance based on the statistical properties of the data stream, the system prepares the parameters to be inherently more robust to quantization effects, preventing information loss before it happens

Inventive Principle:

Principle #10Preliminary action

2Ease of manufacture

If general optimization tools are used for quantization, then implementation simplicity is improved, but entropy coding performance deteriorates because specific properties of entropy coding variables are not considered

Engineering Contradiction:

Improveimplementation simplicityVSAvoidentropy coding performance

Core Design Contradiction:

Ease of manufactureVSLoss of information

Solution Approach 1:

The patent applies a specialized parameter transformation specifically tailored for entropy coding variables rather than using generic quantization methods. The transformation function is designed to preserve the specific statistical properties and relationships that are critical for entropy coding performance, treating these variables differently from other neural network parameters

Inventive Principle:

Principle #3Local quality

3Adaptability or versatility

If standard probability distribution parameters are used, then compatibility with existing methods is improved, but quantization effects deteriorate because the distribution is not optimized for low-precision representation

Engineering Contradiction:

ImprovecompatibilityVSAvoidquantization accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

Instead of quantizing the standard probability distribution parameters directly, the patent inverts the approach by first defining the parameter on a transformed scale optimized for quantization, then mapping it back to the standard parameter space. This inversion allows the system to benefit from both standard parameter compatibility and quantization optimization

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS11876969B2Neural-network media compression using quantized entropy coding distribution parameters

Publication Date: 2024.01.16 QUALCOMM INC

AI summary

This disclosure describes entropy coding techniques for media data coded using neural-based techniques. A media coder is configured to determine a probability distribution function parameter for a data element of a data stream coded by a neural-based media compression technique, wherein the probability distribution function parameter is a function of a standard deviation of a probability distribution function of the data stream, determine a code vector based on the probability distribution function parameter, and entropy code the data element using the code vector.