Power-of-Two Quantized Neural Processing for Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network-based processing in video coding is non-reproducible and complex, requiring high memory and computation resources, which is undesirable for efficient video encoding and decoding.
Innovation Solution
Implement a neural network-based processing method using quantized tensors with scaling factors of powers of two, minimizing operations through bit shifts and avoiding zero points, and incorporating quantization-aware training to reduce complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural network-based processing is applied to video coding to improve compression efficiency, then compression performance is improved, but processing complexity and computational resources increase significantly
Solution Approach 1:
The patent applies quantization to transform neural network weights and activations from floating-point to integer domain with specific bit depths (e.g., 8-bit or 4-bit quantization). Scaling factors are carefully selected as powers of two to enable efficient computation. This parameter transformation reduces computational complexity while maintaining acceptable compression performance.
Solution Approach 2:
The patent extracts and removes the zero-point offset from the quantization process, using only positive scaling factors. This simplifies the quantization and dequantization operations by eliminating the need for zero-point storage and computation, thereby reducing processing complexity while preserving the essential neural network functionality for video coding.
2Productivity
If neural network-based processing is applied to video coding to improve compression efficiency, then compression performance is improved, but memory requirements increase due to weight storage
Solution Approach 1:
The patent quantizes neural network weights to lower precision (e.g., 8-bit or 4-bit integers instead of 32-bit floating-point), directly reducing the memory footprint of weight tensors. This parameter transformation enables the same neural network model to occupy significantly less memory while maintaining adequate compression performance.
Solution Approach 2:
The patent divides the neural network processing into discrete quantized layers, where each layer's weights are independently quantized and stored. This segmentation allows for efficient memory management and enables the use of specialized hardware structures that can process quantized data with reduced memory bandwidth requirements.
3Device complexity
If standard quantization with zero points is used to reduce complexity, then computational operations increase due to additional zero-point handling
Solution Approach 1:
The patent explicitly removes the zero-point component from the quantization scheme, using only positive scaling factors. This extraction eliminates the need to store and compute zero-points during forward propagation, significantly simplifying the computational operations while maintaining the benefits of quantization for reducing processing complexity.
Solution Approach 2:
Instead of using the conventional quantization approach that includes zero-points to represent both positive and negative values, the patent inverts the approach by using only positive scaling factors and handling negative values through separate weight quantization. This inverted approach reduces computational overhead while achieving the same dynamic range coverage.
Data Source
AI summary
At least a method and an apparatus are presented for efficiently encoding or decoding video by applying a neural network-based processing to a tensor of input data to generate a tensor of output data. For example, the quantization of the tensors is limited to a scaling by a power of 2. For example, the tensor product layer, the bias addition layer and the activation are fused to reduce the number of operations and increase the available bits to represent the values.