Integer Neural Compression for Low-Complexity Image and Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network-based image and video compression technologies face challenges in achieving efficient compression and reconstruction quality due to the complexity of modeling probability distributions and quantization processes, particularly in high-dimensional data, and the need for scalable and interoperable solutions.
Innovation Solution
The method employs a neural network that performs convolution operations and activation functions using only integer values, utilizing integer weights and biases, and employs bit-shifting and quantization to ensure integer outputs, while converting visual media data into bitstreams without non-integer operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural network-based compression is used to improve compression efficiency and quality, then compression performance is improved, but computational complexity increases due to floating-point operations
Solution Approach 1:
The patent changes the numerical parameter type from floating-point to integer throughout the neural network processing pipeline. This includes using integer weights, integer biases, and integer activation functions, which transforms the computational operations to be simpler and more efficient while maintaining compression performance.
Solution Approach 2:
The patent substitutes the traditional floating-point arithmetic mechanism with integer arithmetic mechanism. This replacement eliminates the need for complex floating-point units and their associated computational overhead, thereby reducing device complexity while preserving the essential neural network functionality for compression.
2Productivity
If integer operations are used to reduce computational complexity, then processing speed is improved, but precision may be reduced compared to floating-point operations
Solution Approach 1:
The patent introduces a new dimension of quantization bit-depth optimization. By carefully selecting and optimizing the bit-depth for integer representations at different stages of the neural network, the patent achieves a balance between processing speed and precision, ensuring that integer operations maintain sufficient accuracy for high-quality compression.
Solution Approach 2:
The patent optimizes the integer parameter ranges and bit-depth allocations throughout the network to maintain precision. By adjusting these parameters strategically, the patent ensures that integer operations achieve both speed improvements and adequate compression quality without the precision loss typically associated with integer arithmetic.
3Manufacturing precision
If complex probability distribution modeling is used to improve compression quality, then reconstruction quality is improved, but device complexity increases
Solution Approach 1:
The patent simplifies the probability distribution modeling by changing from continuous floating-point distributions to discrete integer-based distributions. This parameter change reduces the complexity of modeling while maintaining the ability to capture essential statistical characteristics of the data, thereby improving reconstruction quality without excessive device complexity.
Solution Approach 2:
The patent applies different levels of modeling complexity locally to different parts of the data processing pipeline. By using simpler integer-based models where sufficient and more sophisticated approaches only where necessary, the patent achieves good reconstruction quality while minimizing overall device complexity.
Data Source
AI summary
A mechanism for processing video data is disclosed. A determination is made to apply a neural subnetwork to perform a convolution operation using only integer values. The neural subnetwork also performs an activation function on the output of the convolution operation using only integer values. A conversion is performed between a visual media data and a bitstream based on the convolution operation and the activation function.