Neural Network Quantization Method for Reducing Storage and Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network architectures require large storage space and high processing bandwidth due to their multi-bit nature, leading to increased costs.
Innovation Solution
A neural network quantization method that reduces the number of bits by determining quantization parameters for weights and input data of a target layer without distorting the maximum absolute value, thereby reducing storage space and processing bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-bit neural network architectures are used, then processing precision is maintained, but storage space and processing bandwidth increase
Solution Approach 1:
The patent applies parameter changes by transitioning from multi-bit (e.g., 32-bit) weight and input data representations to low-bit (e.g., 8-bit or lower) quantized representations. This is achieved by determining quantization parameters including maximum absolute values for weights and input data, then scaling and rounding the original values to fit into fewer bits while maintaining the principle of not distorting maximum absolute values. The result is a significant reduction in storage space requirements while preserving acceptable processing precision.
2Measurement precision
If multi-bit neural network architectures are used, then processing precision is maintained, but processing bandwidth increases
Solution Approach 1:
The patent reduces processing bandwidth by changing the bit-depth parameter of weight and input data from multi-bit to low-bit quantized formats. By determining quantization parameters and applying scaling operations, the patent enables data to be processed with fewer bits, directly reducing the bandwidth required for data transmission and processing operations while maintaining acceptable precision through careful quantization parameter selection.
3Quantity of substance
If quantization is applied to reduce bit operations, then storage space and power consumption are reduced, but processing speed may be affected
Solution Approach 1:
The patent applies preliminary action by pre-determining quantization parameters including maximum absolute values for weights and input data before the actual quantization process. This preliminary analysis allows the system to establish appropriate scaling factors and rounding rules in advance, enabling fast quantization operations during inference without sacrificing processing speed. The quantization parameters are computed once and then reused, making the quantization process itself computationally efficient.
Data Source
AI summary
The invention provides a neural network quantization method and device and a related product. The neural network quantization method is used for quantizing data of a computation layer of a neural network. The technical scheme provided by the invention has the advantage of low cost.