Neural Network Quantization Parameter Determination Method
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require large storage space and high processing bandwidth due to high-precision data representation, leading to increased costs and resource consumption, particularly in artificial intelligence processor chips.
Innovation Solution
A method for determining neural network quantization parameters to convert high-precision data into low-precision fixed-point data, reducing storage space and improving computing performance by using an artificial intelligence processor to analyze and quantify data such as neurons, weights, and biases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-precision data representation is used in neural networks, then measurement precision is improved, but volume of stationary object increases
Solution Approach 1:
The patent applies parameter changes by converting neural network data from high-precision floating-point format to low-precision fixed-point format. This transformation changes the numerical representation parameters, reducing the bits required per data element from 32 bits (float32) to 8 bits (fix8), thereby achieving 4x compression while maintaining acceptable precision for neural network operations.
Solution Approach 2:
The patent creates a quantized copy of the neural network data that approximates the original high-precision data. The quantization process generates fixed-point representations that serve as compressed copies, enabling storage and processing with reduced precision while preserving the essential functional characteristics needed for accurate neural network inference.
2Measurement precision
If high-precision data representation is used in neural networks, then measurement precision is improved, but processing bandwidth increases
Solution Approach 1:
The patent changes the data representation parameters from floating-point to fixed-point format, which fundamentally alters how data is processed. Fixed-point arithmetic requires fewer computational resources and less bandwidth compared to floating-point operations, enabling faster processing and reduced memory access requirements while maintaining sufficient precision for neural network computations.
3Volume of stationary object
If data quantization is performed, then volume of stationary object decreases, but manufacturing precision increases
Solution Approach 1:
The patent carefully adjusts quantization parameters such as scale factors and zero-points to minimize precision loss during the conversion from floating-point to fixed-point format. By optimizing these parameters, the system achieves effective compression while maintaining the precision required for accurate neural network operations, balancing storage efficiency with computational accuracy.
Data Source
AI summary
The present disclosure relates to a neural network quantization parameter determination method and related products. A board card in the related products includes a memory device, an interface device, a control device, and an artificial intelligence chip, in which the artificial intelligence chip is connected with the memory device, the control device, and the interface device respectively. The memory device is configured to store data, and the interface device is configured to transmit data between the artificial intelligence chip and an external device. The control device is configured to monitor the state of the artificial intelligence chip. The board card can be used to perform an artificial intelligence computation.