Neural Network Compilation with Adaptive Clipping and Cross-Layer Equalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing tensor quantization methods for artificial neural networks often result in significant precision loss when converting floating-point representations to fixed-point representations, especially when integrated into dedicated hardware circuits.
Innovation Solution
A compilation process that modifies the topology of an artificial neural network by replacing the linear unit layer with an adaptive scaling layer, followed by cross-layer equalization and bias absorption, allowing for tensor quantization after training without requiring all training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tensor quantization is applied to convert floating-point representations to fixed-point representations for integration into dedicated hardware circuits, then execution performance and energy efficiency are improved, but precision loss occurs during the conversion process
Solution Approach 1:
The patent applies cross-layer equalization and bias absorption techniques during the quantization compilation process before the neural network is deployed to hardware. These preliminary actions adjust the quantized weights and biases to compensate for expected precision losses, ensuring that the quantized model maintains accuracy close to the original floating-point model while benefiting from fixed-point execution performance
Solution Approach 2:
The patent modifies the quantization parameters (such as scale factors and zero-points) through cross-layer equalization and bias absorption operations. By changing these parameters during the compilation process, the system optimizes the fixed-point representation to minimize precision loss while maintaining the performance benefits of hardware acceleration
2Measurement precision
If quantization-aware training (QAT) is performed during the training phase to improve quantization accuracy, then precision is improved, but training complexity and computational requirements increase significantly
Solution Approach 1:
The patent extracts the quantization optimization process from the training phase and moves it to the post-training compilation phase. By separating these concerns, the system achieves quantization accuracy improvement through cross-layer equalization and bias absorption without the need for complex quantization-aware training, thus reducing training complexity while maintaining precision
3Ease of operation
If post-training quantization (PTQ) is applied to simplify the quantization process, then ease of operation is improved, but precision loss increases compared to quantization-aware training
Solution Approach 1:
The patent enhances post-training quantization by applying cross-layer equalization and bias absorption techniques that modify the quantization parameters (scales and zero-points) during compilation. This approach maintains the simplicity of post-training quantization while significantly improving accuracy by optimizing the fixed-point representation parameters without requiring retraining
4Measurement precision
If floating-point representations are used throughout the neural network, then measurement precision is improved, but execution speed and energy efficiency decrease when deployed on dedicated hardware circuits
Solution Approach 1:
The patent transforms the neural network from floating-point to optimized fixed-point representation by applying cross-layer equalization and bias absorption during compilation. This parameter change enables the network to execute efficiently on dedicated hardware circuits with integer arithmetic while maintaining accuracy close to the original floating-point model through careful optimization of quantization parameters
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
According to one aspect, a computer-implemented method is proposed for compiling a first trained artificial neural network, the first trained artificial neural network comprising at least one succession (SUCC_2) of layers including a depth convolution layer, then a saturation rectified linear unit layer, then a two-dimensional convolution layer, the method comprising: - an equalization (13) between layers, - a replacement (15) of the saturation rectified linear unit layer by an adaptive clipping layer per channel so as to obtain an artificial neural network with modified topology, - a tensor quantization (16) of the layers of the artificial neural network with modified topology, - a compilation (17) of the artificial neural network with modified topology and quantized.