ANN Quantization Interval Adjustment for FPGA Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently processing artificial neural networks (ANNs) due to high computational loads on CPUs and power consumption issues with GPUs, while Field Programmable Gate Arrays (FPGAs) require complex programming and lack integration with AI frameworks, and existing solutions fail to address the massive data movement and computation scale required for real industrial applications.
Innovation Solution
The development of a system and method for quantization of ANNs that allows computations on non-floating point values, enabling the selection of quantization intervals for input values, weights, and other parameters, which can be adjusted individually for each layer or filter, reducing neuron saturations in integer-based computations and facilitating efficient processing on FPGAs, ASICs, or other programmable logic devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPUs are used to process large ANNs to overcome CPU throughput limitations, then computational throughput is improved, but power consumption increases excessively
Solution Approach 1:
The patent applies parameter changes by transforming the data representation from floating-point to fixed-point quantized formats. This changes the computational parameters to enable efficient integer arithmetic on FPGAs, achieving high throughput while reducing power consumption compared to GPU floating-point operations.
Solution Approach 2:
The patent substitutes the GPU mechanical system with an FPGA-based system that uses quantized fixed-point arithmetic. This replacement enables parallel integer operations that are more energy-efficient while maintaining the computational throughput required for large-scale ANN processing.
2Productivity
If FPGAs are used to compute ANNs to achieve parallelism, then computational efficiency is improved, but programming complexity increases significantly
Solution Approach 1:
The patent simplifies FPGA programming by changing the computational parameters to fixed-point quantized representations. This allows the use of standard integer arithmetic operations and memory access patterns, reducing programming complexity while maintaining parallel computational efficiency.
Solution Approach 2:
The patent creates a universal quantization framework that can be applied to various ANN architectures and FPGA configurations. This multi-functional approach provides a standardized method for converting floating-point ANNs to fixed-point representations, simplifying the programming task across different applications.
3Measurement precision
If quantization intervals are adjusted individually for each layer or filter, then computation accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies preliminary action by performing quantization interval adjustment during the training phase or offline preparation. The optimal quantization parameters are pre-computed and stored, allowing fast inference during deployment without requiring time-consuming real-time optimization, thus achieving high accuracy with minimal processing time.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Systems and methods for performing a quantization of artificial neural networks (ANNs) are provided. An example method may include receiving a description of an ANN and input data associated with the ANN, wherein the input data are represented according to a first data type; selecting a first value interval of the first data type to be mapped to a second value interval of a second data type; performing, based on the input data and the description of the ANN, the computations of one or more neurons of the ANN, wherein the computations are performed for at least one value within the second value interval, the at least one value being a result of mapping a value of the first value interval to a value of the second value interval; determining, a measure of saturations in neurons of the ANN, and adjusting, based on the measure of saturations, the value intervals.