ANN Quantization Interval Adjustment for FPGA Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solutions for computing Artificial Neural Networks (ANNs) face challenges with high power consumption, latency, and inefficiency in processing units like CPUs and GPUs, and lack integration with AI frameworks, especially when it comes to large-scale data movement and parallel computation.
Innovation Solution
A system and method for quantization of ANNs that maps floating-point data to fixed-point data, adjusting quantization intervals to reduce saturations and optimize computations, allowing for efficient processing on processors like FPGAs without requiring extensive programming efforts, and enabling integration with AI frameworks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If GPUs are used to process large ANNs due to higher throughput capacity, then computational speed is improved, but power consumption increases excessively
Solution Approach 1:
The patent changes the data representation parameter from floating-point to fixed-point format, enabling efficient computation on FPGAs with lower power consumption while maintaining acceptable accuracy for ANN operations
Solution Approach 2:
The patent substitutes GPU-based parallel processing with FPGA-based fixed-point arithmetic, replacing a high-power general-purpose parallel processor with a lower-power reconfigurable hardware system optimized for specific computational patterns
2Productivity
If FPGAs are programmed to compute ANNs in parallel, then computational efficiency is improved, but programming complexity increases significantly
Solution Approach 1:
The patent creates a universal fixed-point quantization framework that can be applied to various ANN architectures and operations, providing a standardized approach that simplifies FPGA programming while maintaining high computational efficiency
Solution Approach 2:
By standardizing on fixed-point arithmetic with specific bit-width parameters, the patent simplifies the programming model for FPGAs, making it more accessible while maintaining parallel computational efficiency
3Measurement precision
If quantization intervals are reduced to decrease saturations in fixed-point computations, then computational accuracy is improved, but the number of required fixed-point values increases
Solution Approach 1:
The patent optimizes the quantization interval parameters to achieve the best balance between accuracy and resource usage, using statistical analysis of activation distributions to determine appropriate fixed-point representations
Data Source
AI summary
Systems and methods for performing a quantization of artificial neural networks (ANNs) are provided. An example method may include receiving a description of an ANN and sets of inputs to neurons of the ANN, the description including sets of weights of the inputs, the weights being of a first data type, determining a first interval of the first data type to be mapped to a second interval of a second data type; performing computations of sums of products of the weights and the inputs to obtain a set of sum results, wherein the computations are performed using at least one number within the second interval, the number being a result of mapping of a number of the first interval to a number of the second interval, determining a measure of saturations in sum results, and adjusting, based on the measure of saturations, one of the first and second intervals.