ANN Quantization Scheme Using Symmetric Bit-Stream Dot Products
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks face a trade-off between increased power efficiency and reduced accuracy due to quantization, which decreases performance at low-precision levels, especially when using asymmetric quantization schemes.
Innovation Solution
A processor-implemented artificial neural network quantization scheme that uses a symmetric quantization scheme, encoding input data and weight parameters into bit streams, applying them to a binary neural network operator with XNOR and popcount operations, and accumulating results to optimize dot product calculations, thereby improving operation efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If quantization is applied to increase power efficiency and reduce computational operations, then power efficiency and operation speed improve, but accuracy of the artificial neural network decreases
Solution Approach 1:
The patent changes the quantization parameters by introducing a symmetric quantization scheme where positive and negative quantization levels are completely symmetric and exclude zero. This parameter change allows the system to maintain better accuracy while still achieving power efficiency gains from quantization, resolving the contradiction between power efficiency and accuracy.
Solution Approach 2:
The patent applies asymmetry by excluding zero from quantization levels and creating a symmetric distribution around zero (e.g., {-1, 1} for 1-bit, {-2, -1, 1, 2} for 2-bit). This asymmetric treatment of zero (completely excluding it) creates a symmetric quantization scheme that improves accuracy while maintaining the power efficiency benefits of low-precision quantization.
2Productivity
If lower-bit mathematical operations are performed with quantized parameters, then operation speed increases, but manufacturing precision of calculation results decreases
Solution Approach 1:
The patent changes the bit-width parameters systematically, using 1-bit for binary neural networks and extending to 2-bit or higher with symmetric quantization. This parameter change enables lower-bit operations to achieve better precision by eliminating the asymmetric error accumulation that occurs in conventional quantization schemes, thus improving calculation precision while maintaining high operation speed.
Solution Approach 2:
The patent uses copying by representing quantized values through multiple bit patterns in a symmetric scheme. For example, instead of using asymmetric representations that accumulate error, the system copies the symmetric structure across all quantization levels, ensuring that positive and negative values are treated equally and precision is maintained throughout the calculation process.
3Device complexity
If asymmetric quantization schemes are used to simplify implementation, then device complexity reduces, but accuracy deteriorates
Solution Approach 1:
The patent applies asymmetry in a deliberate way by excluding zero from quantization levels. This asymmetric design choice (removing zero) actually creates symmetry in the remaining levels, simplifying the implementation while improving accuracy. The asymmetric exclusion of zero leads to symmetric quantization schemes that are both simple to implement and accurate.
Solution Approach 2:
The patent changes the quantization parameter set from conventional asymmetric schemes (including zero) to symmetric schemes (excluding zero). This parameter change maintains implementation simplicity by using uniform quantization steps while dramatically improving accuracy by eliminating the asymmetric bias that plagues conventional quantization methods.
Data Source
AI summary
A processor-implemented artificial neural network quantization scheme implementation method and apparatus are provided. The method includes receiving input data corresponding to a first M-dimensional vector, receiving a weight parameter corresponding to a second M-dimensional vector, encoding the input data into first bit streams, each having “N” layers, with a predetermined quantization scheme, encoding the weight parameter into second bit streams, each having “N” layers, with the quantization scheme, applying corresponding first and second bit streams to a binary neural network operator, for each of possible combinations between layers of the first bit streams and layers of the second bit streams, receiving a dot product result output based on a result obtained by shifting a BNN operation result corresponding to each of the combinations by a number of corresponding bits and accumulating the shifted BNN operation result, from the BNN operator, and quantizing the dot product result using the quantization scheme.