Logarithmic Arithmetic Inference Accelerator for Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing inference accelerators face energy inefficiencies in performing dot product calculations and other operations, particularly when dealing with increasing bit lengths, as they require more energy and time to produce accurate results.
Innovation Solution
The use of logarithmic-based arithmetic, where input values are represented as a sign and an exponent, allowing for efficient multiplication and addition operations by decomposing exponents into quotient and remainder components, and using asynchronous accumulators to compute partial sums, reducing energy consumption and bit toggling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional arithmetic operations are performed on integer, fixed-point, or floating-point format values, then accurate results are produced, but energy consumption increases significantly
Solution Approach 1:
The patent transforms the numerical representation from conventional integer, fixed-point, or floating-point formats to logarithmic format. This parameter change in the number system allows multiplication operations to be performed by simply adding exponents, dramatically reducing energy consumption while maintaining calculation accuracy for neural network inference operations
2Ease of operation
If logarithmic format values are added by converting to integers, computing sum, and converting back, then addition is performed, but operation complexity increases
Solution Approach 1:
The patent segments the exponent of logarithmic format values into quotient components and remainder components. This segmentation allows the addition operation to be decomposed into sorting quotient components based on remainder components, summing sorted quotient components using asynchronous accumulators, and multiplying partial sums by remainder components, thereby simplifying the overall addition process while maintaining accuracy
3Measurement precision
If the number of bits used to represent each value increases, then calculation accuracy improves, but energy consumption and calculation time increase
Solution Approach 1:
The patent changes the parameter of numerical representation to logarithmic format, where values are expressed as exponents. This allows the system to perform accurate calculations with fewer bits because the logarithmic representation inherently compresses the dynamic range, reducing both energy consumption and calculation time while maintaining the necessary precision for neural network operations
Data Source
AI summary
Neural networks, in many cases, include convolution layers that are configured to perform many convolution operations that require multiplication and addition operations. Compared with performing multiplication on integer, fixed-point, or floating-point format values, performing multiplication on logarithmic format values is straightforward and energy efficient as the exponents are simply added. However, performing addition on logarithmic format values is more complex. Conventionally, addition is performed by converting the logarithmic format values to integers, computing the sum, and then converting the sum back into the logarithmic format. Instead, logarithmic format values may be added by decomposing the exponents into separate quotient and remainder components, sorting the quotient components based on the remainder components, summing the sorted quotient components using an asynchronous accumulator to produce partial sums, and multiplying the partial sums by the remainder components to produce a sum. The sum may then be converted back into the logarithmic format.