Hardware Accelerator Softmax Approximation with Lookup Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require significant computational resources for complex softmax operations, leading to inefficiencies in processing massive input data and complicating software development and power consumption.
Innovation Solution
A hardware accelerator employs a lookup table to store reciprocal exponential function values, mapping input data differences to indexes, and uses shift registers for efficient softmax approximation, reducing computational complexity without dividers or multipliers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional softmax computation is used in neural networks, then computational accuracy is maintained, but computational complexity and power consumption increase significantly
Solution Approach 1:
The patent changes the computational parameters by using logarithmic transformation and piecewise linear approximation. Instead of directly computing exponential functions and normalizing, the method transforms the softmax computation into a form that uses log-sum-exp approximation with pre-computed lookup tables, fundamentally changing how the calculation is performed to reduce complexity while maintaining accuracy
Solution Approach 2:
The patent applies preliminary action by pre-computing and storing exponential function values in lookup tables before runtime. The maximum value subtraction is performed early in the computation pipeline, and the piecewise linear approximation parameters are pre-calculated, allowing the actual softmax computation to use simpler operations with lower complexity
2Measurement precision
If traditional softmax computation is used, then accurate probability distribution is obtained, but power consumption increases
Solution Approach 1:
The patent changes computational parameters to use logarithmic domain operations and piecewise linear functions that require fewer arithmetic operations. The transformation of softmax into a form using pre-computed logarithmic values reduces the number of high-power multiplication and division operations, directly lowering power consumption while preserving probability distribution accuracy
Solution Approach 2:
The patent extracts the computationally intensive exponential calculation part and replaces it with pre-computed lookup table values. By separating the expensive exponential computation from the runtime operation and storing it in advance, the actual softmax computation uses simpler table lookups and basic arithmetic, reducing real-time power consumption
3Productivity
If lookup tables are used for softmax approximation, then computational speed increases, but hardware resources are consumed
Solution Approach 1:
The patent applies segmentation by dividing the exponential function lookup table into multiple smaller piecewise segments. Instead of storing one large table covering the entire input range, the computation is split into multiple ranges, each with its own smaller lookup table or approximation parameters. This reduces the total memory resources required while maintaining high processing speed through localized lookups
Solution Approach 2:
The patent changes the representation parameters by using piecewise linear approximation coefficients stored in compact formats. The lookup tables store only the necessary breakpoint values and slope parameters rather than full exponential values, reducing memory requirements. The linear approximation parameters are optimized to fit within smaller hardware resource constraints while maintaining accuracy
4Reliability
If maximum value subtraction is performed before lookup, then numerical stability is improved, but additional computational steps are required
Solution Approach 1:
The patent applies preliminary action by performing the maximum value subtraction early in the computation pipeline, before the lookup table operation. This preprocessing step is integrated into the overall computation flow, and the result is immediately used as input to the lookup table, so while an additional step is added, it enables the use of simpler and more efficient lookup operations that compensate for the extra step
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of operating a hardware accelerator, includes loading a lookup table, mapping each of input data values of input data to an index of indexes in the lookup table based on an input data distribution of the input data, and obtaining output data values corresponding to the input data values using the lookup table. The output data values are proportional to corresponding softmax values of the input data values.