Neural Network Activation Accelerator Using Lookup Table Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerators are inefficient in implementing non-linear activation operations due to the high computational requirements and power consumption associated with these operations.
Innovation Solution
The proposed activation accelerator employs a look-up table and multiple activation pipelines, each comprising a range conversion unit, an index generation unit, a look-up table interface unit, and an interpolation unit, to efficiently apply non-linear activation functions by storing values representing the activation function in a look-up table and using interpolation to estimate the activation function output for input values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If traditional computational methods are used for non-linear activation operations, then accuracy is maintained, but computational overhead and power consumption increase significantly
Solution Approach 1:
The patent creates a copy of the activation function in the form of a look-up table (LUT) that stores pre-computed input-output pairs. Instead of performing complex non-linear calculations during inference, the system copies the function's behavior into a static table structure that can be queried directly, dramatically reducing computational overhead and power consumption while maintaining accuracy through interpolation.
Solution Approach 2:
The activation function values are pre-computed and stored in the look-up table before runtime. This preliminary action eliminates the need for real-time computation of non-linear functions during neural network inference, shifting the computational burden to an offline preprocessing stage where power consumption is less critical.
2Productivity
If a look-up table is used to store activation function values, then computational overhead is reduced, but memory requirements increase
Solution Approach 1:
The look-up table is segmented into multiple pipelines, each handling a specific range of input values. This segmentation allows the system to distribute the storage requirements across multiple smaller tables rather than requiring one large table, improving memory management efficiency and allowing parallel access patterns that reduce bottlenecks.
Solution Approach 2:
Different segments of the look-up table can be optimized for different input ranges, allowing the system to allocate memory resources more efficiently. Each pipeline's local segment stores only the values relevant to its specific input range, reducing redundant storage and optimizing the memory-to-computation ratio for each operational context.
3Measurement precision
If interpolation is used to estimate activation function outputs, then accuracy is improved, but computational complexity increases
Solution Approach 1:
The interpolation process is made dynamic and adaptive based on the specific input value and the required precision. The system adjusts the interpolation order and methodology according to the input characteristics, using simpler methods when sufficient and more complex methods only when necessary to achieve the desired accuracy, thereby optimizing the balance between precision and complexity.
Solution Approach 2:
The interpolation unit changes its operational parameters (such as the number of lookup points used, the interpolation order, and the precision level) based on the input value and performance requirements. This allows the system to achieve high accuracy when needed while reducing computational complexity for less critical operations, effectively adapting the complexity to the task at hand.
Data Source
AI summary
An activation accelerator for a neural network accelerator includes a look-up table that stores a plurality of values representing a non-linear activation function, and activation pipelines. Each activation pipeline comprises a range conversion unit that receives an input value and generates a converted value from the input value, an index generation unit that generates an index and an interpolation value from the converted value, a look-up table interface unit that retrieves multiple values from the look-up table based on the index, and an interpolation unit that receives information identifying a rounding mode of a plurality of rounding modes, and generates an estimate of a result of the non-linear activation function for the input value from an interpolation output generated by interpolating between the multiple values retrieved from the look-up table using the interpolation value and rounding in accordance with the identified rounding mode.