Layer-Specific LUT Quantization for Neural Activation Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerators face challenges in efficiently processing nonlinear activation functions, leading to reduced accuracy and the need for retraining or iterative processes to maintain performance.
Innovation Solution
An electronic device that processes nonlinear activation functions through dividing input value ranges into sections, using look-up tables (LUTs) to perform integer operations, with adjustable precision based on error thresholds, allowing for accurate and efficient processing without retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If polynomial approximation is used for nonlinear activation function, then processing efficiency is improved, but accuracy is greatly reduced
Solution Approach 1:
The patent divides the input value range into multiple sections and creates separate LUTs for each section. This segmentation allows the system to maintain high processing efficiency through table lookup while improving accuracy by having dedicated tables for different input ranges, avoiding the accuracy loss of polynomial approximation.
Solution Approach 2:
The patent creates LUTs that store pre-computed activation function values as copies, allowing direct lookup instead of real-time computation. This copying approach maintains accuracy by storing exact values while improving efficiency through O(1) retrieval, resolving the contradiction between polynomial approximation speed and accuracy.
2Device complexity
If a single LUT is used for multiple nonlinear activation functions, then device complexity is reduced, but accuracy is limited
Solution Approach 1:
The patent designs a LUT structure that can serve multiple activation functions by organizing tables hierarchically. Each layer has its own LUTs tailored to its specific activation function, but the overall architecture provides a universal solution that handles different functions accurately without requiring completely separate hardware for each.
Solution Approach 2:
The patent applies different LUT configurations to different layers based on their specific needs. Each layer receives LUTs optimized for its activation function and input range characteristics, allowing high accuracy for each specific function while maintaining a relatively simple overall structure through the standardized LUT interface.
3Measurement precision
If input value range is divided into many sections, then accuracy is improved, but device complexity increases
Solution Approach 1:
The patent implements dynamic section division where the number and size of LUT sections are adjusted based on the specific activation function and input range characteristics of each layer. This dynamic adaptation allows the system to achieve high accuracy where needed while maintaining simplicity where possible, avoiding the need for uniformly high complexity across all layers.
Solution Approach 2:
The patent changes parameters such as LUT section count, section size, and quantization precision based on the specific requirements of each activation function and layer. This parameter adaptation allows the system to optimize accuracy for critical layers while using coarser approximations for less sensitive layers, balancing overall accuracy with device complexity.
Data Source
AI summary
An electronic device, including: at least one processor configured to accelerate at least one operation of a neural network model; and a memory configured to store instructions which cause the electronic device to: identify a range of input values corresponding to each layer from among a plurality of layers related to a nonlinear activation function of the neural network model, divide the range of input values into a plurality of sections having a first predetermined number for the each layer based on the range of input values, acquire, for the each layer, a first look up table (LUT) including a plurality of integer input values and a plurality of integer output values by quantizing input data corresponding to the neural network model based on the plurality of sections, and process an operation according to the nonlinear activation function as an integer operation based on the first LUT.