Lookup Table Data Processing for Neural Network Activation Functions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems for neural networks and automatic speech recognition face inefficiencies in performing division operations, particularly with activation functions like Softmax, due to the need for dedicated operators that cause latency and require significant memory resources.
Innovation Solution
The method involves normalizing input data, determining dividend and divisor data using lookup tables, and calculating output data without direct division operations, utilizing a processor to read values from memory-based lookup tables, thereby avoiding the use of dividers and optimizing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If dedicated division operators are used for activation functions like Softmax, then computational accuracy is maintained, but processing latency increases and memory requirements increase
Solution Approach 1:
The patent creates a lookup table that stores pre-computed division results for activation functions. Instead of using a dedicated division operator to calculate each result in real-time, the system copies pre-computed values from the lookup table, thereby maintaining computational accuracy while significantly reducing processing latency.
Solution Approach 2:
The patent performs the division operations in advance during system initialization or training phase, storing the results in the lookup table. This preliminary action eliminates the need for time-consuming division operations during actual inference or processing, thus reducing latency while preserving accuracy.
2Measurement precision
If dedicated division operators are used for activation functions like Softmax, then computational accuracy is maintained, but memory resource consumption increases
Solution Approach 1:
The patent uses a lookup table that stores pre-computed division results. This approach replaces the need for large dedicated division operator hardware structures with a more memory-efficient table of pre-computed values, reducing overall memory resource consumption while maintaining computational accuracy.
Solution Approach 2:
The patent changes the storage format from storing raw input values to storing pre-computed output values in the lookup table. This parameter change allows the system to use less memory by storing only the necessary division results rather than maintaining complex division operator structures.
3Power
If direct division operations are performed, then computational results are obtained, but processing speed decreases
Solution Approach 1:
The patent implements a lookup table that stores pre-computed division results. During processing, the system simply copies the required value from the table rather than performing division operations, which dramatically increases processing speed and computational throughput while maintaining accurate results.
Solution Approach 2:
The patent performs division computations in advance and stores the results in the lookup table. This preliminary computation eliminates the need for slow real-time division operations during processing, thereby significantly improving processing speed and throughput.
Data Source
AI summary
A processor-implemented data processing method includes: normalizing input data of an activation function comprising a division operation; determining dividend data corresponding to a dividend of the division operation by reading, from a memory, a value of a first lookup table addressed by the normalized input data; determining divisor data corresponding to a divisor of the division operation by accumulating the dividend data; and determining output data of the activation function corresponding to an output of the division operation obtained by reading, from the memory, a value of a second lookup table addressed by the dividend data and the divisor data.