Hardware Acceleration Circuit for Softmax Function Calculation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face inefficiencies and increased power consumption when calculating Softmax function values in artificial neural networks, particularly due to job migration overheads between deep learning accelerators or neural network processing units and central processing units or graphics processing units.
Innovation Solution
A hardware acceleration circuit is provided, including a storage module for lookup tables, a lookup table circuit to generate exponential function values and reciprocals, an adder for summing these values, and a multiplier to compute Softmax function values, thereby accelerating the calculation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If Softmax function calculation is performed using CPU or GPU, then calculation flexibility is maintained, but job migration overhead between DLA/NPU and CPU/GPU increases and data processing speed decreases
Solution Approach 1:
The patent introduces lookup tables as an intermediary mechanism between the neural network processing unit and the Softmax function calculation. The lookup tables pre-store exponential function values and reciprocal values, allowing the system to quickly retrieve pre-computed results without migrating jobs between DLA/NPU and CPU/GPU, thus eliminating job migration overhead while maintaining fast processing speed
Solution Approach 2:
The patent applies preliminary action by pre-calculating and storing exponential function values and reciprocal values in lookup tables before actual Softmax calculations are needed. This pre-computation allows the system to perform Softmax calculations by simply retrieving and combining pre-stored values, dramatically reducing calculation time and eliminating the need for job migration during runtime
2Productivity
If Softmax function calculation is performed using CPU or GPU, then general-purpose computing capability is utilized, but system bandwidth consumption increases and power consumption increases
Solution Approach 1:
The lookup tables serve as an intermediary that stores pre-computed exponential and reciprocal values locally within the DLA/NPU. This eliminates the need for frequent data transfers between DLA/NPU and CPU/GPU, reducing system bandwidth consumption and the associated power consumption while maintaining high processing speed
Solution Approach 2:
The patent uses copying by creating lookup tables that contain copies of pre-computed exponential function values and reciprocal values. Instead of performing complex calculations during runtime, the system copies and combines these pre-stored values to produce Softmax results, significantly reducing computational overhead and power consumption
Data Source
AI summary
A hardware acceleration circuit, a data processing acceleration method, a chip, and an accelerator are provided. The circuit includes: a storage module, configured to store first and second lookup tables; a lookup table circuit, configured to output, in response to respective index values of data elements and based on the first lookup table, exponential function values corresponding to the data elements; and output, in response to an index value of an addition operation result and based on the second lookup table, a reciprocal corresponding to the addition operation result; an adder, configured to obtain the addition operation result by adding the exponential function values for the lookup table circuit; and a multiplier, configured to output a multiplication operation result of an exponential function value of an ith data element and the reciprocal of the addition operation result, to obtain a Softmax function value of the ith data element.