Softmax Processor Accelerator with Offset Indexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processors face challenges in efficiently processing the softmax function, which is a complex operation in artificial neural networks, due to high computational complexity and resource consumption when using separate processors.
Innovation Solution
A processor with an integrated accelerator that includes an input module, a distribution information processing module, and a look-up table (LUT) module, which processes input data to generate distribution information, identify maximum values, calculate offset values, and determine index values to output the softmax function values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a separate processor is used to process the softmax function, then processing capability is improved, but time and resources are consumed due to data transmission
Solution Approach 1:
The patent integrates the softmax processing function directly into the neural network processor by incorporating a look-up table module and distribution information processing module within the same processor architecture. This merging eliminates the need for separate processor communication and data transmission, allowing softmax operations to be performed internally alongside neural network operations, thereby resolving the time loss caused by inter-processor data transmission while maintaining enhanced processing capability
2Power
If a separate processor is used to process the softmax function, then processing capability is improved, but resource consumption increases
Solution Approach 1:
The patent combines softmax processing resources within the neural network processor itself, sharing computational resources and memory structures. The look-up table module utilizes the processor's existing computational units and memory resources, eliminating the need for separate dedicated resources in a second processor, thereby reducing overall resource consumption while maintaining processing capability
Solution Approach 2:
The neural network processor is designed with multi-functionality to handle both neural network operations and softmax operations using the same computational resources. The processor can dynamically allocate its computational units and memory for different operations, making the resources universal and eliminating the need for dedicated separate resources for softmax processing
3Measurement precision
If the softmax function is processed with high computational complexity, then accuracy is improved, but processing speed deteriorates
Solution Approach 1:
The patent pre-computes and stores softmax values in a look-up table during the processor initialization phase. By performing the computationally intensive softmax calculations in advance and storing the results, the processor can quickly retrieve pre-computed values during actual inference, thereby maintaining high accuracy while dramatically improving processing speed and reducing real-time computational complexity
Data Source
AI summary
A method performed by at least one processor configured to implement an accelerator for processing a softmax function includes: obtaining input data comprising a plurality of quantized input values; generating input data distribution information indicating a plurality of frequencies corresponding to the plurality of quantized input values included in the input data; identifying a largest value from among the plurality of quantized input values as a first maximum value, based on the input data distribution information; determining an offset value based on a difference between the first maximum value and a second maximum value, wherein the second maximum value indicates a maximum quantization value that is representable by an input value of the input data; determining a plurality of index values by applying the offset value to each quantized input value of the plurality of quantized input values; and outputting a value of the softmax function corresponding to each of the plurality of quantized input values based on the plurality of index values.