Softmax Fractional Exponential Computation on Matrix Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing fractional exponentials within a softmax activation function in software diminishes the performance of neural networks due to increased computation time, reducing throughput.
Innovation Solution
Utilizing a hardware accelerator configured for matrix multiplication, specifically a systolic array with processing elements, to apply a binary scaling operation and a polynomial convert function to compute fractional exponentials efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fractional exponentials are computed in software, then implementation flexibility is maintained, but computation time increases and throughput decreases
Solution Approach 1:
The patent replaces software-based computation with hardware-based computation using a specialized processing unit. The fractional exponential computation is implemented as hardware circuitry that performs polynomial conversion and exponentiation operations in parallel, eliminating the sequential processing bottleneck of software and dramatically reducing computation time while increasing throughput.
Solution Approach 2:
The patent changes the computational approach by representing fractional exponentials as polynomial conversions in hardware. Instead of using general-purpose software algorithms, the system transforms the mathematical operation into a hardware-friendly polynomial evaluation that can be computed efficiently through dedicated circuit operations, changing the parameter of computation method from software algorithm to hardware polynomial evaluation.
2Productivity
If fractional exponentials are computed in hardware, then computation time is reduced and throughput increases, but device complexity increases
Solution Approach 1:
The patent segments the fractional exponential computation into distinct hardware modules: a polynomial conversion unit that converts fractional exponents to polynomial form, and a separate exponentiation unit that computes the result. This segmentation allows each module to be optimized independently and enables parallel processing, reducing overall computation time while managing hardware complexity through modular design.
Solution Approach 2:
The patent designs the hardware processing unit to be multi-functional, capable of performing polynomial conversion, exponentiation, and other neural network operations. This universal design allows the same hardware infrastructure to support multiple functions, reducing the need for separate dedicated circuits for each operation and thereby managing device complexity while maintaining high throughput.
Data Source
AI summary
The present disclosure is directed to a method for computing a fractional exponential for a softmax activation function. The method includes applying a binary scaling operation to a plurality of logits to generate a plurality of scaled logits. The method further includes applying, by a hardware accelerator configured for matrix multiplication, a polynomial convert function to each of the plurality of scaled logits. The method further includes obtaining, via the hardware accelerator, feedback based on applying the polynomial convert function, the feedback comprising a fractional exponential for each of the plurality of scaled logits.