Associative Memory Softmax Computation via Partial Mantissa Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Softmax regression computations on large datasets are inefficient due to non-linearity, high memory requirements, and accuracy issues such as overflow, especially when using current associative memory devices, which struggle to achieve micro-millisecond speed and low latency required by deep learning algorithms.
Innovation Solution
The method involves dividing a multi-bit mantissa into smaller partial mantissas, offline calculating and storing partial exponents in a lookup table, and using these to concurrently compute exponents and perform softmax, softplus, sigmoid, or tanh operations, with normalization to achieve exact softmax regression results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Taylor Series computation is used for high accuracy exponent calculation, then measurement precision is improved, but device complexity and computation time increase significantly
Solution Approach 1:
The mantissa A is divided into multiple partial mantissas A_j, allowing the exponent calculation to be segmented into smaller independent computations. Each partial mantissa can be processed separately using pre-computed lookup tables, reducing the overall computational complexity while maintaining accuracy.
Solution Approach 2:
The patent pre-computes and stores exponent values for all possible partial mantissa values in lookup tables during an offline phase. This preliminary action eliminates the need for complex real-time Taylor Series computations during actual softmax operations, significantly reducing device complexity and computation time while preserving high accuracy.
2Productivity
If softmax regression is implemented on large datasets with current associative memory devices, then productivity is improved, but measurement precision deteriorates due to overflow and dynamic range issues
Solution Approach 1:
By segmenting the mantissa into partial mantissas and computing partial exponents independently through lookup tables, the patent avoids the dynamic range issues that arise when computing the full exponent directly. This segmentation prevents overflow while maintaining computational efficiency for large datasets.
Solution Approach 2:
The patent introduces partial exponents as intermediary values that mediate between the input mantissa and the final softmax computation. These partial exponents serve as intermediate representations that maintain precision while enabling efficient parallel computation across large datasets.
3Measurement precision
If Taylor Series with many coefficients is used for high accuracy, then measurement precision is improved, but loss of time increases due to multiple multiplication and division operations
Solution Approach 1:
The patent performs the time-consuming Taylor Series computations in advance during an offline phase, storing the results in lookup tables. During actual softmax operations, only simple table lookups and multiplications are required, dramatically reducing computation time while maintaining the high accuracy benefits of using many Taylor Series coefficients.
Solution Approach 2:
The patent creates pre-computed copies of exponent values for all possible partial mantissa configurations. Instead of performing complex calculations repeatedly during softmax operations, the system copies and retrieves pre-computed values from lookup tables, eliminating redundant computations and reducing time loss.
4Measurement precision
If large memory units are used to store intermediate results for softmax regression, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent segments the computation into partial mantissas and partial exponents, allowing intermediate results to be stored in a distributed manner across smaller memory units rather than requiring a single large memory unit. This segmentation reduces memory complexity while maintaining the precision needed for accurate softmax calculations.
Data Source
AI summary
A method for an associative memory device includes dividing a multi-bit mantissa A of a number X to a plurality of smaller partial mantissas Aj, offline calculating a plurality of partial exponents F(Aj) for each possible value of each partial mantissa Aj and storing the plurality of partial exponents F(Aj) in a look up table (LUT) of the associative memory device. A system includes an associative memory array to store a plurality of partial mantissas Ai of a mantissa A of a number X and an exponent calculator to utilize the partial mantissas to compute e in the power of X.