Electronic Device Softmax Computation via Repeated Powering and Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices face challenges in performing Softmax function computations efficiently, particularly when dealing with large input ranges and homomorphic ciphertext, leading to numerical instability and increased computational burden.
Innovation Solution
The electronic device employs powering and normalizing operations repeatedly on an initial approximation range to obtain a Softmax function computation result, which is then transmitted to an external device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Softmax function computation is performed using exponential function on large input range, then computation accuracy is improved, but numerical stability deteriorates and computational burden increases
Solution Approach 1:
The patent transforms the Softmax computation by changing the parameter representation from direct exponential function to a polynomial approximation form. Specifically, it uses the transformation: Softmax(x) = exp(x) / sum(exp(x)) is converted to a polynomial-based approximation that avoids direct computation of large exponential values, thereby maintaining numerical stability while preserving computation accuracy.
Solution Approach 2:
The patent employs pre-computed approximation polynomials that can be quickly evaluated without requiring expensive exponential function calculations. These polynomial approximations serve as disposable computational shortcuts that provide sufficient accuracy for the Softmax operation without the numerical instability of direct exponential computation on large input ranges.
2Reliability
If Softmax function computation is performed on homomorphic ciphertext, then data security is improved, but computational burden increases significantly
Solution Approach 1:
The patent changes the computational parameters by transforming the exponential-based Softmax function into a polynomial approximation form that is more compatible with homomorphic encryption operations. This transformation reduces the computational complexity from O(n) exponential operations to O(1) polynomial evaluations, significantly improving computation speed while maintaining data security through homomorphic ciphertext processing.
Solution Approach 2:
The patent substitutes the mechanical exponential function computation with a mathematical polynomial approximation system. This substitution replaces the computationally intensive exponential operations with simpler polynomial evaluations that can be efficiently performed on homomorphic ciphertext, thereby reducing the computational burden while preserving the security benefits of encrypted processing.
3Productivity
If repeated powering and normalizing operations are performed on approximation range, then computation speed is improved, but device complexity increases
Solution Approach 1:
The patent segments the Softmax computation into distinct operational phases: (1) polynomial approximation evaluation, (2) powering operations on the approximation, and (3) normalizing operations to obtain the final result. This segmentation allows each phase to be optimized independently and executed efficiently, improving overall computation speed while making the algorithmic complexity more manageable through structured organization.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed is an electronic device. The device includes an interface; a memory; and at least one processor. The processor is configured to store data in the memory if the data is received from an external device through the interface, obtain a Softmax function computation result by performing powering and normalizing operations at least once repeatedly on a range from an initial approximation range to a target approximation range during a Softmax function computation process for the data, and transmit the obtained Softmax function computation result to the external device through the interface. Accordingly, the device may perform a Softmax function computation quickly and accurately.