Electronic Device Softmax Processing with Repeated Powering Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices face challenges in performing Softmax function computations efficiently, especially when dealing with large input ranges and homomorphic ciphertext, leading to numerical instability and increased computational burden.
Innovation Solution
The electronic device employs powering and normalizing operations repeatedly on an initial approximation range to a target range, using methods like squaring and normalizing, to obtain Softmax function results accurately and quickly, even with homomorphic ciphertext.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Softmax function computation is performed using exponential function on large input values, then the computation can be performed, but numerical instability occurs and computational burden increases significantly
Solution Approach 1:
The patent transforms the Softmax computation by changing the parameter representation from direct exponential computation to a two-step process: first computing logarithms of input values, then applying exponential only to the log-values. This parameter transformation (using log-space representation) prevents numerical overflow while maintaining computational accuracy, directly resolving the contradiction between numerical stability and computational burden
Solution Approach 2:
The computation is segmented into distinct stages: logarithm computation stage, exponential computation stage on transformed values, and normalization stage. By dividing the computation into segments, each handling specific aspects of the transformation, the patent avoids computing exponentials on large original values, thereby improving numerical stability while managing computational complexity through structured processing
2Reliability
If Softmax function computation is performed on homomorphic ciphertext, then data security is maintained, but the computation result becomes extremely large or small making utilization difficult
Solution Approach 1:
The patent applies parameter transformation by working in logarithmic space for homomorphic ciphertext computations. Instead of directly computing exponentials on encrypted large values that produce extremely large or small results, the system computes logarithms first, performs exponential operations on the transformed (smaller) values, and maintains encrypted representations throughout. This parameter change enables secure homomorphic computation while producing usable results within manageable numerical ranges
Solution Approach 2:
The logarithmic transformation acts as an intermediary step between the original large input values and the exponential computation. By introducing this intermediate representation layer, the patent enables secure homomorphic encryption to be applied while avoiding the extreme numerical values that would otherwise result, thus maintaining both data security and result usability
3Productivity
If traditional Softmax computation methods are used, then the function can be computed, but the computation speed is slow and efficiency is low
Solution Approach 1:
The patent performs preliminary logarithmic transformation of input values before the main exponential computation. This preliminary action (computing logs first) prepares the data in a form that enables faster and more stable exponential operations, ultimately improving computation speed while maintaining or enhancing accuracy compared to direct exponential computation on original large values
Data Source
AI summary
Disclosed is an electronic device. The device includes an interface; a memory; and at least one processor. The processor is configured to store data in the memory if the data is received from an external device through the interface, obtain a Softmax function computation result by performing powering and normalizing operations at least once repeatedly on a range from an initial approximation range to a target approximation range during a Softmax function computation process for the data, and transmit the obtained Softmax function computation result to the external device through the interface. Accordingly, the device may perform a Softmax function computation quickly and accurately.