Secure Softmax Calculation Using Batch Mapping for Speed and Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional secure computing methods for softmax functions face challenges in achieving both high processing speed and accuracy due to the inclusion of exponential functions and divisions, leading to low accuracy, especially for larger values.
Innovation Solution
A secure softmax function calculation system utilizing secure batch mappings to efficiently calculate softmax functions by dividing both the numerator and denominator of the softmax equation, employing secure batch mappings to handle exponential and inverse functions, allowing for high-speed and accurate secure computing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional secure computing methods are used to calculate softmax functions, then security is maintained, but processing speed and accuracy deteriorate due to difficulties with exponential functions and divisions
Solution Approach 1:
The patent transforms the softmax calculation by changing the mathematical parameters and form. Specifically, it divides both numerator and denominator by exp(ui) to transform the exponential function calculation into a form that involves only multiplication and addition of encrypted values, which are operations that secure computing handles efficiently. This parameter transformation resolves the contradiction by maintaining security while dramatically improving processing speed.
2Productivity
If ReLU function is used to approximate exp(x) in secure computing, then processing speed is improved, but accuracy deteriorates especially for large values of x
Solution Approach 1:
Instead of using ReLU approximation, the patent changes the mathematical form by dividing numerator and denominator by exp(ui), transforming the problem into one that requires only multiplication and addition operations on encrypted values. This parameter transformation maintains both high processing speed and high calculation accuracy, avoiding the accuracy degradation that occurs with ReLU approximation for large x values.
3Measurement precision
If exact softmax calculation is performed in secure computing, then accuracy is maintained, but processing speed deteriorates due to computational complexity of exponential and division operations
Solution Approach 1:
The patent applies parameter transformation by dividing both numerator and denominator by exp(ui), which converts the computationally intensive exponential and division operations into simpler multiplication and addition operations on encrypted values. This maintains exact calculation accuracy while dramatically improving processing speed, as the transformed operations are much more efficient in secure computing environments.
4Productivity
If secure batch mapping is used to process shares of input vectors, then processing speed is improved, but system complexity increases
Solution Approach 1:
The patent segments the softmax calculation into distinct phases: (1) computing differences ui−uj, (2) calculating exp(ui−uj) for each pair, (3) summing the exponential values, and (4) performing the final division. By segmenting the computation and applying secure batch mapping to process multiple shares simultaneously in each phase, the system achieves high processing speed while managing complexity through structured organization of operations.
Data Source
AI summary
Techniques for performing secure computing of softmax functions at high speed and with high accuracy are provided. A secure softmax function calculation system that calculates a share ([[softmax (u1)]], . . . , [[softmax (uJ)]]) from a share ([[u1]], . . . , [[uJ]]) includes a subtraction means for calculating a share ([[u1−u1]], [[u2−u1]], . . . , [[uJ−uJ]]), a first secure batch mapping calculation means for calculating, [[exp (u1−u1)]], [[exp (u2−u1)]], . . . , [[exp (uJ−uJ)]], an addition means for calculating a share([[∑ j=1Jexp(uj-u1)]],… ,[[∑ j=1Jexp(uj-uJ)]],and a second secure batch mapping calculation means for calculating a share ([[softmax (u1)], . . . , [[softmax (uJ)]]).