Secure Adam Gradient Descent With Lookup Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional gradient descent methods in secure computing face challenges such as high processing costs due to calculations involving square roots and divisions, and slow convergence due to the large number of learning times required.
Innovation Solution
Implementing the Adam optimization technique using secure batch mapping to conceal gradients and parameters, utilizing secure computation apparatuses to perform gradient descent calculations efficiently while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If Adam optimization technique is used to improve convergence speed, then the number of learning times is reduced, but processing costs increase due to square root and division calculations
Solution Approach 1:
The patent pre-calculates and stores lookup tables for square root and division operations before the main learning process. During Adam optimization, these expensive calculations are replaced by table lookups, significantly reducing processing costs while maintaining convergence speed improvements.
Solution Approach 2:
The patent creates approximate copies of the expensive mathematical functions (square root and division) using pre-computed lookup tables. These tables provide approximate results that are sufficient for gradient descent convergence, avoiding the need for actual square root and division calculations during training.
2Device complexity
If simple gradient descent method is used to reduce processing costs, then calculation complexity is reduced, but convergence speed decreases due to large number of learning times required
Solution Approach 1:
The patent merges the advantages of both simple gradient descent and Adam optimization by implementing Adam's adaptive learning rate mechanism using lookup tables. This combines the fast convergence of Adam with the computational simplicity of basic gradient descent, achieving both goals simultaneously.
3Reliability
If secure batch mapping is used to conceal gradients and parameters, then security is improved, but processing overhead increases
Solution Approach 1:
The patent extracts the security functionality into a separate secure batch mapping layer that operates independently from the main optimization logic. This allows the Adam optimization with lookup tables to proceed efficiently while security constraints are applied as a separate transformation layer, minimizing processing overhead.
Data Source
AI summary
A calculation of a gradient descent method in secure computing is performed at high speed while maintaining accuracy. A secure gradient descent computation method calculates a gradient descent method while keeping a gradient and a parameter concealed. An initialization unit initializes concealed values [M], [V] of matrices M, V (S11). A gradient calculation unit determines concealed value [G] of a matrix G of a gradient g (S12). A parameter update unit calculates [M]β1 [M]+(1−β1) [G] (S13-1), calculates [V]←β2 [V]+(1−β2) [G]◯[G] (S13-2), calculates [M{circumflex over ( )}]←β{circumflex over ( )}1, t [M] (S13-3), calculates [V{circumflex over ( )}]←β{circumflex over ( )}2, t [V] (S13-4), calculates [G{circumflex over ( )}]←Adam ([V{circumflex over ( )}]) (S13-5), calculates [G{circumflex over ( )}]←[G{circumflex over ( )}]◯[M{circumflex over ( )}] (S13-6), and calculates [W]←[W]−[G{circumflex over ( )}] (S13-7).