Neural Network Activation Replacement for Low-Latency Private Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cryptographic neural network inference services suffer from high latency due to slow cryptographic activation functions, which hinder efficient and accurate private neural network inference.
Innovation Solution
A system and method that replace non-linear activation channels in neural networks with less resource-intensive polynomial approximations or pruning, using a planner to determine channel-wise activation replacements based on prediction settings, optimizing the number of replacements to maintain accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If cryptographic activation functions are used in neural network inference, then data privacy is protected, but inference latency increases by many orders of magnitude
Solution Approach 1:
The patent segments the activation function computation by replacing non-linear cryptographic activations with linear or simplified polynomial approximations in certain network layers, while retaining full cryptographic activations only where necessary for security-critical operations. This selective segmentation reduces overall computational latency while maintaining privacy protection where required.
Solution Approach 2:
The patent changes the mathematical parameters of activation functions by substituting complex non-linear cryptographic activations with simpler linear or low-degree polynomial functions. This parameter simplification dramatically reduces computation time while the system maintains acceptable accuracy through careful selection of replacement strategies and network architecture adjustments.
2Measurement precision
If non-linear activation channels are retained in all neural network layers, then inference accuracy is maintained, but computational efficiency and speed decrease
Solution Approach 1:
The patent applies partial action by replacing non-linear activation channels with linear or simplified polynomial versions in specific layers where full non-linearity is less critical. This partial replacement strategy achieves sufficient inference accuracy while dramatically improving computational efficiency, avoiding the need to maintain full non-linear activations throughout the entire network.
Solution Approach 2:
The patent implements local quality by applying different activation function types to different layers or channels of the neural network based on their specific functional requirements. Critical layers retain full non-linear cryptographic activations for accuracy and security, while less critical layers use simplified activations for efficiency, creating a heterogeneous activation strategy optimized for both accuracy and speed.
3Reliability
If cryptographic activation functions are used, then secure private inference is achieved, but the complexity of the computational system increases
Solution Approach 1:
The patent segments the computational system into different processing paths: secure cryptographic inference paths for privacy-critical operations and simplified linear/polynomial paths for less sensitive computations. This segmentation reduces overall system complexity by allowing parallel processing with different complexity levels while maintaining security where required.
Solution Approach 2:
The patent creates a universal activation framework that can dynamically select between cryptographic, linear, and polynomial activation functions based on the specific inference requirements. This multi-functional approach allows the same neural network architecture to operate in both fully secure mode and high-performance mode, reducing the need for separate specialized systems.
Data Source
AI summary
A method of a server device is provided. The method of a server device includes retrieving a prediction input and a prediction setting, replacing at least one non-linear activation channel in a neural network with at least one replacement channel based on the received prediction setting, generating a prediction based on the received prediction input based on the neural network with the at least one replacement channel, and outputting the generated prediction.