Polynomial Activation for Encrypted CNN Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for securing convolutional neural networks (CNNs) using homomorphic encryption are limited by the incompatibility with non-linear and pooling layers, particularly the ReLU activation function and MaxPool operation, which restricts the size and efficiency of CNNs, especially in deep networks.
Innovation Solution
Implementing a CNN with a non-linear layer using a polynomial function of degree at least two to approximate the activation function and batch normalization, along with an average pooling layer, allowing secure learning and classification in the encrypted domain without compromising network size or efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If homomorphic encryption is used to secure CNN data, then data privacy is protected, but the network size and efficiency are restricted due to incompatibility with non-linear and pooling layers
Solution Approach 1:
The patent changes the parameter of the activation function from the standard ReLU (max(0,x)) to a polynomial function (x^2 or x^3). This parameter change makes the function compatible with homomorphic encryption operations while preserving the essential non-linear transformation capability needed for CNN functionality.
Solution Approach 2:
The patent changes the pooling operation from MaxPool to AveragePool. This parameter change allows the pooling operation to be performed on encrypted data using homomorphic encryption, as average pooling involves addition and division operations that are compatible with the encryption scheme, unlike maximum operations.
2Reliability
If homomorphic encryption is used to secure CNN data, then data privacy is protected, but the network depth is restricted preventing deep network architectures
Solution Approach 1:
The patent changes the activation function parameter to a polynomial form that is mathematically compatible with homomorphic encryption operations. This enables the stacking of multiple layers including non-linear transformations, thereby allowing deep network architectures to be constructed while maintaining data privacy through encryption.
3Productivity
If standard ReLU activation function is used in CNN, then network performance is optimized, but compatibility with homomorphic encryption is lost
Solution Approach 1:
The patent modifies the activation function parameter from the piecewise linear ReLU function to a polynomial function (x^2 or x^3). This change maintains the non-linear transformation property essential for network performance while making the function compatible with homomorphic encryption operations that require continuous differentiable functions.
Solution Approach 2:
Instead of trying to make homomorphic encryption compatible with standard ReLU, the patent inverts the approach by designing a custom polynomial activation function that is inherently compatible with homomorphic encryption operations, thereby solving the compatibility issue from the function design side rather than the encryption side.
Data Source
Figure 1
Figure 2a
Figure 2b
AI summary
The present invention relates to a secure method for learning parameters of a convolutional neural network, CNN, for data classification; the method comprising implementing, by data processing means (11a) of a first server (1a), the steps of: (a0) Receiving from a second server (1b) a training database already classified, said training data being homomorphically encrypted; (a1) Learning in the encrypted domain, from said training database, the parameters of a reference CNN comprising at least: - a non-linear layer (POLYNOMIAL) operating a polynomial function of degree at least two approximating an activation function; - a batch normalization (BN) layer before each non-linear layer (POLYNOMIAL);(a2) Transmission to the second server (1b) of the learned parameters, for decryption and use in classification. The present invention also relates to methods of secure classification of input data.