Neural Network Homomorphic Encryption Processing Layer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network models face challenges in processing encrypted data due to high security risks and large processing loads, particularly when using homomorphic encryption, as general activation functions like sigmoid or ReLU are not compatible with existing homomorphic encryption methods, leading to increased communication and processing time.
Innovation Solution
Implementing a neural network model with a processing layer that performs modulo operations on encryption data, allowing decryption to yield results from plaintext data operations without applying nonlinear functions post-processing, and using differentiable activation functions during backward propagation to update setting values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If homomorphic encryption is used to process encrypted data in neural networks, then security is improved, but processing compatibility deteriorates because general activation functions cannot be executed
Solution Approach 1:
The patent changes the parameter of activation function from general nonlinear functions (sigmoid, ReLU) to specific polynomial functions that are compatible with homomorphic encryption operations, enabling encrypted data processing while maintaining security
Solution Approach 2:
The patent introduces an intermediary conversion process where the output of homomorphic encryption operations is transformed back to plaintext space through decryption and re-encryption cycles, allowing general activation functions to be applied indirectly without breaking encryption compatibility
2Adaptability or versatility
If Fully homomorphic encryption or SomeWhat homomorphic encryption is used to execute more operations, then processing capability is improved, but data amount increases causing large processing load
Solution Approach 1:
The patent uses partial homomorphic encryption capabilities (additive and multiplicative operations only) rather than full homomorphic encryption, achieving sufficient processing capability for neural networks while keeping ciphertext size manageable
Solution Approach 2:
The patent segments the neural network processing into distinct phases: encrypted computation phases using homomorphic encryption and decryption phases for activation functions, optimizing resource usage by applying encryption only where necessary
3Adaptability or versatility
If general activation functions are used in neural networks, then processing flexibility is improved, but encrypted data processing becomes impossible
Solution Approach 1:
The patent performs preliminary approximation of general activation functions using polynomial expressions before encryption, allowing the approximated functions to be computed homomorphically on encrypted data while preserving the essential nonlinear behavior needed for neural network processing
Data Source
AI summary
An inference apparatus that executes a predetermined inference process using a neural network model including: an input unit configured to receive encryption data; and a processor configured to execute a process including: executing predetermined process with respect to the encryption data using the neural network model, wherein the neural network model has a processing layer that executes a process of generating second encryption data from a plurality of pieces of first encryption data input from a previous stage and outputting the second encryption data, the second encryption data providing, by being decrypted, a processing result obtained by performing operations with modulo operation with respect to plurality of pieces of plain text data corresponding to the plurality of pieces of first encryption data, the neural network model being configured such that a nonlinear function is not applied to the second encryption data in a posterior stage of the processing layer.