Training method and apparatus for a neural network model, device and storage medium
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning in artificial intelligence faces challenges in protecting the privacy of participants' data while enabling efficient joint training of neural network models.
Innovation Solution
A method and apparatus for training neural network models using homomorphic encryption to protect data privacy, allowing joint learning between parties without exposing their data, by encrypting feature and tag data and performing ciphertext computations, while updating network parameters locally.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If federated learning is used to protect participant privacy, then data security is improved, but computation complexity increases
Solution Approach 1:
The patent introduces homomorphic encryption as an intermediary mechanism that enables secure computation without requiring direct data sharing. The encryption system acts as a mediator between privacy protection requirements and computation needs, allowing the neural network to process encrypted data directly through ciphertext forward propagation and ciphertext backpropagation, thus protecting participant privacy while maintaining computational efficiency
Solution Approach 2:
The patent changes the state of data from plaintext to ciphertext form throughout the training process. By transforming data parameters from unencrypted to encrypted state, the system achieves privacy protection while enabling secure joint training through homomorphic encryption operations that preserve mathematical properties needed for neural network computation
2Reliability
If homomorphic encryption is used to encrypt data, then data privacy is improved, but training efficiency deteriorates
Solution Approach 1:
The patent replaces traditional decryption-encryption mechanical operations with direct homomorphic ciphertext computation. Instead of decrypting data for processing and re-encrypting results, the system substitutes this mechanical process with homomorphic encryption operations that allow direct computation on ciphertext, significantly improving training efficiency while maintaining data privacy
Solution Approach 2:
The patent enables continuous useful action by allowing the neural network to perform forward propagation and backpropagation continuously on encrypted data without interruption for decryption. The homomorphic encryption system maintains the continuity of the training process, eliminating the need to stop and decrypt data at various stages, thus improving overall training efficiency
3Measurement precision
If parties share data for joint training, then model accuracy is improved, but information leakage increases
Solution Approach 1:
Homomorphic encryption serves as an intermediary that enables information exchange for model training without direct data sharing. The encryption system mediates between the need for collaborative training (to improve model accuracy) and the need to prevent information leakage, allowing parties to jointly train the model while keeping their data private through encrypted computation
Solution Approach 2:
The patent uses ciphertext as a copy of the original data that preserves computational properties. Instead of sharing the actual data (which would cause information leakage), parties share and compute on encrypted copies that can be processed to improve model accuracy while preventing access to the underlying sensitive information
Data Source
AI summary
Provided are a training method and apparatus for a neural network model, a device and a storage medium. The training method includes: acquiring a first feature representation ciphertext of a sample user from a first party; determining the tag ciphertext of the sample user and determining the loss error ciphertext and the gradient ciphertext of a second neuron in a second sub-neural network based on the second sub-neural network according to the first feature representation ciphertext and the tag ciphertext; controlling the first party to decrypt the gradient ciphertext of the second neuron to obtain a decryption result and updating the network parameter of the second neuron according to the decryption result acquired from the first party; and sending the loss error ciphertext of an association neuron to the first party.