Residual Decomposition for Privacy-Preserving Federated Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Federated learning methods using differential privacy for model training suffer from performance loss in model accuracy due to inherent noise, making it difficult to protect privacy while ensuring accurate data utilization.
Innovation Solution
The method involves encrypting labels using residual decomposition, allowing the data sender to perturb residuals, and the data receiver to reconstruct model parameters based on model parameter correction amounts, ensuring privacy protection and high accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If differential privacy is used to protect privacy data in Federated learning, then privacy protection is improved, but model accuracy deteriorates due to inherent noise
Solution Approach 1:
The patent introduces an intermediary mechanism using homomorphic encryption and residual decomposition. Instead of directly sharing noisy differentially private labels, the system decomposes labels into residuals and uses homomorphic encryption to allow the unlabeled party to compute model parameters on encrypted data without decrypting the actual labels. This intermediary cryptographic layer preserves both privacy and accuracy by enabling computation on protected data.
Solution Approach 2:
The patent segments the label information into residual components through residual decomposition. By breaking down the label into residual parts and processing these segments through homomorphic encryption, the system maintains the mathematical properties needed for accurate model training while preventing direct access to the original private label data, thus resolving the accuracy-privacy tradeoff.
2Measurement precision
If centralized collection of detailed data is used for model training, then model accuracy is improved, but data security and privacy protection deteriorate
Solution Approach 1:
The patent uses homomorphic encryption as an intermediary that enables centralized model training without centralized data collection. The unlabeled party can perform computations on encrypted label data from the labeled party, achieving the benefits of centralized training while maintaining distributed data security. The encryption layer acts as a mediator that allows data processing without data exposure.
Solution Approach 2:
The patent extracts only the necessary computational value from the private label data through residual decomposition and homomorphic encryption. Instead of collecting and storing detailed private data, the system extracts the essential information needed for model training in an encrypted form, allowing accurate modeling while removing the security risks associated with centralized data storage.
3Measurement precision
If residual decomposition with homomorphic encryption is used, then model accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent performs residual decomposition as a preliminary action before the main model training process. By pre-processing the label data into residual components and encrypting them in advance, the system reduces the computational burden during the actual training phase. The complex cryptographic operations are performed once on the label data, enabling simpler subsequent training operations on the encrypted residuals.
Data Source
AI summary
A data processing method and apparatus, an electronic device, and a storage medium acquire label residual values determined for respective data samples, each label residual value being used for representing a degree of deviation between a real label of a corresponding data sample and a label predicted by a target model; perform residual decomposition on the respective data samples based on respective label residual values to obtain model parameter correction amounts corresponding to the respective data samples; and send the model parameter correction amounts to a data receiver. The data receiver reconstructs model parameter information of the target model based on the model parameter correction amounts. Perturbing residuals through residual decomposition enables a data sender to protect the real labels owned thereby, while the data sender can send necessary correction information to the data receiver to reconstruct accurate model parameter information to ensure the model training performance.