Federated Private Adversarial Training for Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models face challenges in training and defending against adversarial attacks, especially in federated environments where data privacy and security are critical, due to cross-boundary restrictions and the risk of model poisoning.
Innovation Solution
A method for federated private adversarial training that isolates private data and generates obfuscated adversarial perturbations to harden machine learning models against attacks, while maintaining user privacy and controlling access to training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If federated learning is used to train machine learning models in distributed environments, then data privacy is improved, but the model becomes vulnerable to adversarial attacks and model poisoning
Solution Approach 1:
The patent applies preliminary anti-action by generating adversarial perturbations before the actual training process and incorporating them into the training data. This pre-emptive measure prepares the model to resist future adversarial attacks by exposing it to perturbed examples during training, thereby transforming the vulnerability to adversarial attacks while maintaining federated learning's privacy benefits
Solution Approach 2:
The patent introduces an intermediary mechanism that generates and manages adversarial perturbations separately from the private training data. This intermediary component creates perturbed versions of the data and facilitates their integration into the federated training process without exposing the underlying private data, thus resolving the contradiction between privacy protection and adversarial defense
2Reliability
If adversarial training is performed with true perturbations only, then model robustness is improved, but the training process becomes inefficient and time-consuming
Solution Approach 1:
The patent applies partial action by mixing true adversarial perturbations with false (random) perturbations in the training process. Instead of using only true perturbations which would be time-consuming to generate and process, the system uses a mixture where true perturbations provide robustness improvements and false perturbations fill in the training examples, thereby reducing training time while maintaining model robustness
Solution Approach 2:
The patent changes the parameter composition of the adversarial training set by controlling the ratio of true to false perturbations. This parameter adjustment allows the system to optimize between training efficiency and model robustness, finding the optimal balance point where sufficient robustness is achieved without excessive training time consumption
3Reliability
If private data is isolated and encrypted during federated training, then data security is improved, but the complexity of the training system increases
Solution Approach 1:
The patent extracts the adversarial perturbation generation function from the private data itself and implements it as a separate component. By taking out the perturbation generation process and making it independent, the system can encrypt and isolate private data more effectively while managing complexity through modularization, as the perturbation generation no longer requires direct access to the sensitive data
Data Source
AI summary
One or more computer processors transmit a machine learning model and an associated loss function to a worker, wherein the worker isolates private data. The one or more computer processors receive a plurality of encrypted gradients computed utilizing the transmitted machine learning model, the associated loss function, and the isolated private data. The one or more computer processors generate a plurality of adversarial perturbations, wherein the plurality of adversarial perturbations includes true perturbations and false perturbations. The one or more computer processors obfuscate the generated plurality of adversarial perturbations. The one or more computer processors transmit the obfuscated adversarial perturbations to the worker. The one or more computer processors harden the machine learning model utilizing the transmitted obfuscated adversarial perturbations and the private data.