Two-Phase Machine Learning for Membership Inference Defense
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning methods are vulnerable to membership inference attacks (MI attacks) that leak confidential information, with current defenses either sacrificing accuracy for improved resistance or introducing noise that affects inference results.
Innovation Solution
A two-phase learning process that trains a learning model, a discriminator, and a defender using teacher labels and adversarial learning to make inference results indistinguishable between training and non-training data, ensuring both accuracy and resistance to MI attacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If adversarial regularization is used to defend against MI attacks, then resistance to MI attacks is improved, but inference accuracy deteriorates
Solution Approach 1:
The patent segments the training process into two distinct phases: a first learning phase that optimizes for inference accuracy without adversarial constraints, and a second learning phase that optimizes for MI attack resistance. This segmentation allows each phase to focus on its specific objective without compromising the other, resolving the contradiction between accuracy and defense performance.
Solution Approach 2:
The patent implements dynamic switching between different training objectives by controlling the execution of the first and second learning phases. The system can adaptively adjust the balance between accuracy optimization and defense optimization by controlling which phase is executed and how many times, allowing flexible trade-off management based on specific requirements.
2Reliability
If noise is added to inference results for defense, then resistance to MI attacks is improved, but inference accuracy deteriorates
Solution Approach 1:
The patent applies preliminary action by training the defender model in advance during the second learning phase to understand the characteristics of MI attacks and develop countermeasures. This preliminary training enables the defender to provide targeted corrections during inference without adding random noise, thereby maintaining accuracy while achieving defense.
3Reliability
If parameter λ is increased to prioritize discriminator performance, then resistance to MI attacks is improved, but inference accuracy deteriorates
Solution Approach 1:
The patent dynamically controls the optimization target through the parameter λ in the loss function during the second learning phase. By adjusting λ, the system can flexibly balance the discriminator's performance (which improves MI attack resistance) against the inference algorithm's accuracy, allowing adaptive trade-off management based on specific security and performance requirements.
Data Source
AI summary
The invention includes a first learning phase that a machine learning is performed using first dataset to create a learning model f; and a second learning phase that the first or a second dataset is randomly selected; a result by inputting the first and the second datasets to the learning model f is inputted to a discriminator h having a parameter ω; a machine learning is performed using a result and a ground truth data to train the parameter ω, the result being obtained by having the discriminator h discriminate whether the input data belongs to the first or the second dataset; and when the first dataset is selected, a result by inputting the data of the first dataset to the learning model f is inputted to a defender u to train a parameter τ thereof by using an output of the discriminator h through the defender u.