Soft-Label Machine Learning Ensembles Against Membership Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing countermeasures against membership inference attacks in machine learning require learning data without privacy information, which is difficult to obtain in fields like medicine and finance, and involve significant computational overhead.
Innovation Solution
A machine learning apparatus that generates multiple first learning models from privacy-containing data, integrates these models, and assigns soft labels to new data using integrated models, reducing the need for non-private data and computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If learning data without privacy information is used for training, then resistance to membership inference attacks is improved, but availability of training data deteriorates
Solution Approach 1:
The learning data is divided into multiple pieces (n pieces), and models are trained on different combinations of these pieces. This segmentation allows the system to use privacy-containing data in a controlled manner while still achieving resistance to membership inference attacks through the ensemble of multiple models.
Solution Approach 2:
Multiple models are trained in a nested structure where each model is trained on different subsets of the divided learning data. The models are then integrated through voting or averaging, creating a layered defense mechanism that provides privacy protection while utilizing the full dataset.
2Reliability
If learning data without privacy information is used, then privacy protection is improved, but model accuracy deteriorates
Solution Approach 1:
Multiple models trained on different subsets of privacy-containing data are merged through integration (voting or averaging). This merging allows the system to maintain high model accuracy by leveraging information from all data pieces while achieving privacy protection through the collective decision-making process.
Solution Approach 2:
The system changes the parameter of data privacy status by using privacy-containing data for training multiple individual models, but then changes the output parameter through integration to produce a final model that achieves both accuracy and privacy protection.
3Reliability
If n×(n−1) pieces of learning data are trained as in Non-Patent Literature 2, then resistance to membership inference attacks is improved, but computational cost increases
Solution Approach 1:
Instead of training all n×(n−1) combinations of learning data as in the related art, the present invention trains only n models on n divided pieces of learning data. This partial action approach achieves the same privacy protection effect with significantly reduced computational cost by avoiding redundant training on overlapping data subsets.
4Reliability
If multiple models are integrated from n pieces of learning data, then resistance to membership inference attacks is improved, but device complexity increases
Solution Approach 1:
The system uses homogeneous integration methods (voting or averaging) to combine multiple models. This homogeneous approach simplifies the integration process compared to complex heterogeneous methods, reducing device complexity while maintaining the privacy protection benefits of ensemble modeling.
Data Source
AI summary
A first learning unit (22) generates a first learning model by employing each of n pieces of learning data, as a subject, and performing training using subject learning data. A model integration unit (23) generates an integrated model by integrating m pieces of first learning models selected from n pieces of first learning models. A data generation unit (24) generates new learning data by rewriting a label assigned to subject data with a soft label which is a result obtained by giving to the integrated model, the subject data which is learning data other than learning data used for the training in the generation of the m pieces of first learning models that are the basis of the integrated models, as input. A second learning unit (25) generates a second learning model by performing training using the new learning data.