Machine-Learning Model Hardening Against Training Data Poisoning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learning models are vulnerable to training data attacks, particularly when using publicly accessible data, which can lead to unintended model behavior and dangerous predictions, and current defense techniques fail to inherently immunize the models against such attacks.
Innovation Solution
A method and system that identifies high-impact training data during the initial training phase, generates artificial pseudo-malicious data sets using a categorical generative adversarial network (CatGAN), and retrains the model with these data to harden it against malicious inputs, ensuring it can withstand attacks without additional filters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If defense techniques like data encryption, data sanitization, or robust statistics are applied, then the model is protected against malicious training data, but the system complexity increases and public data cannot be used for training
Solution Approach 1:
The patent applies preliminary action by performing an initial training phase with controlled training data before the main training phase. This initial training establishes a baseline model that is then used to identify high-impact samples. By preparing the model in advance with clean data, the system creates a reference point that enables subsequent detection of malicious influences without requiring complex external filtering mechanisms during the main training phase.
Solution Approach 2:
The patent converts the harmful effect of high-impact samples into a beneficial detection mechanism. Instead of trying to prevent all potentially harmful data from entering training, the system deliberately uses controlled training to create a baseline, then measures deviations from this baseline. The high-impact samples that would normally be considered harmful are actually used as indicators to identify and exclude malicious data, transforming the problem of sample influence into a solution for detecting poisoning attacks.
2Measurement precision
If the model is trained with a larger training data set including public data, then the model performance and accuracy improve, but the model becomes more vulnerable to training data attacks
Solution Approach 1:
The patent introduces an intermediary mechanism in the form of the initial trained model that mediates between the controlled training data and the larger public training data set. This baseline model acts as a reference point that enables the system to process and evaluate public data without directly exposing the final model to unverified malicious samples. The intermediary allows the system to leverage public data for improved accuracy while maintaining protection through comparative analysis.
Solution Approach 2:
The system implements feedback by using the initial trained model to evaluate samples from the larger training data set and identify high-impact samples. This feedback loop allows the system to detect potentially malicious data by measuring its influence on the baseline model, then adjust the training process accordingly. The feedback mechanism enables continuous monitoring and adjustment, ensuring that public data improves model performance without introducing malicious influences.
3Reliability
If attackers insert maliciously crafted samples into the training data, then the error rate of the ML model increases significantly, but detecting these changes is difficult for humans or analytic systems
Solution Approach 1:
The system performs preliminary training with controlled data to establish a baseline model before exposure to potentially malicious public data. This preliminary action creates a reference state that makes deviations caused by malicious samples detectable. The baseline model serves as an early warning system that can identify when public data begins to corrupt the training process, enabling detection before significant damage occurs.
Solution Approach 2:
The patent replaces manual or analytic detection methods with an automated computational mechanism. Instead of relying on humans or complex analytic systems to detect malicious samples, the system uses the baseline model to automatically measure the impact of each training sample. This mechanical substitution transforms the detection problem into a quantifiable measurement task that can be performed efficiently and objectively by the trained model itself.
Data Source
AI summary
A computer-implemented method or protecting a machine-learning model against training data attacks is disclosed. The method comprises performing an initial training of a machine-learning system with controlled training data, thereby building a trained initial machine-learning model and identifying high-impact training data from a larger training data set than in the controlled training data, wherein the identified individual training data have an impact on a training cycle of the training of machine-learning model, wherein the impact is larger than a predefined impact threshold value. The method also comprises building an artificial pseudo-malicious training data set from the identified high-impact training data and retraining the machine-learning system comprising the trained initial machine-learning model using the artificial pseudo-malicious training data set.