Machine Learning Model Parameter Adjustment for Data Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning techniques lack an effective method to remove or mitigate the training effects of undesired input training data samples once a model has been trained, leading to potential inaccuracies or legal and regulatory issues.
Innovation Solution
A method to adjust a trained machine learning model by creating an Expunged Model that reduces or eliminates the influence of specific subsets of training data, achieved by computing and applying element-wise differences in the model's matrices based on undesired training data batches, resulting in an Updated Model with reduced impact from undesired data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a machine learning model is trained on a large dataset including undesired data samples, then the model's training completeness is improved, but the model's compliance with legal and regulatory requirements deteriorates
Solution Approach 1:
The patent extracts and removes the influence of specific undesired training data samples from an already-trained model. This is achieved by identifying the subset of training data to be removed and applying mathematical operations to eliminate their contribution to the model's learned parameters, thereby achieving compliance without retraining the entire model.
Solution Approach 2:
The patent changes the model parameters (weights and biases) through mathematical operations that subtract the influence of undesired data. By computing gradient updates based on the undesired data subset and applying counter-gradients, the model parameters are adjusted to reduce the impact of specific training samples while maintaining overall model functionality.
2Object-affected harmful factors
If a machine learning model is retrained from scratch to remove undesired data, then the model's compliance is improved, but the cost and time required deteriorates
Solution Approach 1:
The patent performs preliminary adjustments to the trained model by computing and applying parameter changes that remove undesired data influence. This preliminary action modifies the existing model parameters before deployment, achieving compliance without the need for complete retraining, thus saving significant time and computational resources.
Solution Approach 2:
The patent discards the harmful influence of undesired training data while recovering and preserving the useful learned patterns from the rest of the training set. By selectively removing only the unwanted data's contribution through mathematical operations, the model retains its overall knowledge while eliminating specific compliance issues.
3Object-affected harmful factors
If a machine learning model is retrained from scratch to remove undesired data, then the model's compliance is improved, but the computational cost deteriorates
Solution Approach 1:
The patent extracts only the specific parameter adjustments needed to remove undesired data influence, rather than performing a complete retraining process. This extraction approach computes the necessary weight and bias changes based on the undesired data subset, significantly reducing computational cost and energy consumption compared to full model retraining.
Solution Approach 2:
The patent efficiently changes model parameters through targeted mathematical operations that directly adjust weights and biases to eliminate undesired data influence. This parameter adjustment approach requires minimal computational resources compared to iterative retraining processes, making it cost-effective for achieving compliance.
4Measurement precision
If the influence of undesired training data is completely removed from a trained model, then the model's accuracy on desired data may be improved, but the model's overall performance deteriorates
Solution Approach 1:
The patent applies local quality changes by selectively adjusting model parameters only in the regions affected by undesired training data. Rather than globally modifying the entire model, the method targets specific weight and bias parameters that were influenced by the unwanted data subset, preserving the overall model performance while improving accuracy on desired data.
Solution Approach 2:
The patent carefully controls parameter changes to maintain model stability. By computing gradient updates based on the undesired data and applying counter-gradients with appropriate learning rates, the method adjusts parameters to remove harmful influences while preserving the model's overall performance characteristics and generalization ability.
Data Source
AI summary
Many factors contribute to the final predictive accuracy of a trained machine learning model, but the model's predictive behavior depends unavoidably on each of the samples used to train it. After a model is trained, some of the training samples may be found later to be undesired. Undesired samples could represent personal information of a consumer who has asked to be “forgotten”, incorrectly measured or low-quality input data, irrelevant data used to train a base model later used in transfer learning, or samples found to be unwanted for any other reason. Heretofore, the only technique available to remove the effect of undesired samples was to retrain the model with those samples removed from the training set. This invention provides a method to remove the effect of undesired samples after the model is trained, without the need to retrain the entire model from scratch.