Machine Unlearning via Weight Segmentation and Influence Functions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine unlearning algorithms face challenges in achieving computational efficiency, rigor, and interpretability, making it difficult to retroactively remove sensitive information from deployed models without requiring full retraining or access to the original dataset.
Innovation Solution
The proposed method involves receiving retain and forget samples, generating retain and forget outputs from a pretrained model, computing influence function-based weights, freezing retain weights, resetting forget weights, and retraining forget weights using retain samples to form an unlearned model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full retraining is performed to remove sensitive information, then unlearning rigor is improved, but computational efficiency deteriorates
Solution Approach 1:
The patent segments the model weights into retain weights and forget weights based on their contribution to sensitive information. By identifying and separately handling forget weights through influence function analysis, the method enables targeted unlearning without requiring full model retraining, thus resolving the contradiction between unlearning rigor and computational efficiency
Solution Approach 2:
The patent extracts and identifies specific weights that contribute to sensitive information using influence function computations. By isolating these forget weights from the overall model parameters, the method allows selective unlearning of only the necessary components, achieving rigorous unlearning while maintaining computational efficiency
2Productivity
If existing machine unlearning algorithms are used, then computational efficiency is improved, but unlearning rigor deteriorates
Solution Approach 1:
The patent employs influence function analysis to compute how individual training samples affect model weights. This feedback mechanism identifies which weights are most influenced by sensitive data, enabling rigorous unlearning decisions while maintaining computational efficiency through targeted rather than exhaustive processing
3Ease of operation
If retain weights are frozen and forget weights are retrained, then unlearning interpretability is improved, but model accuracy may deteriorate
Solution Approach 1:
The patent changes the state of forget weights from their trained values to an initial state (e.g., zeros or random initialization), while keeping retain weights frozen. This parameter change approach provides clear interpretability of the unlearning process and allows controlled adjustment to maintain model accuracy through selective retraining of only the affected weights
Data Source
AI summary
A method may include: receiving a set of retain samples comprising retain features to retain in a pretrained machine learning model, and a set of forget samples comprising forget features to remove from the pretrained machine learning model; providing the set of retain samples to the pretrained machine learning model resulting in a retain output and the set of forget samples to the pretrained machine learning model, resulting in a forget output; generating a set of retain weights and a set of forget weights based on the retain output and the forget output; freezing the set of retain weights; setting each forget weight to an initial state; executing a training epoch using the pretrained machine learning model and the retain samples that retrains the forget weights using the retain samples; combining the retrained forget weights with the retained weights to form an unlearned machine learning model.