Generative AI Manifold Alteration for Selective Sample Unlearning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative AI models often produce incorrect outputs due to errors during training, such as pairing incorrect data, leading to undesired responses, and existing solutions like fine-tuning are inadequate for correcting these errors post-deployment.
Innovation Solution
A method and system that adjusts the loss function to increase the loss for identified undesirable samples, altering the manifold to unlearn these samples, thereby modifying the generative AI model's behavior without retraining the entire model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the model is retrained entirely to correct errors, then the reliability of generated content is improved, but the time and computational resources required increase significantly
Solution Approach 1:
The patent segments the training process by identifying and isolating specific undesirable samples from the training dataset. Instead of retraining on the entire dataset, the method selectively removes or reweights particular samples that contribute to errors, thereby correcting reliability issues while minimizing the scope of retraining required
Solution Approach 2:
The patent modifies training parameters by adjusting the loss function weights for specific samples. By changing the parameter weights associated with undesirable samples rather than retraining with uniform parameters, the method improves content reliability while avoiding the time cost of complete retraining
2Adaptability or versatility
If fine-tuning is applied to correct post-deployment errors, then the adaptability of the model is improved, but the precision of correction is insufficient
Solution Approach 1:
The patent applies local quality by targeting specific regions of the training data (undesirable samples) with corrective actions. The method identifies particular samples that cause errors and applies localized adjustments to their loss weights, achieving precise correction rather than broad fine-tuning that lacks precision
3Manufacturing precision
If the entire model is retrained to avoid undesirable outputs, then the fidelity of generated samples is improved, but the computational resources and time required increase significantly
Solution Approach 1:
The patent extracts and removes undesirable samples from the training set by identifying them through evaluation metrics. By taking out only the problematic samples rather than retraining on the complete dataset, the method achieves high fidelity in generated samples while reducing computational resource requirements
Data Source
AI summary
The present disclosure relates to systems and methods for modifying an output of a generative artificial intelligence (AI) model. The systems and methods control the loss function for samples under consideration by adjusting a value of the loss function for the samples and decreasing a probability that the generative AI model uses the samples in generating responses.