Generative AI Attribution Model Using Training-Stage Change Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data attribution methods for generative AI models, particularly diffusion models, suffer from inaccuracies and inefficiencies, especially in fine-tuned or customized scenarios, as they rely on post-training analysis and fail to leverage valuable training stage information.
Innovation Solution
A method that monitors and aggregates changes in internal representations of generative AI models during the training process, particularly in fine-tuning, to create an attribution table, and trains a separate attribution model using this data for improved accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If post-training analysis methods are used for data attribution in generative AI models, then implementation simplicity is maintained, but attribution accuracy deteriorates
Solution Approach 1:
The patent applies preliminary action by collecting attribution information during the training process itself, rather than performing analysis after training is complete. The system monitors and records how training samples influence model parameters as training progresses, storing this information for later attribution queries. This approach captures accurate attribution signals at the source while keeping the post-deployment system simple.
2Reliability
If training dataset is restricted to legally safe subsets, then copyright compliance is improved, but model performance deteriorates
Solution Approach 1:
The patent introduces an intermediary attribution system that acts as a mediator between the training data and generated outputs. This system tracks and attributes the influence of specific training samples on generated content, enabling legal compliance verification without restricting the training dataset. The attribution mechanism allows users to verify copyright compliance while maintaining access to comprehensive training data for optimal model performance.
3Measurement precision
If classical loss gradient methods are applied to generative models, then method simplicity is maintained, but attribution accuracy deteriorates
Solution Approach 1:
The patent replaces the mechanical gradient-based attribution approach with a monitoring-based system that directly observes parameter changes during training. Instead of computing loss gradients after training, the system records actual parameter updates caused by each training sample throughout the training process. This substitution captures the true influence of training data on model parameters, providing more accurate attribution for generative models.
Data Source
AI summary
A computer-implemented method of training a machine learning attribution model configured to provide data attribution to an output generation of a generative artificial intelligence (AI) model, comprising: determining changes in the generative AI model during a training process; aggregating the changes into an attribution table; and training the attribution model comprising inputting data from the attribution table into the attribution model.