Generative AI Attribution Using Fine-Tuning Change Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data attribution methods for generative AI models, particularly diffusion models, suffer from inaccuracies and inefficiencies, especially when dealing with fine-tuned or customized models, and are not suitable for mixed-concept generations, leading to legal and ethical challenges related to copyright infringement.
Innovation Solution
A method that leverages the training stage of generative AI models by monitoring internal representations during fine-tuning and aggregating changes into an attribution table, followed by training a separate attribution model using a specialized loss function to provide nuanced insights into the model's behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If post-training analysis methods are used for data attribution, then the method is simpler to implement, but the accuracy of attribution is insufficient
Solution Approach 1:
The patent applies preliminary action by performing data attribution analysis during the training process itself, rather than after training is complete. The system monitors and records the influence of training data on model parameters in real-time during training, creating an attribution table that captures the actual contribution of each training sample. This preliminary measurement during training enables accurate post-training queries about data influence without requiring complex re-analysis.
2Productivity
If loss-derivative based approaches are used, then the method is more efficient, but it cannot accurately handle mixed-concept generations and fine-tuned models
Solution Approach 1:
The system performs preliminary monitoring and recording of training data influence during the training process, creating a comprehensive attribution table that captures actual data contributions. This preliminary action stores detailed information about which training samples influenced which model parameters and outputs, enabling accurate attribution queries later without requiring complex re-computation.
Solution Approach 2:
The patent creates a copy of the training process information in the form of an attribution table, which stores the influence relationships between training data and model outputs. This copied information can then be queried efficiently during inference to determine data attribution for specific generated outputs, avoiding the need to re-analyze the entire training process.
3Object-affected harmful factors
If training data is restricted to legally safe subsets, then copyright concerns are avoided, but model performance deteriorates
Solution Approach 1:
The system provides feedback about the actual influence of training data on model outputs through the attribution table. This feedback mechanism allows users to understand which training samples contributed to specific generated images, enabling informed decisions about data usage, copyright compliance, and model behavior without restricting the training data to safe subsets.
4Productivity
If data attribution is performed after training, then the model is ready for use, but valuable training process information is lost
Solution Approach 1:
The system performs preliminary monitoring and recording of training data influence during the training process itself. The attribution table is built incrementally as training progresses, capturing the influence relationships between training samples and model parameters at various stages. This preliminary action preserves valuable training process information that would otherwise be lost, while the model remains ready for use.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method of training a machine learning attribution model configured to provide data attribution to an output generation of a generative artificial intelligence (AI) model, comprising: determining changes in the generative AI model during a training process; aggregating the changes into an attribution table; and training the attribution model comprising inputting data from the attribution table into the attribution model.