Systems and methods for attribution in machine learning

Abstract

This provides a system and method for attribution in machine learning. [Solution] A computer-implemented method for training a machine learning attribution model configured to provide data attribution for output generation of a generative artificial intelligence (AI) model, comprising: a step of determining changes in the generative AI model during the training process; a step of aggregating the changes into an attribution table; and a step of training the attribution model, which includes inputting data from the attribution table into the attribution model.

Description

[Technical Field]

[0001] The present invention relates to a system and method for providing data attribution in generative artificial intelligence models, and more particularly to providing data attribution in diffusion models. [Background technology]

[0002] As generative artificial intelligence (AI) technology advances, the need to understand and control its output becomes increasingly crucial. However, because generative AI technology is complex and inherently opaque, understanding how content is generated remains a critical challenge.

[0003] Generative AI models face challenges related to transparency and intellectual property, for example. One example is the challenge that arises when generated images are affected by copyrighted images from the training data, a plausible scenario with data collected from the internet. In fact, typically, each image generated by these models is affected by a subset of the training data that may contain copyrighted content. This raises legal questions about who owns these newly generated images and potentially to what extent. Thus, understanding how fragments of training data contribute to the model's output (a task known as data attribution) is at the heart of these technical and legal challenges and is crucial for transparency of content origin, legal compliance, and ethical use.

[0004] Generally, data attribution involves identifying influential training data that affect and contribute to the predictions of a trained model. In the context of generative models, it involves mapping the generated output to the training examples that facilitate its creation, which is a crucial step in understanding model behavior. Beyond generative AI, data attribution in the context of deep learning has a well-established history. This typically requires an ad-hoc analysis of the trained model, i.e., performed without access to the training process. Classical approaches employ loss gradients and Hessian algorithms to quantify how each training sample affects the dynamics of the pre-trained weights in their local environment and, consequently, the model's output.

[0005] Therefore, data attribution is important, for example, to provide explainability of the relationship from training data to output, as well as insights into and improvements to the training data. For example, data attribution is important for interpretability and debugging, for example, in understanding the impact of training data on model output, which is key to correcting bias and errors. Another example is that data attribution can be used to improve the robustness of models and to detect and avoid poisoning attacks, for example, by detecting training samples that impair performance. Another example is that data attribution can be used for improved data curation and quality, for example, to help curate high-quality datasets and ensure that models are trained on relevant and diverse data, which affects the quality of model output and can increase efficiency by omitting unused data.

[0006] When addressing concerns about data attribution in the field of generative AI, there are broadly two approaches. The first approach attempts to completely avoid concerns about data attribution by controlling the dataset used for training such that the training dataset is substantially restricted to a subset of data that is considered known and legally safe to use. As a result, any generated image can be associated with any training sample without legal concerns such as copyright-related concerns. This approach comes with a significant cost in that it is inefficient, expensive, and restrictive. Curating a dataset is expensive, and breaking it down into smaller manageable datasets degrades the performance of the model. Furthermore, it does not work for models where the training dataset cannot be controlled. For example, in the case of customized or fine-tuned models where the user controls and selects their own dataset to customize and train the base model. In these examples, the user can freely select copyrighted material. The second approach is to provide data attribution techniques that attempt to determine the degree of attribution of training images to the generated images. These approaches are inaccurate and tend to focus on post-training analysis of the model as described above. Examples of known processes include loss-derivative based approaches that follow the classical theory of data attribution to obtain attribution in the same way as methods originally designed for discriminative models, such as image classifiers, i.e., non-generative AI. Summary of the Invention Problems to be Solved by the Invention

[0007] Therefore, improved data attribution for generative AI models is needed. For example, there is a need for a data attribution method that can improve the accuracy and efficiency in determining the influence and / or contribution of input training images to the generated output.

[0008] Certain aspects of the present disclosure and their embodiments can provide solutions to these or other problems.

Means for Solving the Problems

[0009] Aspects of the present invention are defined by the appended claims. Optional advantageous features are defined in the dependent claims.

[0010] According to one aspect, a computer-implemented method for training a machine learning attribution model configured to provide data attribution for output generation of a generative artificial intelligence (AI) model, the method comprising: determining changes in the generative AI model during a training process; aggregating those changes into an attribution table; and training an attribution model including inputting data from the attribution table into the attribution model.

[0011] Various aspects and embodiments of the present invention will be described below with reference to the drawings, without limitation.

Brief Description of the Drawings

[0012] Hereinafter, preferred embodiments of the present invention will be described in detail by way of example only with reference to the following drawings. [Figure 1] Shows a method. [Figure 2] Shows a process. [Figure 3] Shows an exemplary machine learning model architecture. [Figure 4] Shows a process. [Figure 5] An exemplary data attribution table is shown. [Figure 6] Show the process. [Figure 7] Show the process. [Figure 8] I will show you the method. [Figure 9] This is the results table. [Figure 10] This is the results table. [Figure 11] The device is shown. [Modes for carrying out the invention]

[0013] In the following description, functionally similar parts have the same reference numerals between the drawings. The following describes specific details such as particular aspects, embodiments, or examples for illustrative purposes only, not as limitation. Those skilled in the art will understand that other examples may be employed, separate from these specific details. Hereinafter, aspects and embodiments of the present invention are described, with reference to the accompanying drawings, merely as examples and without limitation.

[0014] Aspects of this application provide an approach to data attribution in the use of generative AI models. Much of the current technology in data attribution concerns only discrimination models (i.e., image classifiers). Among these, specific embodiments of this application concern data attribution in generative AI diffusion models, such as text-to-image diffusion models.

[0015] Text-image diffusion models generate images by mapping noise to images, and generation is often conditional on encoded text input as a prompt (e.g., "Generate an image of an alien on horseback"). The development of diffusion models can be divided into two methodologies: training a base model and fine-tuning it. Training a base model involves compiling a wide range of datasets from various sources. Their large scale leads to control issues regarding copyrighted content, as seen, for example, in the LAION and IMAGEN datasets. In contrast, fine-tuning a diffusion model used for model customization involves smaller, more specific datasets, as well as customizing a pre-trained base model (i.e., foundational model) for new capabilities using efficient fine-tuning (i.e., customization) methodologies. This provides a pathway for adapting diffusion models in low-resource settings. As a result, customization has become a popular tool among both corporate and private creators, increasing the risk of copyright infringement by unintentional creators. Thus, recent developments in this field have extended the data attribution domain to analyze diffusion models.

[0016] However, current technology approaches to data attribution in generative AI models have many shortcomings. For example, current technology approaches focus on base model scenarios that do not involve direct access to the training process. Avoiding reliance on training access in this way is a practical approach for base models due to the expensive, resource-intensive, and time-consuming impact of attempting to analyze training. Therefore, these current technology approaches focus on post-training analysis of base models. However, this approach leads to inaccuracies.

[0017] In particular, current technological approaches can be broadly divided into two categories. First, there are loss derivative-based approaches. These follow the classical theory of data attribution and acquire attribution in a manner similar to methods originally designed for discrimination models. Examples include DTRAK (Non-Patent Literature 1) and DataInf (Non-Patent Literature 2). Second, there are generative analysis approaches that branch off from classical solutions and directly analyze the generation of generative models. An example is GenDataAttribution (Non-Patent Literature 3). However, both of these approaches have disadvantages. For example, since both of these approaches are performed on the final trained model after training, the accuracy of data attribution is reduced, and as a result, valuable information for data attribution embodied during the training phase cannot be utilized, or even recognized. Furthermore, these approaches are incompatible with fine-tuned (i.e., customized) models and do not adequately address mixed-concept generation enabled by diffusion models. In mixed concept generation, different concepts may originate from different domains or represent different styles, objects, or themes. The diffusion model generates new data by blending features from these diverse concepts in a consistent manner to produce a generated image. [Non-Patent Document 1] Zheng, X., Pang, T., Du, C., Jiang, J., Lin, M.: Intriguing properties of data attribution on diffusion models. In: The Twelfth International Conference on Learning Representations (2024), https: / / openreview.net / forum?id=vKViCoKGcB [Non-Patent Document 2] Kwon, Y., Wu, E., Wu, K., Zou, J.: Datainf: Efficiently estimating data influence in loRA-tuned LLMs and diffusion models. In: The Twelfth International Conference on Learning Representations (2024), https: / / openreview.net / forum?id= 9m02ib92Wz [Non-Patent Document 3] Wang, SY, Efros, AA, Zhu, JY, Zhang, R.: Evaluating data attribution for text-to-image models. In Proceedings of the IEEE / CVF International Conference on Computer Vision, pp. 7192-7203. 2023

[0018] For example, DTRAK uses both the attribution and evaluation methods proposed in TRAK (Non-Patent Literature 4), but for diffusion models. TRAK is a loss gradient-based approach originally designed for discrimination models. As part of this process, TRAK proposed an evaluation metric that relates attribution to the leave-out retraining loss. DataInf proposes an inverse loss Hessian approximation for diffusion models. While the true Hessian is summed over the training samples, DataInf performs the inverse of each addendum before the summation, enabling rank-aware algorithms. Here, the evaluation is with respect to the true Hessian, and the theoretical limits of the approximation are derived. [Non-Patent Document 4] Park, SM, Georgiev, K., Ilyas, A., Leclerc, G., & Madry, A. (2023, July). TRAK: Attributing Model Behavior at Scale. In International Conference on Machine Learning (pp. 27074-27113). PMLR

[0019] In GenDataAttribution, the authors use thousands of single-image customized models to create a dataset of generated images, ensuring that a single known training image influences the output. These generated images provide the ground truth data that GenDataAttribution leverages for contrastive learning in the attribution embedding space. Specifically, this method consists of three main steps. First, it generates pairs of real images (references) and their composite corresponding image sets, the composite images being acquired using thousands of stable diffusion models, each generating a single known attribution image, its "reference" image. Second, it trains the models using a contrastive learning approach to attribute each composite set of images to its reference. Third, it obtains soft stochastic influences from the learned feature similarities. However, this method has several drawbacks. First, the number of trained models required to generate the data attribution output is in the thousands. Second, this method cannot provide data attribution for image generation created from mixed concepts.

[0020] Therefore, each of these current technological approaches has clear drawbacks. Each attempts to perform data attribution for image generation models by performing post-training analysis, which results in the loss of valuable information from the training process. In particular, they each calculate image-level data attribution via loss differences, or conceptual-level attribution by analyzing the generated images. These each focus on basic models and cannot handle mixed-concept image generation, and are therefore not relevant to or suitable for fine-tuned customized models. Furthermore, they are not suitable for online learning because attribution needs to be completely recalculated with each model update, thus failing to support continual learning.

[0021] The aspects of this invention have advantageously identified and facilitated the use of the training phase of generative AI models to gain insights into data attribution.

[0022] Aspects of this application have identified, advantageously, that for fine-tuning scenarios, access to the training phase becomes practically feasible due to the fewer resources required, and that leveraging such access can favorably improve the accuracy of data attribution because the training phase holds valuable information that can be advantageously utilized, for example, by allowing decisive insights into how training images shape the generated output. Thus, aspects of this application relate to data attribution in fine-tuning (i.e., customized) diffusion models.

[0023] The aspects of this invention can advantageously provide improved data attribution granularity. For example, in certain embodiments, specialized novel loss functions are used, which advantageously provide subtle differences in the model training process.

[0024] An advantage of this invention is that it can provide data attribution in generative AI models, such as finely tuned and customized diffusion models, with improved accuracy, especially when the attribution correlates with the model's behavior, and furthermore, with improved computational efficiency.

[0025] Aspects and embodiments of the present application advantageously leverage the accessibility of the training process, e.g., the fine-tuning (customization) process, in generative AI models such as diffusion models for improved data attribution. In particular, aspects of the present application can be applied in two broad steps. First, data attribution values ​​are collected through the fine-tuning / customization process and aggregated into an attribution table. In particular, in certain embodiments, the internal (latent) representation of the generative model (e.g., diffusion model) is monitored during the fine-tuning phase. These changes are efficiently monitored and quantified, and attribution is calculated from the quantification of these changes. Second, a separate attribution model is trained on the attribution table data so that it can learn the collected information about how the generative (e.g., diffusion) model uses the training data when generating output, and generalize this to future generations of the model. In particular, in certain embodiments, the training of the attribution model may be via a specialized loss function that advantageously captures finer granularity of attribution, thereby improving accuracy.

[0026] For example, certain aspects of the present application may offer the first integration of two methodologies: exploring training access for data attribution and leveraging generative (e.g., spread) model properties. These aspects provide monitoring the internal representation of a generative (e.g., spread) model for changes during training, aggregating this information for data attribution, and thereby making new contributions to the field. Current technical methodologies cannot explore the use of training access for generative (e.g., spread) model data attribution. Aspects of the present application provide a generative analytics approach in which generation is monitored throughout training.

[0027] Aspects of the present invention are defined by the appended claims. Advantageous optional features are defined in the dependent claims.

[0028] In one respect, a computer-implemented method is provided for training a machine learning attribution model configured to provide data attribution for output generation of a generative artificial intelligence (AI) model, the method comprising: a step of determining changes in the generative AI model during the training process; a step of aggregating those changes into an attribution table; and a step of training the attribution model, which includes inputting data from the attribution table into the attribution model.

[0029] Optionally, the training process is a fine-tuning process.

[0030] Optionally, discerning a change includes discerning changes in the internal representation within the generative AI model while the training data is being input into and processed by the generative AI model during the training process.

[0031] Optionally, determining a change in the internal representation of a generative AI model during the training process includes, while executing the training process, inputting a prompt concept configured to cause the generative AI model to generate an output, and determining the change in the internal representation of the prompt concept.

[0032] Optionally, the generative AI model is a diffusion model. Optionally, the diffusion model is an image-text diffusion model.

[0033] Optionally, the training process includes inputting fine-tuning data into a diffusion model as training data, the fine-tuning data comprising image-concept pairs, each image-concept pair comprising a fine-tuning image and an associated concept comprising a text description relating to the visual content of the image.

[0034] Optionally, determining changes in the internal representation of the diffusion model during the training process includes inputting a prompt concept configured to cause the diffusion model to generate an output image, while simultaneously running the training process, and determining changes in the internal representation of the prompt concept.

[0035] Optionally, the internal representation includes a vector representation of the prompt concept in the cross-attention layer of the diffusion model.

[0036] Optionally, the internal representation includes the value tensor of the cross-attention layer.

[0037] Optionally, the data attribution table includes a data structure that associates each output-generated image generated by a prompt concept with an attribution score that provides a numerical quantification of the contribution of each tuned image to the output-generated image. The attribution score is based on the determined change in the internal representation of the prompt concept.

[0038] Optionally, rows in the data attribution table relate to the fine-tuning image, and columns in the data attribution table relate to the output generated image.

[0039] Optionally, the data attribution table is such that the fine-tuning images are ordered and grouped by concepts taken from associated concepts of a particular image-concept pair, and the output generated images are ordered and grouped by prompt concepts.

[0040] Optionally, training an attribution model includes the steps of: inputting image pairs from a data attribution table into the data attribution model, wherein the image pairs include a fine-tuning image and an output generated image; creating a fine-tuning image embedding for the fine-tuning image in the image embedding space for each image pair; creating an output generated image embedding for the output generated image in the image embedding space; performing a comparison between the fine-tuning image embedding and the output generated image embedding; and determining a predictive attribution score that provides a predicted numerical quantification of the fine-tuning image's contribution to the output generated image based on the comparison.

[0041] Optionally, training may further include the steps of: comparing predicted attribution scores with attribution scores from a data attribution table associated with an image pair; and adjusting the network weights of the attribution model based on the comparison.

[0042] Optionally, further comprising training an attribution model to distinguish conceptually similar pairs and conceptually different pairs of image pairs, for a pair of image pairs, determining a first predicted attribution score for the first image pair, the first image pair being a positive image pair including conceptually similar fine-tuned images and output generated images; and determining a second predicted attribution score for the second image pair, the second image pair being a negative image pair including conceptually different fine-tuned images and output generated images.

[0043] Optionally, for all pairs of image pairs in the data attribution table: further comprising adjusting the network weights of the attribution model based on minimizing a loss function. The loss function is

Number

[0044] Optionally, the attribution model may include a Siamese network.

[0045] Proximity in the image embedding space can be arbitrarily represented by conceptual similarity.

[0046] Optionally, the predicted attribution is determined based on the shifted cosine similarity between the fine-tuned image embedding and the output-generated image embedding in the image embedding space.

[0047] In one aspect, a computer-implemented method is provided for performing data attribution using an attribution model trained in any manner described herein, the method comprising: selecting a generated output generated by a generative AI model described in any part of this specification; inputting the generated output into a data attribution model; and outputting from the data attribution model a data attribution score relating to at least one training input on which the generative AI model was trained, wherein the data attribution score provides a numerical quantification of the contribution of the at least one training input to the generated output.

[0048] In one respect, a computer program is provided that, when executed on a computer, causes the computer to perform the method in any manner described herein.

[0049] Figure 1 shows a training process from one perspective. In particular, the training process may be a computer-implemented method of training a machine learning attribution model configured to provide data attribution for output generation of a generative AI model.

[0050] Step S11 includes determining changes in the generated AI model during the training process.

[0051] Step S13 involves aggregating those changes into an attribution table.

[0052] Step S15 includes training the attribution model, which involves inputting data from the attribution table into the attribution model.

[0053] Advantageously, we determined that the aspects of this application can provide critically important insights into how the training data and training process of a particular generative AI model shape the output it generates, and thus can be utilized to provide accurate insights into the data attribution of the final output generation of a generative AI model.

[0054] Any of the steps in Figure 1 can be performed by an apparatus as described with reference to Figure 11 below.

[0055] In particular, in certain embodiments of step S11, determining change involves monitoring changes in the internal representation of the generative AI model (i.e., vectors, embeddings, and / or encodings, etc.) while training data is input and processed by the generative AI model during training. For example, depending on the particular generative AI model in question, the specific weights and biases of the model being monitored may differ, but the principle of monitoring the internal representation during training remains the same. In other words, changes during training are quantified through the internal representation of the generative AI model being monitored and the changes therein. In further specific embodiments, determining change in the internal representation of the generative AI model during the training process involves inputting a prompt concept configured to cause the generative AI model to generate an output generation into the generative AI model while simultaneously running the training process, and determining changes in the internal representation of the prompt concept.

[0056] In certain embodiments, the training process may be a fine-tuning process, the generative AI model being the base model, and the fine-tuning process involves inputting customized training data, which may be private data, public data related to a specific fine-tuning domain or purpose, or any combination thereof. Advantageously, compared to a labor-intensive base model training phase, monitoring the training during the fine-tuning phase requires fewer resources and can be executed more efficiently, and advantageously, access to the training phase becomes practically feasible. Furthermore, given that the training phase holds valuable information that can be advantageously utilized, such as allowing critically important insights into how training images shape the generated output, leveraging such access can advantageously improve the accuracy of data attribution.

[0057] In certain embodiments, the generative AI model may be a specific type of model. For example, the generative AI model may be a large-scale language model, a generative adversarial network, a neural radiance field, a variational autoencoder, an autoregressive model, a recurrent neural network, a transformer-based model, or any other suitable model. In certain embodiments, the generative AI model may be a diffusion model, and in certain embodiments, an image-text diffusion model.

[0058] Figure 2 is a diagram illustrating a process according to one embodiment, which is a specific embodiment of the process shown in Figure 1. In particular, in the specific embodiment shown in Figure 1, the training process is a fine-tuning process, the generative AI model is a diffusion model, and the diffusion model is a text-to-image (text-to-image) model.

[0059] Therefore, according to a particular embodiment of step S11, changes in the diffusion model 3 during the fine-tuning process can be determined. For example, the determination of changes may be performed by the attribution monitor 1, which may monitor the diffusion model 3 during the fine-tuning process 5. The fine-tuning process 5 may be a sequential iterative fine-tuning process. If the fine-tuning process 5 is sequential iterative, the attribution monitor 1 may perform monitoring of changes throughout the sequential iterative fine-tuning process 5. The diffusion model 3 may be a basic model (i.e., a foundational model), or it may be a diffusion model 3 that has already been fine-tuned but is now undergoing further, additional, or different fine-tuning processes. Fine-tuning (i.e., customized) training data is input into the diffusion model 3 in order to perform the fine-tuning process 5. The diffusion model 3 may be any preferred or appropriate diffusion model, in particular a text-image diffusion model. The fine-tuning data 7 may be from any preferred or appropriate source, for example, publicly available, private, or a combination thereof. If the diffusion model 3 is a text-image diffusion model, the fine-tuning data 7 may include image-concept pairs, each image-concept pair including an image 73 and at least one associated concept 71, the concept including text that provides a description of the visual content of the associated image 73 and / or related to the associated visual content. Thus, the fine-tuning process 5 may include inputting the fine-tuning data 7 as training data into the diffusion model 3, the fine-tuning data 7 including image-concept pairs 71, 73, each image-concept pair 71, 73 including an image 73 and an associated concept 71 which includes a text description related to the visual content of the image.

[0060] When the fine-tuning data 7 is input to the diffusion model 3, the diffusion model 3 processes the fine-tuning data, and in the process, the fine-tuning data 7 changes the diffusion model 3. For example, the weights and biases in the diffusion model 3 may change and be determined, and the internal latent representation (vector representation, etc.) of the fine-tuning data 7 may change and be determined throughout the training. These changes may be monitored by the attribution monitor 1. Typically, during training, no diffusion model output generation is produced as is generated during the inference phase. Instead, the focus during training is on learning to denoise images at various stages of the diffusion process given text prompts 71. Aspects of the present invention may favorably discriminate internal changes in the diffusion model 3, for example, changes in the internal representation generated within the diffusion model 3 during fine-tuning 5, and information representing these changes may be used to assist and improve data attribution in the final use of the trained diffusion model 3 on the prompt-generated output images.

[0061] As will be further described below with reference to Figures 3 and 4, in certain embodiments of the present application, a set of prompt concepts 8 is determined and used to input prompt concepts 8 into a diffusion model 3 to cause the diffusion model 3 to generate output images. In particular, these prompt concepts 8 are input during the fine-tuning process 5 and processed by the diffusion model 3 at the same time that the diffusion model 5 is trained on the fine-tuning data 7 and learns it. Thus, these prompt concepts 8 pass through the diffusion model 3 while the diffusion model 3 is changing based on the fine-tuning data 7. These prompt concepts 8 may be the same as, similar to, or different from the concepts of the fine-tuning dataset 7. Thus, in certain embodiments, determining changes in the internal representation of the generative AI model during the training process includes inputting prompt concepts into a generative AI model configured to cause the generative AI model to generate output generation at the same time as the training process is performed; and determining changes in the internal representation of the prompt concepts. In a further specific embodiment, determining changes in the internal representation of the diffusion model 3 during the training process 5 includes inputting a prompt concept 8 to the diffusion model 3, which is configured to generate an output generated image 93, while simultaneously performing the training process 5, and monitoring changes in the internal representation of the prompt concept.

[0062] According to step S13, these changes may be aggregated in attribution table 9. For example, the changes determined according to step S11 may be processed in some way and quantified numerically, and this information may be collected and aggregated in attribution table 9.

[0063] In particular, the attribution table 9 is an abstract data structure created based on step S11, which identifies changes in the model during training, and the depiction in Figure 2 is a visual representation of the data structure to facilitate visual understanding.

[0064] The attribution table 9 may be configured to associate output-generated images 93, i.e., images generated by the diffusion model 3 as a result of a particular prompt concept 8, with images 73 from the fine-tuning data 7. In particular, as will be further described below with reference to Figures 3 and 4, a numerical quantification of the influence or contribution of each particular fine-tuning image 73 to each particular output-generated image 93 may be determined, and this information may be stored in the attribution table 9 as an attribution score 95. For example, rows in the attribution table 9 may be associated with each of the fine-tuning images 73, columns in the table may be associated with each output-generated image 93 generated in response to a particular prompt concept 8, and cells in the table may be associated with an attribution score that references the fine-tuning image 73 and output-generated image 93 of that particular row and column. Therefore, in a particular embodiment, the data attribution table 9 includes a data structure that associates each output generated image 93 produced by the prompt concept 8 with a data attribution score 95 that provides a numerical quantification of the contribution of each image 73 from the fine-tuning data to the output generated image 93, the data attribution score 95 being based on the determined changes in the internal representation of the prompt concept 8.

[0065] In certain embodiments, as will be further described below, the order of the fine-tuning images 73 in the attribution table 9 may be grouped by concepts such as the same or similar concepts, so that fine-tuning images 73 having the same associated concept 71 are adjacent to each other within a block. Similarly, the order of the output generated images 93 may be the same as the order of the fine-tuning images 73, in other words, the order of conceptual grouping of the fine-tuning images 73 based on concept 71 may be the same as the order of conceptual grouping of the output generated images 93 based on the concept of prompt concept 8. For example, if the first group of fine-tuning images 73 (i.e., the first five rows) have the same associated concept, "cat", then the first group of output generated images 93 (i.e., the first five columns) may have the same (or similar) associated concept "cat" from prompt concept 8. Note that the number of fine-tuning images 73 in a concept group does not need to be the same as the number of output generated images 93 in the associated concept group. Therefore, in a particular embodiment, the fine-tuning image 73 is a row in the attribution table 9, for example, each row is associated with a specific fine-tuning image 73, and the output generated image 93 is a column in the attribution table 9, for example, each column is associated with a specific output generated image 93. The data attribution table 9 can associate each output generated image 93 with the prompt concept 8 that caused it.

[0066] Therefore, in a further specific embodiment, the data attribution table 9 is such that the fine-tuning images 73 are ordered and grouped by concepts 71 taken from associated concepts of a particular image-concept pair, and the output-generated images 93 are ordered and grouped by prompt concepts 8. In a particular embodiment, the order of the fine-tuning images 73 by concepts may be the same as or similar to the order of the output-generated images 93, that is, the order or concepts of both the fine-tuning images 73 in rows and the output-generated images 93 in columns may be the same.

[0067] For visual clarity, the attribution table 9 in Figure 2 shows only two fine-tuning images 73 and two output-generated images 93, but the table can take any number of these.

[0068] According to step S15, the attribution model 11 is trained. This involves inputting data from the attribution table 9 into the attribution model 11. This process is described further below. The attribution model 11 may be any suitable machine learning model, such as a neural network. The attribution model 11 will be discussed later.

[0069] Figure 3 relates to a specific embodiment of diffusion model 3 shown in Figure 2, where diffusion model 3 is a text-image diffusion model. Thus, Figure 3 shows a schematic diagram of a general text-image architecture. In the specific embodiment according to step S11, the determination of changes in diffusion model 3 during the fine-tuning process 5 may relate to changes in various aspects of the text-image diffusion model according to Figure 3.

[0070] In particular, the text-image diffusion model 3 includes a text encoder 13, a denoising layer 15 (e.g., a conditional denoising U-Net), and a cross-attention layer 17. For visual clarity, only the relevant parts of the text-image diffusion model 3 are shown, but the text-image diffusion model 3 may include other preferred or appropriate aspects, such as a variational autoencoder. In this particular embodiment, the aspects of the present application have been identified to the advantage that changes in the diffusion model 3 during the fine-tuning training process 5 can be accurately determined by monitoring the internal representation of the diffusion model 3 during the fine-tuning training process 5, in particular by monitoring the activation of the attention layer. In particular, this provides clear advantages in computational efficiency, computation reduction, and process speed by providing insights into the fine-tuning training process while maintaining efficiency by avoiding full generation pipeline overhead. In a particular embodiment, determining changes according to step S11 advantageously focuses on the efficient use of the cross-attention layer 17 representation to provide insights into data attribution.

[0071] In particular, the prompt concepts 8 are sometimes called monitored prompts. This is because these are prompts (and changes in their internal representations) that will be monitored during the fine-tuning process 5 when the fine-tuning image 73 is input to the diffusion model 3. These prompt concepts 8 may be exactly the same as or similar to the concepts 71 that are predetermined and associated with the fine-tuning image 73.

[0072] In particular, regarding the exemplary prompt concept 8 "cat", this is a text embedding C cat The text is input to the text encoder 13 to create the prompt concept 8, which has an associated noisy input image 23, thereby forming a noise-prompt input pair. This text embedding C catThis controls the generation of the output generated image 93 from a noisy input image 23 (i.e., a Gaussian noise input). The cross-attention layer 17 contains a value tensor V, which is a key vector holding information from the text embedding. This value tensor V is combined with an image feature map based on its weights within the cross-attention layer 17, thereby ensuring that the output generated image 93 reflects the content and style described in the text prompt concept 8. In other words, the value tensor V represents information about the input prompt concept 8, and based on this, the generation model 3 adjusts its output generated image 93. Thus, each prompt concept 8 is mapped to a separate value tensor V, 25 by the cross-attention layer.

[0073] Focusing on changes in the value tensor V of the cross-attention layer 17 to discriminate changes in the diffusion model 3 during the fine-tuning process 5 is particularly advantageous for several reasons. First, it contains analytically valuable information and is efficient to monitor. Second, monitoring V 25 does not require full generation, is constant along reverse diffusion, and saves considerable computation time. Third, V is an informative representation of the output generated image 93, which encapsulates the text embedding and connects directly to the text embedding that controls the generation. Fourth, the goal of conceptual customization fine-tuning is to generate diverse outputs that maintain semantic consistency with the fine-tuning customization concept 71, and monitoring V 25 encapsulates this assumption, as the resulting attributions will be consistent for output generated images 93 from the same prompt concept 8. Fifth, V 25 is easily scaled (the inner parts of V 25 have the same scale—it is multiplied by a probability vector and thus has a consistent scale), making it particularly suitable for monitoring, quantifying, and aggregating changes.

[0074] Therefore, in a particular embodiment of step S11, V 25 is monitored to see how it changes for each input prompt concept 8 throughout the entire fine-tuning training process 5, and the changes in V 25 may then be aggregated in an attribution table 9 in a particular embodiment of step S13. Thus, in a particular embodiment, the monitored changes in the internal representation in the diffusion model include monitoring changes in the vector representation of the prompt concept in the cross-attention layer of the diffusion model, where, for example, the vector representation may be a value tensor of the cross-attention layer. This is done concurrently during the fine-tuning process while the diffusion model 3 is being trained and learned on the fine-tuning dataset 7. Such embodiments are particularly advantageous because they allow the fine-tuning process 5 to be run concurrently, passing the prompt concept 8 through the diffusion model 3 to generate an output generated image 93. Furthermore, this concurrent processing is advantageous because it enables parallelization and thus processing efficiency, in particular because the discrimination and monitoring of changes in the diffusion model 3, such as changes in the cross-attention layer 17, does not interfere with the fine-tuning training process 5 of the diffusion model 3. Here, the fine-tuning data 7 passes through the diffusion model 3, and the diffusion model 3 changes accordingly. Therefore, the value tensor V 25 of the cross-attention layer 17 is constantly being modified by each new fine-tuning data 7 that is input to the diffusion model 3 and training the diffusion model 3. It is possible to determine and quantify these changes by monitoring the changes in the value tensor V 25 representation of the prompt concept 8 being processed by the cross-attention layer 17 when the fine-tuning process 5 is performed.

[0075] Furthermore, the simultaneous input of prompt concepts 8 and the determination of changes in the cross-attention layer representation throughout the fine-tuning training 5 process allow the changes determined at each incremental stage to be associated with the fine-tuning images 71 that were just input into the diffusion model 3 and thus caused those changes. In other words, by repeatedly inputting prompt concepts 8 while simultaneously running the fine-tuning process 5 by inputting fine-tuning images 73 into the diffusion model 3, and monitoring the changes in the internal representation of prompt concepts 8 in the value tensor V 25 of the cross-attention layer 17, the determined changes in the value tensor V 25 can be associated with the fine-tuning images 71 that caused them, or a batch of fine-tuning images 71 that caused them. Thus, it becomes possible to aggregate the determined changes in the attribution table 9 by referring to specific output-generating images 93, their associated prompt concepts 8 that caused them, and each fine-tuning image 73 that contributed to the generated images 93, thereby relating the attribution score 95 to the determined changes and their numerical quantification.

[0076] It should be noted that the fine-tuning process 5 may involve sequentially and iteratively passing all the fine-tuning images 73 through the diffusion model 3 any number of times and in any order. This is done insofar as the diffusion model 3 tracks the current batch of fine-tuning images 71 being trained on, thereby attributing the associated changes in the cross-attention layer value tensor V 25 caused by that batch of fine-tuning images 71 to them as appropriate. Similarly, the prompt concepts 8 can be input to the diffusion model 3 in any order, and they may be input simultaneously for processing efficiency and parallelization benefits, as they do not affect each other.

[0077] In a particular embodiment of step S11, as described above, the prompt concept 8 conditions the generation via the cross-attention layer 17. The input prompt 19 is encoded into a token embedding C, which is integrated into the generation via the cross-attention layer 17. As known in the general formulation (for example from Non-Patent Document 5), W K , W V Let K = W be a matrix whose ∈θ can be adjusted. K c and V=W V c We employ a projection onto a matrix and combine it with a query matrix Q representing the image features in the current diffusion step to bridge the text and image modalities as shown in Equation 1.

number

[0078] Therefore, in aspects of the present application that refer to a specific embodiment in Figure 3 and the cross-attention layer 17, the calculation is Q i =W q f i , K=W k c, V=W v It consists of a weighted sum over c and value features like those in Equation 2.

number

[0079] Therefore, the aspects of this application have advantageously identified that monitoring and analyzing these cross-attention formulas, in particular determining and aggregating changes in V 25 throughout the entire fine-tuning process 5, is advantageously useful for data attribution.

[0080] In particular, in certain embodiments, before performing the fine-tuning process 5 on a clean fine-tuning image 71 D from a fine-tuning dataset 7, a set of predefined prompt concepts 8 is reserved to monitor their development throughout the fine-tuning process 5. This set of predefined monitoring prompts (P monitor ) forms a prompt concept 8. Each prompt concept 8 is a noise-prompt input pair

number

number

[0081] Therefore, in the particular embodiment described above, it has been identified that monitoring and analyzing these cross-attention equations, in particular discriminating and aggregating changes in the value tensor V 25 of the cross-attention layer, is advantageous, where V is as described in Equation 1 above, and V = W v c is where the superscripts "monitor,j" and / or "iter" represent the changes in V through the training iterations j=1,2, etc.

[0082] Figure 4 shows detailed pseudocode (indicated as Algorithm 1) of a specific embodiment described above in relation to Figure 3, thereby defining the monitoring of the fine-tuning process 5 of the diffusion model 3, in particular monitoring the changes in the value tensor V of the cross-attention layer 17 throughout the fine-tuning process 5, and aggregating those changes in the attribution table 9. For illustrative purposes, Figure 4 shows the i-th row and j-th column of the attribution table 9 being processed, which is structured by prompt concepts 8 (monitoring prompts) (columns) as input prompts n along the x-axis at the top, and fine-tuning images 73 (training samples) (rows) as fine-tuning data 7 along the y-axis. Thus, Algorithm 1 shown in Figure 4 represents a specific embodiment of steps S11 and S13. For completeness, Algorithm 1 is included below. [Table 1]

[0083] Furthermore, this process can be repeated multiple times in a sequential iterative manner, for example, batches of fine-tuning data 7 can be repeatedly input. Here, a batch represents all or a subset of the fine-tuning images 73 in the fine-tuning data 7. Advantageously, aspects of the present invention can accurately handle batches of fine-tuning data 7 of any size, particularly through sequential iterative processing. For example, for batches of 1, changes in internal representation are attributed precisely to the training sample of only one fine-tuning image 73. However, larger batches may also experience additional noisy attribution, while maintaining correct attribution. Advantageously, however, the noisy attribution is averaged and smoothed out over epochs, allowing accurate attribution to prevail even with larger batch sizes. For example, according to Equation 3, in iterative step iter, the attribution table 9 has entries M[i,j] for all i in the batch and for all j regardless of their relevance to i. For example, if i1 and i2 represent different concepts A and B, and j is associated with concept A (i.e., from the associated prompt concept 8), then both i1 and i2 are Δ iter V j,monitor Updates are received from [source]. Nevertheless, the correct attribution (j→i1) is consistently applied, while the incorrect attribution (j→i2) becomes negligible over time due to the random distribution across batches and the averaging effect across the entire fine-tuning training process 5. Thus, smaller batches of fine-tuning data 7 achieve higher performance earlier, but advantageously, as the epochs increase, the recall becomes the same for all batch sizes.

[0084] Therefore, in embodiments of the present application, such as those shown in Figures 3 and 4 and described above, changes in V throughout the fine-tuning process 5 may be determined and monitored, and these changes may be aggregated in an attribution table 9. In particular, a data attribution score 95 may be provided, which is a numerical quantification of the contribution of each fine-tuning image 73 to the output generated image 93, and the data attribution score 95 is based on the determined changes in the internal representation of the prompt concept 8.

[0085] Figure 5 shows an example of a data attribution table 9 created according to any of the methods described above. In particular, the data attribution table 9 according to this application can take any size, which is determined by the number of fine-tuning images 73 and the output-generated images 93 determined by the prompt concepts 8. As previously stated, the order of the fine-tuning images 73 may be grouped by concepts, such as the same or similar concepts, so that fine-tuning images 73 with the same associated concepts 71 are adjacent to each other in a block. For example, a first concept group 96 (e.g., "train") is shown, followed by a second concept group 94 ("suit") below it. Similarly, the order of the output-generated images 93 may be the same as the order of the fine-tuning images 73, in other words, the order of conceptual grouping of the fine-tuning images 73 based on concepts 71 is the same as the order of conceptual grouping of the output-generated images 93 based on the concepts of the prompt concepts 8. Note that the number of fine-tuning images 73 in a concept group does not have to be the same as the number of output-generated images 93 in the associated concept group, as shown in Figure 5.

[0086] Furthermore, according to any other described aspect or embodiment, the output generated image 93 of the data attribution table 9 may be formed by input prompt concepts 8 related to single concepts such as “cat” or mixed concepts such as “dancing cat,” as shown along the top of the output generated image 93. In fact, as will be further described below, embodiments of the present application favorably allow for improved accuracy data attribution in both inter-concept and intra-concept scenarios, and this favorable feature is partially facilitated by the structure of the data attribution table 9 as described herein, which allows for both concept group ordering and mixed and single-concept ordering.

[0087] In Figure 5, the data attribution score of 95 is shown as a percentage. However, this is only a representative example. Furthermore, the data attribution score of 95 is represented by hatching density depictions of different sizes, with higher data attribution scores of 95 being represented by higher density hatching (i.e., darker). As seen in Figure 5, the result of conceptually grouping both the fine-tuning image 73 and the output-generated image 93 in the same order is that the data attribution scores of 95 are arranged so that they are highest along the diagonal.

[0088] In certain embodiments, the data attribution score 95 is generated by assigning a real-valued score to each tuned image 71 indicating the importance of each tuned image 73 to the output image generation 93. After the tuned process 5 is completed, the attribution score 95 is unnormalized. Thus, the data attribution score 95 may be normalized, which includes dividing the columns of the attribution table 9 by the sum of the attribution scores 95 to provide valid probabilities.

[0089] According to any of the embodiments described above, in step S13, the changes in the diffusion model 3 are aggregated in the attribution table 9. In step S15, the attribution model 11 is trained. This involves inputting data from the attribution table 9 into the attribution model 11, for example, using data from the attribution table 9 for (target) labels / annotations.

[0090] In particular, in the embodiment of step S15, the attribution table 9 prepared according to any of the described embodiments is considered to accurately represent the data attribution scheme of the diffusion model 3 for the pre-configured monitored prompt concept 8 at this stage. In the embodiment of step S15, this stage is advantageous in that the attribution model 11 may be able to generalize the data attribution scheme of the diffusion model 3 to prompts that have not yet been seen, for example, where the input prompt is not the pre-configured monitored prompt concept 8.

[0091] The attribution model 11 may be any suitable machine learning model, such as a neural network. In a particular embodiment of step S15, the attribution model 11 is a sham (i.e., twin) neural network. In particular, a sham network can compute comparable output vectors using the same weights while working together for two different input vectors, where one of the output vectors may be a pre-computed baseline on which the other vector can be compared. In particular, the attribution model 11 may be configured to generate and learn an image embedding space. The similarity between the generated output image and the fine-tuned training image in terms of proximity in the image embedding space corresponds to attribution. In other words, attribution may also be determined through the similarity of image embeddings in the image embedding space, where images with the same concept are embedded closer to each other in the image attribution space. Thus, in a particular embodiment, proximity in the image embedding space corresponds to conceptual similarity.

[0092] For example, Figure 6 illustrates a specific embodiment of the process according to step S15, where the attribution model 11 is trained by inputting data from the attribution table 9, and the attribution model 11 generates an image embedding space 27. Note that the embedding space 27 in Figure 6 contains an exemplary data representation for visual understanding only, and the concept is indicated by different opacity hatching as shown.

[0093] Therefore, using this image embedding space 27, in a particular embodiment, the objective of the attribution model 11 is to predict attribution scores for each image pair of tuned image 73 and output generated image 93 in the data attribution table 9, and to learn from these predictions by comparing them to the ground truth of actual data attribution scores 95 as provided in the table. In particular, the attribution model 11 aims to learn the attribution table 9 in the sense that it can reproduce (approximately, if not strictly) the attribution scores 95 in the attribution table 9 for any combination of tuned image 73 and output generated image 93. The attribution model 11 learns this information by modifying the internal representation and network weights of the image embedding space 27 so that it is configured (sufficiently) accurately so that it can be used in a reliable and accurate manner to predict data attribution between tuned image data 73 and a new output generated image created by a new, unseen input prompt.

[0094] Therefore, in a particular embodiment, training the attribution model 11 further includes inputting image pairs from the data attribution table 9 into the data attribution model 11, where the image pairs include a fine-tuning image 73 and an output-generating image 93, and for each image pair, creating a fine-tuning image embedding of the fine-tuning image 73 in the image embedding space 27; creating an output-generating image embedding of the output-generating image 93 in the image embedding space 27; performing a comparison between the fine-tuning image embedding and the output-generating image embedding; and determining a predicted data attribution score that provides a numerical quantification of the contribution of the fine-tuning image 73 to the output-generating image 93 based on the comparison. Furthermore, in a particular embodiment, training further includes comparing the predicted data attribution score with a data attribution score 95 from the data attribution table 9 associated with the image pair; and adjusting the network weights of the attribution model 11 based on the comparison. The comparison may also be based on determining whether the loss function is minimized.

[0095] In certain embodiments, the attribution model 11 may be configured to learn the image embedding space 27 using a distance metric learning (DML) process (as described, for example, in Non-Patent Document 3). This process has been advantageously identified as particularly effective in learning image embeddings in scenarios similar to the novel applications of the present invention. As described above, in this image embedding space 27, images with the same concept are embedded closer to each other in the image attribution space 27, and attribution can be determined through the similarity of image embeddings in the image embedding space 27. In particular, in certain embodiments, the attribution model 11 is in the form of a Siam network and is trained to distinguish between conceptually similar (positive) pairs and different (negative) pairs of output-generated images and fine-tuned customization images.

[0096] In particular, Figure 7 shows a specific embodiment of DML training of an attribution model 11 including a sham network. The attribution model 11 is trained using data from an attribution table 9, and in particular using image pairs from a data attribution table 9. In particular, in this specific embodiment of the sham network, two image pairs, namely a positive image pair 29 and a negative image pair 31, must be processed. Here, the positive image pair 29 includes a tweaked image 73 and an output-generated image 93 that are conceptually the same or similar, and the negative image pair 31 includes a tweaked image 73 and an output-generated image 93 that are conceptually dissimilar or different. It is possible to select the positive image pair 29 and the negative image pair 31 in several ways. Firstly, since the concept 71 associated with the tweaked image 73 is known and the prompt concept 8 that generated the output-generated image 93 is known, the positive image pair 29 and the negative image pair 31 can be selected in this way. Alternatively or additionally, conceptual similarity can be determined using attribution scores 95 (determined in any of the aforementioned ways) obtained from data attribution table 9, and thus, for any given pair of images, it can be determined whether it is a positive image pair 29 or a negative image pair 31. For visual convenience, Figure 7 shows only two pairs of images, namely one positive image pair 29 and one negative image pair 31, but training is performed across all image pairs in data attribution table 9.

[0097] Therefore, in certain embodiments, the attribution model is trained to distinguish between conceptually similar and conceptually different pairs of image pairs. This includes, for any pair of image pairs: determining a first predictive attribution score for a first pair of image pairs that are positive image pairs containing conceptually similar tweak and output-generating images, and determining a second predictive attribution score for a second pair of image pairs that are negative image pairs containing conceptually different tweak and output-generating images.

[0098] During training as shown in Figure 7, in the forward pass, each image pair undergoes a two-step transformation, including initial feature extraction by a pre-trained embedder 33. The embedder 33 feeds into a custom scaler layer 35 trained for the task, which outputs the final embedding in the image embedding space 27. The predicted attribution score is then obtained as the vector embedding of the output image generation 93 and the fine-tuning image 73 in that image pair, i.e., the shifted cosine similarity 37 between the fine-tuning image embedding and the output generation image embedding in the image embedding space 27. As background, cosine similarity measures the cosine of the angle between two vectors in a multidimensional space, but sometimes the data may have a bias or offset that can affect the cosine similarity measurement, in which case shifted cosine similarity may be used to adjust for the bias, for example by cantering or shifting the data before calculating the similarity. In particular, in this case, in certain embodiments, the shifted cosine similarity is used to modify the function output to be between [0,1] (as opposed to the original output which is between [-1,1]), which is done by normalization. Thus, the function of shifted cosine similarity is applied here to align the ground truth similarity values ​​of the actual data attribution scores 95 from attribution table 9, which are between [0,1], with the ground truth, thereby favorably improving the learning process.

[0099] Next, the loss function 39 is used to measure the loss of the predicted attribution score to the ground truth attribution score 95 obtained from the attribution table 9 for the associated image pair. In the reverse pass (i.e., backpropagation), the weights and biases of the attribution model 9 are adjusted as appropriate, using information from the comparison using the loss function 39, so that the difference between the predicted and ground truth attribution scores in the image embedding space 27 is learned and the difference is minimized. In other words, the loss function 39 is used to train the attribution model 11 by successively and iteratively generating larger conceptual groups in the image embedding space 27. The order in which the image pairs are processed by the attribution model 11 does not matter and may be done randomly. It does not matter whether the positive image pair 29 and the negative image pair 31 are conceptually similar or partially related to the same concept.

[0100] The embedder 33 may be a pre-trained image-text model that acquires an image and outputs a vector, the scaler 35 may be a custom neural network layer that performs linear transformations (scaling and shifting) on ​​its input, and optionally there may be a mapper layer which is a nonlinear transformation network for transfer learning that transforms the input vector into an output vector. The embedder 33 may be any suitable embedder, and in certain embodiments it may be, for example, CLIP (Non-Patent Literature 3).

[0101] The loss function 39 may be any suitable loss function. In certain embodiments, advantageously, a novel loss function 39 is provided that is specifically tuned for a customized DML model, which is referred to here as the adaptive (DML) loss function. This adaptive loss function 39 is applied in scenarios where the distance between samples in the dataset is predetermined, as in the current data attribution scenario, and the attribution model 11 learns values ​​from the attribution table 9 (i.e., the distance between two images) and distinguishes between their concepts. Advantageously, using this adaptive loss function leads to finer granularity in model prediction. Furthermore, this adaptive loss function provides improved performance over traditional triplet loss functions used in Siam networks, as outlined in Non-Patent Document 3, for example. In particular, unlike the novel adaptive loss function of this application, traditional triplet loss functions only ensure that the anchor sample is closer to positive samples (same class) than to negative samples (different classes) by a certain fixed margin. Here, the margin specifies the minimum difference between the anchor-positive distance and the anchor-negative distance for which the loss is zero. Favorably, the adaptive loss function of this application integrates the attribution scores 95 from the attribution table 9 into the loss function in the following manner:

[0102] In particular, (P ap ,P np ), (GT ap GT np ) are treated as positive and negative pairs of predicted attribution and ground truth attribution (attribution score 95 obtained from attribution table 9), respectively, P ap , P np P is the i-th entry in each case. api , P npiLet B be the number of pairs in a batch of fine-tuning data 7, i.e., the number of fine-tuning training images 73. To explain the distance between concepts, the present invention introduces a novel Adaptive Triplet Loss function as shown in Equation 4.

number

number

[0103] In particular, the novel loss function provided herein favorably incorporates attribution values ​​(both predicted and ground truth) into the loss function. When combined with an ordered conceptual grouping structure of the data attribution table, as illustrated with reference to Figure 5, for example, this allows the attribution model to be particularly effective in learning at intra-conceptual (i.e., within one conceptual group 92) as well as inter-conceptual granularity. Current technology systems are not simply able to perform this intra-conceptual learning and are instead limited to coarse conceptual level distinctions. In particular, a data attribution table 9 including an ordered conceptual grouping of multiple fine-tuning images 73, as shown in Figure 5, for example, thereby has multiple fine-tuning images 73 in the same concept. Each of these has a different attribution score 95 with respect to a particular prompt concept 8 and output generated image 93. Thus, the determination, collection, and incorporation of these multiple attribution scores 95 within the same concept 92 allows the attribution model to learn intraconceptual distinctions through a specialized loss function that includes these attribution scores 95. In other words, since the different attribution scores 95 relate to the level of conceptual relevance of multiple different tuned images 73 within the same concept group 92, the conceptual distinctions within this concept group 92 can be learned by the attribution model 11 because the information is incorporated into the specialized loss function, which guides the learning / training process of the attribution model 11.

[0104] Therefore, in a particular embodiment where the data attribution table 9 is ordered by conceptual grouping, as described above with reference to Figure 5, for example, the new adaptive DML loss function thereby provides a hierarchical order of learned attributions, leading to an understanding of both intra- and inter-conceptual relationships, thereby improving the conceptual attribution granularity. Thus, in embodiments of the present application, the novel adaptive DML loss function combined with the novel data attribution table 9 can allow for advantageous utilization of the structure of the attribution table 9, thus allowing for a finer analysis of inter-conceptual relationships, ultimately leading to an increase in the granularity of the final predictive attribution of individual concepts in the trained attribution model 11. In contrast, attribution methods of current techniques using contrastive attribution are limited to making only coarse distinctions between concepts, rather than distinctions between intra- and inter-conceptual relationships.

[0105] Therefore, according to the arbitrary embodiment described above, the loss function operates to maximize the similarity (minimize the difference) between the ground truth attribution score 95 from the attribution table 9 and the predicted attribution score. Thus, the attribution model 11 is trained. Training may proceed for each image pair combination from the attribution table 11, i.e., for all cells in the attribution table, and this may be performed sequentially and iteratively until the loss function is minimized.

[0106] Therefore, according to any of the embodiments described above, the attribution model 11 of the present application is trained with data from the attribution table 9 according to step S15, thereby the attribution model 11 can then be used to predict the attribution of previously unseen output generation—such as image generation in the context of a diffusion model—to training data, such as a finely tuned (customized) image dataset 7. For example, once steps S11-S15 described above are performed according to any of the embodiments described above, the attribution model 11 may be frozen, i.e., its network and weights may be frozen, thereby the attribution model 11 is considered trained and can then be used to perform data attribution. In other words, the information contained in the data attribution table 9, as determined in steps S11 and S13, is favorably generalized and learned by the attribution model 11, thereby enabling accurate data attribution to novel, previously unseen output generation images, such as those generated by the finely tuned diffusion model 3.

[0107] Figure 8 illustrates the use of a trained attribution model in a particular aspect. In particular, the use of a trained attribution model involves performing data attribution using an attribution model trained according to any of the aforementioned aspects or embodiments of the present application.

[0108] Step S81 includes selecting a generated output generated by the generating AI model of any of the aforementioned aspects or embodiments of the present application.

[0109] Step S83 involves inputting the generated output into a data attribution model.

[0110] Step S85 includes outputting a data attribution score from the data attribution model relating to at least one training input used when the generative AI model was trained, the data attribution score providing a numerical quantification of the contribution of the at least one training input to the generated output.

[0111] Any of the steps in Figure 8 can be performed by an apparatus as described with reference to Figure 11 below.

[0112] Advantageously, this aspect of the present invention provides an accurate and efficient way to perform data attribution on the output of a generative AI model, for example, on images generated by a diffusion model trained on existing image data.

[0113] In particular, the attribution model 11 is any attribution model 11 as described above after its training in steps S11 to S15. Therefore, the trained attribution model 11 can be used to perform data attribution on any output generated by the aforementioned diffusion model 3, for example, on any output whose output is the output generated image 93, against any aforementioned generative AI model that has been trained against it.

[0114] For example, when a prompt, such as an unfamiliar prompt, is applied to a generative AI model, the generative AI model processes the prompt and produces a generated output. For example, in the context of the text-image diffusion model 3, the generated output is the generated image. This generated output, such as the generated image, is then input into an attribution model, thereby the attribution model processes the generated output and outputs a data attribution score itself. The data attribution score is a score that provides a numerical quantification of the contribution of at least one of the training inputs to the generated output. Furthermore, the data attribution model can provide data attribution scores for all training inputs available to it, such as the fine-tuning training image data 73 mentioned above.

[0115] In certain embodiments, the data attribution model 11 can create a data attribution table using new prompts and new output generation, providing a data attribution score for each training input, such as finely tuned image data 73, thereby creating and completing columns in the data attribution table. These new columns may be added to the original data attribution table 9 from which the attribution model 11 was trained, and the attribution model 11 may be trained at a later stage on the new data attribution table thus created.

[0116] Therefore, according to this aspect, accurate and efficient data attribution is provided for the output of generative AI models, particularly the text-image diffusion model 3.

[0117] Aspects of the present invention provide novel and advantageous methods for training and providing data attribution models that can provide accurate and efficient data attribution to generative AI models such as diffusion models. The aspects of the present invention include several aspects, each individually novel and advantageous, and include, for example, leveraging access to training / fine-tuning stage data, determining and providing an efficient representation of the generative process within the generative model (e.g., a value tensor V in the cross-attention layer), and providing sequential iterative updates that support continuous learning (such as in an online environment).

[0118] Therefore, the aspects of this application provide an attribution model with advantageously improved accuracy and efficiency for generative AI, particularly for image diffusion models. A generated image is input to the attribution model, and technical image analysis and processing are performed on the generated image to determine the attribution of that generated image to a training image. Such an improved attribution model can find useful and advantageous applications in several different fields, such as the following: • Model debugging, such as identifying problems like overfitting, underfitting, or bias in the data; • Content attribution, for example, protecting intellectual property rights by efficiently identifying works on a platform that provides AI-generated content to users and attributing them to the original creators. In these contexts, both accuracy and efficiency are extremely important. For example, detecting contaminated / mislabeled samples by identifying incorrect, misleading, or intentionally harmful data; For example, a continuous learning application compatible with generative models that are continuously trained and updated. • Trustworthy AI systems: Supporting interpretability by explaining the behavior and predictions of a model, for example, why a model made a particular prediction, is crucial for developing trustworthy AI systems.

[0119] Figure 9 shows a table of results for applying attribution models trained by various existing technology processes (GenDataAttribution and DTRAK mentioned above) and attribution models trained by the training process described herein (labeled "the present application") to the CustomCONncept101 dataset for a data attribution task. Figure 10 shows a table of results for applying attribution models trained by various existing technology processes (GenDataAttribution and DTRAK mentioned above) and attribution models trained by the training process described herein (labeled "the present application") to the Archive dataset for a data attribution task. In both cases, the generative model used was custom diffusion (Non-Patent Literature 6). [Non-Patent Document 6] Kumari, N., Zhang, B., Zhang, R., Shechtman, E., Zhu, JY: Multi-concept customization of text-to-image diffusion. In: Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. pp. 1931-1941 (2023)

[0120] The experimental setup and procedure for both Figure 9 and Figure 10 are the same and are therefore described only once here. To estimate the capabilities of the attribution model trained according to this application and to generalize the learned data attribution table, thus providing accurate predicted attribution scores, generated images (with respect to training) that have not been seen before are used, generated with unseen prompts. Attribution is tested at the semantic concept level. Image concepts are used as ground truth labels, and the attribution model is evaluated for its ability to assign high attribution scores to input images belonging to the same concepts as those generated. The metrics used were recall@K and precision@K, which are commonly applied in DML settings, as well as Spearman's rank correlation, which was used to compare the ordering of predicted attributions with the original attributions generated by the data attribution table.

[0121] The experimental procedure was as follows: Analysis was performed on data attribution tables constructed for multiple concept model customizations (5 and 10 concepts), and the datasets were split accordingly (for example, in the CustomConcepts101 dataset with 101 concepts, 20 5-concept customizations and 10 10-concept customizations were monitored). For each table, three attribution models were trained with diverse seeds, yielding hundreds of attribution models. The experiment included non-mixed concept images, where each generated image contained one concept, and mixed concept images, where each generated image contained two concepts from 5 or 10 customization concepts. For recall@K and precision@K of generated images not seen before, K=5 was set for non-mixed concepts and K=10 for mixed concepts (a larger K is used because those attributions span more images).

[0122] The results are as follows. Interconcept evaluations are obtained for the "Precedent," GenDataAttribution, and D-TRAK datasets, CustomConcept101 (Figure 9) and Archive (Figure 10). As can be seen, the "Precedent" model consistently outperformed GenDataAttribution and DTRAK, achieving the highest scores for all metrics in both unmixed and mixed-concept experiments. In particular, the performance of the "Precedent" model was significantly superior in mixed-concept experiments compared to unmixed-concept experiments. This demonstrates the favorable capability of the attribution model by the "Precedent" model when used, for example, as an "artistic style" detector, as demonstrated by the results for the Archive dataset.

[0123] The significant performance gap between attribution models trained according to the method outlined herein and attribution models of the current technology clearly demonstrates the advantageously improved capabilities of aspects of this invention in providing accurate data attribution, particularly with respect to more complex mixed-concept generation.

[0124] In any aspect or embodiment of this application, the images used as input to any of the described models, including training images and base models and fine-tuning models, may be real-world image data from a real-world sensor such as a camera.

[0125] Aspects of the present invention provide a novel data attribution method with improved accuracy and efficiency, particularly well suited to fine-tuned (customized) image-text diffusion models. Advantageously, aspects of the present invention provide direct monitoring of the diffusion model's internal representation during training, improving the accuracy of subsequent data attribution learning and the data attribution accuracy of the final output-generated images in the final fine-tuned diffusion model. Following the monitoring phase, the creation of an attribution model that receives information from this monitoring is performed. The attribution model can be trained on an attribution table that aggregates the information determined during monitoring of the diffusion model's internal representation during fine-tuning.

[0126] The aspects of the present methodology offer a unique perspective on how training data influences image generation, improving accuracy while maintaining efficiency. Furthermore, evaluation of the aspects of the present methodology with customized and artistic style datasets (both important use cases for data attribution) demonstrates clear advantages in terms of intra- and inter-conceptual granularity and precision levels, thereby providing a valuable tool in the field of data attribution.

[0127] Example implementation of a computer system Figure 11 is a block diagram of an information processing apparatus 10 or computing device 10 embodying the present invention, such as a data storage server, which can be used to implement some or all of the operation of the method embodying the present invention and to perform some or all of the tasks of the apparatus of one embodiment. The computing device 10 can be used to implement any of the method steps and / or any of the processes described above.

[0128] The computing device 10 comprises a processor 993 and memory 994. Optionally, the computing device also includes a network interface 997 for communicating with other such computing devices. Optionally, the computing device also includes one or more input mechanisms, such as a keyboard and a mouse 996, and one or more display units, such as a monitor 995. These elements can facilitate user interaction. The components are connectable to each other via a bus 992.

[0129] Memory 994 may include computer-readable media, and the term may refer to one or more media configured to carry computer-executable instructions (e.g., a centralized or distributed database and / or associated caches and servers). Computer-executable instructions may include, for example, instructions and data that are accessible by a computer (e.g., one or more processors) and cause the computer to perform one or more functions or operations. For example, computer-executable instructions may include instructions for implementing the methods disclosed herein, or any method steps disclosed herein and / or any processes described above. Thus, the term “computer-readable storage medium” may also include any medium capable of storing, encoding, or carrying a set of instructions for machine execution and causing a machine to perform any one or more of the method steps of the Disclosure. Thus, the term “computer-readable storage medium” may be interpreted to include, but not limited to, solid-state memory, optical media, and magnetic media. Such computer-readable media may include, but are not limited to, non-temporary computer-readable storage media, including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, and flash memory devices (e.g., solid-state memory devices).

[0130] The processor 993 is configured to control the computing device and perform processing operations, for example, by executing computer program code stored in memory 994 to perform any of the method steps described herein. Memory 994 stores data read and written by the processor 993, and can store training data and / or network weights and / or patches and / or updated patches and / or embeddings and / or vectors and / or graphs and / or representations and / or difference magnitudes and / or formulas and / or other data described above, and / or programs for performing any of the method steps and / or processes described above. As referred to herein, the processor may include one or more general-purpose processing devices such as a microprocessor, a central processing unit, etc. The processor may include a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing a processor or combination of instruction sets. The processor may also include one or more dedicated processing devices such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and network processors. In one or more embodiments, the processor is configured to execute the operations and instructions for performing the operations described herein. The processor 993 may be considered to include any of the modules described above. Any operations described as being implemented by a module may be implemented by a computer, for example by the processor 993, as an method.

[0131] Optionally, the apparatus 10 includes a display unit 995 capable of displaying a representation of data stored by a computing device. The network interface (network I / F) 997 may be connected to a network such as the Internet, and can connect to other such computing devices via the network. The network I / F 997 can control the input and output of data to and from other devices via the network.

[0132] Other peripheral devices such as microphones, speakers, printers, power supply units, fans, cases, scanners, and trackballs may also be included in the computing device.

[0133] A method of embodying the present invention may be performed on a computing device / apparatus 10 as shown in Figure 11. Such a computing device does not need to have all the components shown in Figure 11, but may consist of a subset of those components. For example, apparatus 10 may comprise a processor 993 and a memory 994 connected to the processor 993. Alternatively, apparatus 10 may comprise a processor 993, a memory 994 connected to the processor 993, and a display 995. A method of embodying the present invention may be performed by a single computing device that communicates with one or more data storage servers over a network. The computing device may be a data storage device itself that stores at least a portion of the data.

[0134] The method for embodying the present invention may be performed by a plurality of computing devices working together. One or more of the plurality of computing devices may be a data storage server that stores at least a portion of the data.

[0135] The present invention may be implemented in digital electronic circuits, or in computer hardware, firmware, software, or a combination thereof. The present invention may be implemented as a computer program or computer program product, i.e., a computer program tangibly embodied in a non-temporary information carrier, such as a machine-readable storage device or a propagating signal, for execution by or control of the operation of one or more hardware modules.

[0136] Computer programs may be in the form of standalone programs, computer program portions, or two or more computer programs, and may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a data processing environment. Computer programs may be deployed to run on one module or more modules at one site, or they may be distributed across multiple sites and interconnected by a communication network.

[0137] The method steps of the present invention may be performed by one or more programmable processors that execute a computer program to perform the functions of the present invention by acting on input data to generate an output. The apparatus of the present invention may be implemented as programmed hardware or as a dedicated logic circuit including, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0138] Processors suitable for executing computer programs include, for example, both general-purpose and dedicated microprocessors, and any one or more processors in any type of digital computer. Generally, a processor receives instructions and data from read-only memory or random-access memory or both. An essential element of a computer is a processor for executing instructions, coupled with one or more memory devices for storing instructions and data.

[0139] For the purposes of this disclosure, the term “machine learning model” encompasses the following concepts within its scope: A learning algorithm that includes a process or instructions through which data can be used in the training process to generate a model artifact for performing a given task or for representing a real-world process or system. • Model artifacts, including the computational architecture created by such a training process to perform the task, and • The process performed by a model artifact to complete a task.

[0140] Therefore, references to "machine learning models," "models," "model parameters," and "model information" can be understood as relating to any one or more of the above concepts that fall within the scope of "machine learning models." The above-described embodiments of the present invention may be used, advantageously, independently of any other embodiment, or in any feasible combination with one or more other embodiments.

[0141] Alternative Embodiments The embodiments described above are not limiting to the present invention, but rather illustrative. Any alternative embodiments that become apparent from the above description may fall within the scope of the present invention.

[0142] Alternative statements of the present invention are listed below as numbered clauses. [Clause 1] A computer-implemented method for training a machine learning attribution model configured to provide data attribution for output generation of a generative artificial intelligence (AI) model: A step in which changes in the generated AI model are determined during the training process; The steps include: consolidating the aforementioned changes into an attribution table; A step of training the attribution model, which includes inputting data from the attribution table into the attribution model. Methods that include... [Article 2] The aforementioned training process is a fine-tuning process for the computer implementation of any preceding clause. [Clause 3] A computer implementation of any preceding clause, wherein determining the change includes determining the change in the internal representation in the generative AI model while the training data is input to the generative AI model and processed by the generative AI model during the training process. [Clause 4] Determining the changes in the internal representation of the generative AI model during the training process is done simultaneously with executing the training process: A prompt concept configured to cause the generation AI model to generate output is input to the generation AI model; To determine the change in the internal representation of the prompt concept. The computer implementation methods described in Clause 3, including those described in Clause 3. [Article 5] The aforementioned generative AI model is a diffusion model, as described in any one of clauses 2 to 4, and is implemented by computer. [Article 6] The diffusion model is an image-text diffusion model, as described in the computer implementation method of Clause 5. [Article 7] The training process comprises inputting fine-tuning data into the diffusion model as training data, the fine-tuning data comprising image-concept pairs, each image-concept pair comprising a fine-tuning image and an associated concept comprising a text description relating to the visual content of the image, as described in the computer-implemented method of Clause 6. [Clause 8] Determining the changes in the internal representation of the diffusion model during the training process is done simultaneously with performing the training process: A prompt concept configured to cause the diffusion model to generate an output image is input to the diffusion model; To determine the change in the internal representation of the prompt concept. The computer implementation methods described in Clause 7, including those described in Clause 7. [Article 9] The computer implementation method according to Clause 8, wherein the internal representation includes a vector representation of the prompt concept in the cross-attention layer of the diffusion model. [Clause 10] The computer implementation method according to Clause 9, wherein the internal representation includes the value tensor of the cross-attention layer. [Clause 11] The data attribution table includes a data structure that associates each output-generated image generated by the prompt concept with an attribution score that provides a numerical quantification of the contribution of each fine-tuning image to the output-generated image, wherein the attribution score is based on a determined change in the internal representation of the prompt concept, as described in any one of clauses 8 to 10. [Article 12] The computer-implemented method described in Clause 11, wherein the rows of the data attribution table relate to the fine-tuning image, and the columns of the data attribution table relate to the output generated image. [Article 13] The computer implementation method described in Clause 12, wherein the data attribution table is such that the fine-tuning images are ordered and grouped by concepts taken from associated concepts of specific image-concept pairs, and the output generated images are ordered and grouped by the prompt concepts. [Article 14] Training the aforementioned attribution model involves: The process includes inputting image pairs from the data attribution table into the data attribution model, wherein the image pairs include a fine-tuning image and an output generation image, and for each image pair, In the image embedding space, create an embedded image of the fine-tuning image; In the aforementioned image embedding space, create an output generation image embedding of the output generation image; A comparison is performed between the aforementioned fine-tuning image embedding and the aforementioned output generation image embedding; Based on the comparison, the process includes determining a predictive attribution score that provides a predicted numerical quantification of the contribution of the fine-tuning image to the output generated image. A computer implementation method as described in any one of clauses 11 through 13. [Article 15] The aforementioned training involves: The predicted attribution score is compared with the attribution score from the data attribution table associated with the image pair; Based on the above comparison, the process includes adjusting the network weights of the attribution model. The computer implementation method described in Clause 14. [Article 16] The attribution model is further trained to distinguish between conceptually similar and conceptually different pairs of images, for each pair of images: A step of determining a first predictive attribution score for a first pair of images, wherein the first pair of images is a positive image pair comprising a conceptually similar fine-tuning image and an output generation image; A step in which a second predictive attribution score is determined for a second pair of images, wherein the second pair of images is a negative image pair that includes a conceptually different fine-tuning image and an output generation image. The computer implementation methods described in Clause 14, including those described therein. [Article 17] For all pairs of image pairs in the aforementioned data attribution table: This includes a step of adjusting the network weights of the attribution model based on minimizing the loss function, wherein the loss function is

number

Claims

1. A computer-implemented method for training a machine learning attribution model configured to provide data attribution for generating the output of a generative artificial intelligence (AI) model: A step in which changes in the generated AI model are determined during the training process; The steps include: aggregating the aforementioned changes into an attribution table; A step of training the attribution model, which includes inputting data from the attribution table into the attribution model. A computer implementation method that includes this.

2. The computer-implemented method according to claim 1, wherein determining the change includes determining the change in the internal representation in the generative AI model while the training data is input to the generative AI model and processed by the generative AI model during the training process.

3. Determining the changes in the internal representation of the generative AI model during the training process is done simultaneously with executing the training process: A prompt concept configured to cause the generation AI model to generate output is input to the generation AI model; To determine the change in the internal representation of the prompt concept. The computer-implemented method according to claim 2, including the method described in claim 2.

4. The aforementioned generative AI model is a diffusion model. The aforementioned diffusion model is an image-text diffusion model. The computer-implemented method according to claim 2.

5. The computer-implemented method according to claim 4, wherein the training process comprises inputting fine-tuning data into the diffusion model as training data, the fine-tuning data comprising image-concept pairs, each image-concept pair comprising a fine-tuning image and an associated concept comprising a text description relating to the visual content of the image.

6. Determining the changes in the internal representation of the diffusion model during the training process is done simultaneously with executing the training process: A prompt concept configured to cause the diffusion model to generate an output image is input to the diffusion model; To determine the change in the internal representation of the prompt concept. The computer-implemented method according to claim 5, including the method described in claim 5.

7. The internal representation includes a vector representation of the prompt concept in the cross-attention layer of the diffusion model, The internal representation includes the value tensor of the cross-attention layer, The computer-implemented method according to claim 6.

8. The computer-implemented method according to claim 7, wherein the data attribution table includes a data structure that associates each output-generating image generated by the prompt concept with an attribution score that provides a numerical quantification of the contribution of each fine-tuning image to the output-generating image, the attribution score being based on a determined change in the internal representation of the prompt concept.

9. The rows of the data attribution table relate to the fine-tuning image, and the columns of the data attribution table relate to the output generated image. The data attribution table is such that the fine-tuning images are ordered and grouped by concepts taken from associated concepts of specific image-concept pairs, and the output generated images are ordered and grouped by the prompt concepts. The computer-implemented method according to claim 8.

10. Training the aforementioned attribution model involves: The process includes inputting image pairs from the data attribution table into the data attribution model, wherein the image pairs include a fine-tuning image and an output generation image, and for each image pair, In the image embedding space, create an embedded image of the fine-tuning image; In the aforementioned image embedding space, create an output generation image embedding of the output generation image; A comparison is performed between the aforementioned fine-tuning image embedding and the aforementioned output generation image embedding; Based on the comparison, the process includes determining a predictive attribution score that provides a predicted numerical quantification of the contribution of the fine-tuning image to the output generated image. The computer-implemented method according to claim 9.

11. The attribution model is further trained to distinguish between conceptually similar and conceptually different pairs of images, for each pair of images: A step of determining a first predictive attribution score for a first pair of images, wherein the first pair of images is a positive image pair comprising a conceptually similar fine-tuning image and an output generation image; A step in which a second predictive attribution score is determined for a second pair of images, wherein the second pair of images is a negative image pair that includes a conceptually different fine-tuning image and an output generation image. A computer-implemented method according to claim 10, including the method described in claim 10.

12. For all pairs of image pairs in the aforementioned data attribution table: This includes a step of adjusting the network weights of the attribution model based on minimizing the loss function, wherein the loss function is [Number 9] And here: L 1 L 1 The loss function is the mean absolute error; P ap This is the predicted attribution score for the positive image pair; P np This is the predicted attribution score for the aforementioned negative image pair; GT ap This is the attribution score of the positive image pair from the data attribution table; GT np This is the attribution score of the negative image pair from the data attribution table; B is the number of fine-tuning images in the data attribution table; P npi P ap It is the i-th entry; P api is the i-th entry of P np ; m i This is a margin m derived from the difference between the attribution score of the positive image pair and the attribution score of the negative image pair. i =GT api -GT npi That is, The computer-implemented method according to claim 11.

13. The proximity in the aforementioned image embedding space corresponds to conceptual similarity, The predicted attribution is determined based on the shifted cosine similarity between the fine-tuning image embedding and the output generation image embedding in the image embedding space. The computer-implemented method according to claim 12.

14. A computer-implemented method for performing data attribution using an attribution model trained according to any one of claims 1 to 13: A step of selecting a generated output generated by a generation AI model according to any one of claims 1 to 13; The steps include: inputting the generated output into the data attribution model; The process includes the step of outputting a data attribution score from the data attribution model relating to at least one training input used when the generative AI model was trained, wherein the data attribution score provides a numerical quantification of the contribution of the at least one training input to the generative output. method.

15. A computer program that, when executed on a computer, causes the computer to perform the method according to any one of claims 1 to 13.

Images