Knowledge editing in text-to-image model

Abstract

Embodiments of the present disclosure relate to knowledge editing in a text-to-image model. Knowledge editing techniques for text-to-image models and other generative machine learning models are described. In an example, a location is identified within a text-to-image model by a model editing system. The location is configured to affect generation of visual attributes by the text-to-image model as part of the digital image. The edited text-to-image model is formed by editing the text-to-image model based on the location. When a subsequent digital image is generated from the edited text-to-image model, the edit causes a change to the visual properties. Subsequent digital images are generated from the edited text-to-image model as digital images with changes to visual properties.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 581,974, filed on September 11, 2023, entitled “Knowledge Edit in a Text-to-Image Model,” and U.S. Patent Application No. 18 / 425,335, filed on January 29, 2024, entitled “Knowledge Edit in a Text-to-Image Model,” the disclosures of which are incorporated herein by reference in their entirety. Background Art

[0003] The functionality available via machine learning models continues to expand, from the initial error correction and object recognition techniques to generative techniques that can be used to generate text, digital images, etc. To support this expanded functionality, the amount of training data used to train machine learning models continues to expand. In some cases, this expansion involves the use of more than a billion items of training data.

[0004] Therefore, technical challenges arise in gaining insight into which training data items are used as part of training, and in gaining the impact of the use of these training data items on the functionality of the machine learning model in performing the desired operations. Summary of the invention

[0005] Knowledge editing techniques for text-to-image models and other generative machine learning models are described. In an example, locations are identified within the text-to-image model by a model editing system. The identification of the locations is configured to influence the generation of visual attributes as part of a digital image by the text-to-image model. To do so in one or more examples, a causal mediation analysis is performed.

[0006] The edited text-to-image model is generated using a model editing system by editing the text-to-image model based on position. The editing causes changes to visual attributes when a subsequent digital image is generated.

[0007] This Summary introduces some concepts in a simplified form that will be further described in the Detailed Description below. Therefore, this Summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The detailed description is described with reference to the accompanying drawings. The entities represented in the drawings refer to one or more entities and thus reference to the entities in singular or plural form may be made interchangeably in the discussion.

[0009] Figure 1 is an illustration of a digital media environment in an example implementation that is operable to employ knowledge location and editing techniques for a machine learning model as described herein.

[0010] Figure 2 A system is depicted in greater detail showing the operation of a model editing system that finds a location within a text-to-image model associated with a visual attribute and edits the location.

[0011] Figure 3 Depicted are example implementations of a user interface configured to receive visual attribute input.

[0012] Figure 4 Depicts an example implementation of an authentication user interface.

[0013] Figure 5 Depicted in more detail is an implementation of a configuration showing an example architecture for a symmetric encoder-decoder machine learning model.

[0014] Figure 6 The system in an example implementation showing the operation of an encoder-decoder knowledge tracing module by a causal mediation analysis module is depicted in more detail.

[0015] Figure 7 The system in an example implementation showing the operation of a text encoder knowledge tracking module by a causal mediation analysis module is depicted in more detail.

[0016] Figure 8 Depicts the system in an example implementation showing in more detail the operation of the conceptual editing module of the model editing system in editing a text encoder machine learning model.

[0017] Fig. 9 is a flow chart depicting an algorithm as a step-by-step procedure in an example implementation of operations executable to achieve a result of forming an edited text-to-image model based on locations identified as corresponding to visual attributes in the text-to-image model.

[0018] Fig.10 is a flow chart depicting an algorithm as a step-by-step process in an example implementation of operations executable to achieve the result of forming an edited text-to-image model based on an uncorrupted machine learning model, a corrupted machine learning model, and a restored machine learning model.

[0019] Fig.11is a flow chart depicting an algorithm as a step-by-step procedure in an example implementation of operations executable toward achieving a result of forming an edited text-to-image model by editing the text-to-image model based on locations configured to affect generation of visual attributes.

[0020] Fig.12 An example system is shown that includes various components of an example device that can be implemented as described in reference Figures 1 to 11 Any type of computing device described and / or utilized to implement embodiments of the techniques described herein. DETAILED DESCRIPTION

[0021] Overview

[0022] Machine learning models continue to expand the amount of functionality made available by the models. Examples of such functionality include error correction, natural language processing, and image analysis, and have been extended to generative techniques that can be used to generate text, digital images, and the like. For example, a text-to-image model is configured to generate a digital image based on a text input. However, the training of a text-to-image model typically involves a large number of training data items (e.g., digital images and captions), the number of which can reach billions. Therefore, training a text-to-image model using this large number of training data items introduces numerous technical challenges. These technical challenges are typically caused by a lack of insight into the individual items used in the training and the corresponding functionality that is made possible by the use of these items.

[0023] In real-world implementations, for example, concerns have been raised regarding the legality of text-to-image models for rendering trademarked objects, copyrighted artistic styles, etc. Additional concerns relate to the generation of inappropriate or insensitive content, illegal content, the generation of stale digital images, etc. Concerns involving stale data, for example, relate to the generation of outdated content (e.g., generating digital images of past presidents instead of the current president).

[0024] Conventional techniques used to address these concerns involve expensive retraining of text-to-image models on training datasets that do not include these concerns, which may still involve large amounts of training data. Even so, due to the generative nature of text-to-image models, adequate protection against these potential issues is difficult, if not impossible, in conventional techniques due to the limited insight provided by the text-to-image models in determining how these digital images were created (i.e., "what" prompted the generation of digital content that exhibits potential issues).

[0025] To address these and other technical challenges, knowledge editing techniques in text-to-image models are described that are also applicable to other generative machine learning models. The knowledge editing techniques support the ability to understand how knowledge is stored within a text-to-image model, and then use that information to edit the model, for example, to prevent the problems described above. In real-world examples, these techniques support increased computational efficiency that is a thousand times faster than traditional techniques, for example, these techniques can execute in less than a second to update or remove concepts from a given generative machine learning model.

[0026] In one or more examples involving a text-to-image model, the model is trained using a large number of training data items (e.g., digital image / caption combinations) to generate digital images based solely on input text. The text-to-image model is received as input by a model editing system. The model editing system is configured to output a user interface via which visual attribute input is received to specify visual attributes of interest (e.g., to be changed or to be updated). For example, visual attributes can be configured to specify objects, styles, colors, viewpoints, actions, concepts, etc. Other non-visual attributes may also be considered, for example, in support of text generation.

[0027] The task of the model editing system is then to find locations within the text-to-image model that support the ability of the text-to-image model to generate visual attributes. To this end, causal mediation analysis is performed by the model editing system to analyze causal inferences, i.e., what causal influences the generation of the response variable, through changes in the response variable of the visual attribute based on interventions on the intermediate variables of interest of the text-to-image model.

[0028] For example, the model editing system accepts a text-to-image model as an uncorrupted machine learning model. The model editing system then generates a corrupted machine learning model based on the text-to-image model. For example, the model editing system corrupts the text-to-image model by introducing Gaussian noise to a symmetric encoder-decoder machine learning model or a text encoder machine learning model of the text-to-image model.

[0029] The model editing system then generates a restored machine learning model based on the damaged machine learning model. For example, the model editing system applies activations of one or more nodes in one or more layers derived from the text-to-image model (i.e., as an undamaged machine learning model) to the damaged machine learning model to form a restored machine learning model.

[0030] The restored machine learning model is then used to generate a candidate digital image (e.g., based on the visual attributes). The candidate digital image is compared by the model editing system to a digital image generated by the text-to-image model based on the visual attributes. If the candidate digital image includes a visual attribute based on the comparison, a location within the text-to-image model corresponding to the visual attribute is located (e.g., by one or more nodes from one or more layers from which activations are copied). The model editing system then outputs a model location indication indicating the found location, which is a basis for the knowledge used to generate the visual attribute, which can then be used to edit the text-to-image model.

[0031] For example, the model editing system may be configured to employ methods for ablating or updating concepts supported in a closed form by the text-to-image model. As previously described, these techniques support improved efficiency that is a thousand times faster than traditional fine-tuning-based methods. In an editing example involving a text encoder machine learning model of a text-to-image model, the weight matrix is ​​located from the self-attention layer. The projection weight matrix is ​​then identified and the location is updated using a caption pair (k, v), where "k" (ie, key) is the original caption and the value "v" is the caption to which the key "k" will be mapped. For example, to remove the style of Van Gogh, the key "k" is set to "Style of Van Gogh" and the value "v" is set to "Style of Generic Watercolor".

[0032] In this manner, the model editing system is configured to find locations in the text-to-image model that correspond to visual attributes (or other attributes) and edit both the text-to-image model, such as to ablate the visual attributes (e.g., remove the ability to generate a branded image), update the visual attributes (e.g., to reflect the current president), etc. Although the following discussion describes the use of a text-to-image model, these techniques may also be applied to other generative machine learning models, such as text generation, conversational AI, language models, etc.

[0033] In the following discussion, an example environment is described that employs the techniques described herein. Example procedures that can be performed in the example environment and other environments are also described. Therefore, execution of the example procedures is not limited to the example environment, and the example environment is not limited to execution of the example procedures.

[0034] Sample machine learning knowledge editing environment

[0035] Figure 11 is an illustration of a digital media environment 100 in an example implementation that is operable to employ knowledge location and editing techniques for machine learning models as described herein. The illustrated environment 100 includes a service provider system 102 and a client device 104 communicatively coupled to each other via a network 106. The computing devices implementing the service provider system 102 and the client device 104 may be configured in various ways.

[0036] For example, a computing device may be configured as a desktop computer, a laptop computer, a mobile device (e.g., assuming a handheld configuration such as a tablet computer or a mobile phone), etc. Thus, computing devices range from full-resource devices with large amounts of memory and processor resources (e.g., personal computers, game consoles) to low-resource devices with limited memory and / or processing resources (e.g., mobile devices). In addition, while a single computing device is described in some examples, a computing device also represents a plurality of different devices, such as those utilized by an enterprise to perform operations such as Fig.12 Multiple servers operating “on the cloud” as described in .

[0037] The client device 104 includes a communication module 108 that represents functionality for communicating with a service manager module 110 of the service provider system 102 via the network 106. The service manager module 110 is configured to implement a digital service 112. The digital service 112 may be used to expose various functionality to the client device 104, an example of which is illustrated as a generative artificial intelligence service 114. The generative artificial intelligence service 114 is configured to create new digital content based on received input. For example, the generative artificial intelligence service 114 may be configured to generate text, digital images, computer programs, digital music, etc.

[0038] In the illustrated example, the generative artificial intelligence service 114 employs a text-to-image model 116. The text-to-image model 116 is configured to receive a text input 118 (e.g., only text) and generate a digital image 120 from the input. The text-to-image model 116 can be configured in a variety of ways, examples of which include the use of a generative adversarial network (GAN), a diffusion model, etc. Figure 3 As further described, the diffusion model is trained by applying a series of reversible transformations to transform a simple initial distribution into a complex distribution to implement the diffusion process to generate an output (eg, digital image 120).

[0039] As described previously, conventional generative machine learning techniques face numerous concerns. Examples of such concerns include the generation of inappropriate or insensitive content, illegal content, trademarked or copyrighted content, the generation of stale digital images with outdated objects, etc.

[0040] Conventional techniques used to address these technical challenges involve expensive retraining of text-to-image models on training datasets that do not include the concern, which may still involve large amounts of training data. Furthermore, due to the generative nature of the text-to-image model, adequate protection against the potential problem is difficult, if not impossible, in conventional techniques due to the limited insight provided by the text-to-image model in determining how the digital image was created (i.e., "what" prompted the generation of digital content that exhibits the potential problem).

[0041] Central to the concern and the techniques used to address it is that training and retraining of text-to-image models (similar to other deep learning models) is expensive. Thus, conventional techniques for correcting the behavior of a deep learning model (e.g., disabling the ability to generate trademarked objects or copyrighted art styles, updating knowledge, limiting output to child-friendly images, etc.) involve expensive retraining efforts on a dataset that does not include training data items with the behavior to be corrected.

[0042] To address these and other technical issues and concerns, a model editing system 122 is implemented to edit the underlying structure of the text-to-image model 116 to form an edited text-to-image model 124 that addresses the concerns. The model editing system 122 is configured to generate the edited text-to-image model 124 so that the edited model cannot generate a digital image 120 to include visual attributes, update visual attributes in ablation examples, and cannot generate outdated visual attributes, etc.

[0043] In the illustrated example, model editing system 122 is configured to update text-to-image model 116 from stale digital image 126 to updated digital image 128 to accurately reflect the generation of digital image 120 for “Current Football Champions” text input 118. In this manner, model editing system 122 differs from conventional techniques that rely on filtering of input (e.g., for offensive information, trademarks, etc.) that can be defeated using alternative wording, such as filtering of the name of a cartoon character that is defeated by describing behaviors and situations associated with the cartoon character (such as, “the lizard from the commercial”).

[0044] In one or more implementations, the model editing system 122 is configured to implement several types of post-training behavior corrections (such as concept ablation, knowledge updating, etc.) within one second on a single graphics processing unit (GPU). Specifically, the input to the model editing system 122 is the text-to-image model 116 and the visual attribute input related to the model behavior to be edited, for example, removing the ability to generate digital images with a specific trademark. The output of the model editing system 122 is an edited text-to-image model 124 that does not have the ability to render the trademark in question while keeping other generative capabilities intact.

[0045] The model editing system 122 can be configured to continuously perform multiple edits to the text-to-image model 116. For example, the model editing system 122 generates an edited text-to-image model 124 to remove the ability to render multiple branded objects while also updating world knowledge updates (e.g., the current president). In business scenarios, this functionality supports improved protection for all parties and is flexible to customer or public feedback so as to respond quickly to edit requests. In this way, the model editing system 122 avoids the inefficiencies of traditional techniques involved in retraining and fine-tuning machine learning models by identifying precise locations associated with specific behaviors (e.g., visual attributes), examples of which are further described as implementing causal mediation analysis. The model editing system 122 also supports the use of sub-second closed-form optimization to edit the weights of the machine learning model associated with the location, which results in the generation of an edited machine learning model with updated behaviors. Further discussion of these and other examples is included in the following sections and is shown in the corresponding figures.

[0046] In summary, the functions, features, and concepts described in the above examples and the following examples are employed in the context of the example processes described in this section. In addition, the functions, features, and concepts described in this article with respect to different figures and examples are interchangeable with each other and are not limited to implementation in the context of a particular figure or process. In addition, blocks associated with different representative processes and corresponding figures in this article may be applied together and / or may be combined in different ways. Therefore, the individual functions, features, and concepts described in this article with respect to different example environments, devices, components, figures, and processes may be used in any appropriate combination and are not limited to the specific combinations represented by the examples listed in this specification.

[0047] Example Knowledge Editing for Generative Machine Learning Models

[0048] The following discussion describes knowledge editing techniques for a generative machine learning model that is implementable using the described systems and devices and is executable without retraining the model. Aspects of each of the processes are implemented in hardware, firmware, software, or a combination thereof. The process is shown as a set of blocks that specify operations executable by hardware, and is not necessarily limited to the order shown for performing the operations by the corresponding blocks. For example, the blocks of the process specify operations that are programmable by hardware (e.g., a processor, microprocessor, controller, firmware) as instructions, thereby creating a special-purpose machine for performing the algorithm as illustrated by the flowchart. Therefore, the instructions may be stored on a computer-readable storage medium that causes the hardware to execute the algorithm.

[0049] Figure 2 A system 200 showing the operation of the model editing system 122 is depicted in more detail, the system finding a location within a text-to-image model associated with a visual attribute and editing the location. In this example, the text-to-image model 116 is implemented using a symmetric encoder-decoder machine learning model (illustrated as a symmetric encoder-decoder ML model 202). For example, the symmetric encoder-decoder ML model 202 can be configured in various ways, an example of which is illustrated as a denoising U-Net model 204. The text encoder ML model 206 can be configured in various ways, an example of which is a CLIP-ViT-L / 336px text encoder. The examples of the denoising U-Net model 204 and the CLIP-VIT-L / 336px text encoder are incorporated by reference in their entirety as described in “High-resolution Image Synthesis with Latent Diffusion Models” by Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjorn Ommer. Corr, abs / 2112.10752, 2021.

[0050] The model editing system 122 is illustrated as implementing a causal mediation analysis module 208. The causal mediation analysis module 208 implements techniques for causal inference by analyzing changes in a response variable following an intervention on one or more intermediate variables of interest (also referred to as "mediators").

[0051] For example, the causal mediation analysis module 208 may be used to address internal model components (e.g., specific neurons or layers) of the text-to-image model 116 as intermediaries along a directed acyclic graph between inputs and outputs. For the text-to-image model 116, the causal mediation analysis may be used to trace the causal effects of these internal model components within the symmetric encoder-decoder ML model 202 and the text encoder ML model 206 that are employed to generate digital images with corresponding visual attributes.

[0052] For example, the causal mediation analysis module 208 may be used to employ the encoder-decoder knowledge tracing module 210 to identify causal components in the symmetric encoder-decoder ML model 202, and employ the text-encoder knowledge tracing module 212 to identify causal components in the text encoder ML model 206. For example, the encoder-decoder knowledge tracing module 210 may be configured to identify causal model components at the granularity of nodes and / or layers including nodes. For the text encoder knowledge tracing module 212, causal tracing may be performed at the granularity of hidden states of token embeddings in "c" different layers for the text encoder ML model 206.

[0053] Therefore, the causal mediation analysis module 208 is configured to localize the knowledge within the text-to-image model 116 to specific nodes and / or layers. The localized knowledge of the causal mediation analysis module 208 is then available to the concept editing module 214 to edit specific nodes and / or layers of the text-to-image model 116 to form an edited text-to-image model 124.

[0054] In the illustrated example, model editing system 122 receives visual attribute input 216 identifying a visual attribute. Figure 3 An example implementation 300 of a user interface 302 configured to receive visual attribute input 216 is depicted. Visual attribute input 216 can take various forms, illustrated examples of which include object, style, color, viewpoint, action, concept, etc. Thus, user interface 302 is configured to identify visual attributes that are the subject of edits to a machine learning model, e.g., for ablation, updating, etc.

[0055] Return to Figure 2, the causal mediation analysis module 208 is then used to locate the corresponding position in the text-to-image model 116, for example, using the encoder-decoder knowledge tracking module 210 and / or the text-encoder knowledge tracking module 212 as described above. The model position indication is then output from the causal mediation analysis module 208 to the concept editing module 214. For example, the model position indication identifies the nodes and / or layers of the text-to-image model 116 that are associated with the generation of the visual attributes. The concept editing module 214 can then be used to edit the identified locations within the text-to-image model 116 for generating an edited text-to-image model 124. In one or more implementations, a verification user interface 218 is output to verify that the model editing system 122 achieved the desired results, and / or to cause the model editing system 122 to continue using the causal mediation analysis module 208 and the concept editing module 214 on additional iterations to achieve the desired results, such as with respect to Figure 4 as further described.

[0056] For example, the denoising U-Net model 204 may be formed using seventy different layers distributed across three types of blocks (e.g., (i) downstream blocks, (ii) intermediate blocks, and (iii) upstream blocks). Each of these blocks contains a varying number of cross-attention layers, self-attention layers, and residual layers. γ There are twelve blocks in total, each of which includes a self-attention layer and a multi-layer perceptron (MLP) layer. Given a caption “c” in one or more instances (e.g., as an example of text input 118), the denoising U-Net model 204 generates a digital image 120 “x” starting from random Gaussian noise. The digital image 120 “x” encapsulates the visual attributes embedded in the caption “c”. For example, the caption “c” may contain visual attributes corresponding to an object, style, color, viewpoint, action, concept, etc.

[0057] Figure 5 The implementation of a configuration showing an example architecture of a symmetric encoder-decoder ML model 202 is depicted in more detail. A diffusion model is implemented to denoise the data through multiple iterative learning. In the example, noise is added to the data following a Markov chain across multiple time steps "t∈[0,T]". From an initial random real image "x 0 ", the noise image at time step "t" is defined as In the above expression, the variable "α t " defines the intensity of the applied random Gaussian noise, which decreases gradually with increasing time steps, so that By “∈ θ (x t , c, t)” is pre-trained to transform the noisy image “xt "Denoise to obtain "x t-1 ”.

[0058] In one or more examples, to the denoising network "∈ θ (.)” is obtained by comparing the conditional input “c” with the original digital image “x 0 "Paired Text Encoder" c=v γ (c)” is the text embedding of the caption “c”. For a given image-text pair represented by “(x, c)”, the pre-training objective for the diffusion model can be defined as follows:

[0059]

[0060] Where "θ" is a set of learnable parameters. For improved training efficiency, the denoising operation and the denoising operation can be performed in the latent space defined by "z = ε(x)". In this case, the pre-training objective learns to denoise in the latent space as follows:

[0061]

[0062] Among them, "z t =ε(x t )” and “ε” are encoders. During inference, where the goal is to synthesize an image given a text condition “c”, random Gaussian noise is iteratively denoised over a fixed range of time steps to produce the final image.

[0063] Figure 6 The system 600 is depicted in more detail in an example implementation showing the operation of the encoder-decoder knowledge tracing module 210 by the causal mediation analysis module 208. To begin in this example, the model input module 602 receives an uncorrupted machine learning model, as depicted by the uncorrupted ML model 604. For example, the uncorrupted ML model 604 may be configured as a symmetric encoder-decoder ML model 202 from the unchanged text-to-image model 116.

[0064] The model input module 602 is then passed to the corrupted model generation module 606 to generate a corrupted machine learning model (illustrated as corrupted ML model 608) from the uncorrupted ML model 604. To this end, the corrupted model generation module 606 employs the Gaussian noise generation module 610 to corrupt the uncorrupted ML model 604 using Gaussian noise, e.g., by applying noise to the activations of the nodes of the uncorrupted ML model 604.

[0065] Then, in this example, the recovery module 612 is utilized to generate a recovered machine learning model (illustrated as the recovered ML model 614) from the corrupted ML model 608. As part thereof, the replacement module 616 is utilized to replace the activations at one or more nodes and / or one or more layers of the corrupted ML model 608 with the activations from the uncorrupted ML model 604. For example, the replacement module 616 selects a layer within the uncorrupted ML model 604 and passes the activations from the layer as replacements for the activations at the corresponding layer of the corrupted ML model 608, thereby forming the recovered ML model 614.

[0066] The restored ML model 614 is then passed to a candidate digital image generation module 618 to generate a candidate digital image 620. For example, the candidate digital image generation module 618 utilizes a text input corresponding to the visual attribute in question (e.g., the visual attribute input 216), and generates the candidate digital image 620 thereby.

[0067] The knowledge detection module 622 is then utilized by the encoder-decoder knowledge tracking module 210 to compare the candidate digital image 620 with a digital image generated by the uncorrupted ML model 604 (i.e., the symmetric encoder-decoder ML model 202 in this example). The comparison is utilized to detect the similarity of the candidate digital image 620 with the digital image generated by the uncorrupted ML model 604, and therefore to detect whether the replaced activations of the nodes and / or layers from the uncorrupted ML model 604 cause a change. The comparison module 624 can be configured to perform the comparison in a variety of ways, examples of which include scoring techniques that measure visual similarity, such as cosine similarity used as an indirect estimation effect as part of causal relationship determination, perceptual similarity, etc.

[0068] This technique may be performed over multiple iterations (i.e., time steps) until a threshold similarity is reached (e.g., based on a scoring technique) to locate which nodes and / or layers of the uncorrupted ML model 604, when copied to the corrupted ML model 608 to form the restored ML model 614, cause the expression of the visual attribute. When this threshold amount of similarity is reached, a model location indication 626 is then output by the causal mediation analysis module 208 to the concept editing module 214 to serve as a basis for editing the text-to-image model 116, further discussion of which may be found with respect to Figure 8 be found.

[0069] In one or more implementations, the classifier-free guidance is used by the causal mediation analysis module 208 during inference to adjust image generation by incorporating scores from the conditional diffusion model and the unconditional diffusion model at each of the time steps. In particular, at each time step, the classifier-free guidance is used in the following manner:

[0070]

[0071] In order to use the denoising U-Net model 204“∈ θ ”, three types of models are employed as described above: (i) uncorrupted ML model 604 “∈θ”, where no classifier guidance is used by default; (ii) corrupted ML model 608 where the subject (e.g., Figure 3 Picnic basket) word embedding corrupted by Gaussian noise; (iii) and the restored ML model 614 It is similar to The uncorrupted ML model 604 is restored except for one or more of its nodes and / or layers at each time step without classifier guidance.

[0072] List of given layers let Indicates the “i” whose importance will be evaluated th " layer. and Indicates "a i ” layer. To find the activations of “a” for a specific visual attribute embedded in the caption “c” i The relative importance of the effect of the layer, replaced by the module 616 in the corrupted ML model 608 The following replacement operation is performed on the recovered ML model 614

[0073]

[0074] Next, by replacing the activations from the uncorrupted ML model 604 into the corrupted ML model 608 for “a i "layer The recovered ML model 614 without classifier guidance is executed to obtain the final latent code “z 0 ”.

[0075]

[0076] Then, the final latent code "z 0 " is passed through the decoder to obtain the 0” candidate digital image 620. In this way, the encoder-decoder knowledge tracking module 210 can be used by the causal mediation analysis module 208 to locate the parts of the symmetric encoder-decoder ML model 202 that affect the generation of visual attributes. Similar techniques can also be used by the text encoder knowledge tracking module 212 for the text encoder ML model 206, further discussion of which is included in the following description and shown in the corresponding figures.

[0077] Figure 7 The system 700 is depicted in more detail in an example implementation showing the operation of the text encoder knowledge tracking module 212 by the causal mediation analysis module 208. In this example, the causal mediation analysis module 208 employs the text encoder knowledge tracking module 212 to localize the portion of the text encoder ML model 206 associated with the visual attribute. To do so, the text encoder knowledge tracking module 212 employs a method similar to that described with respect to Figure 6 The described techniques are for a symmetric encoder-decoder ML model 202 .

[0078] The model input module 702 receives an uncorrupted machine learning model, depicted as an uncorrupted ML model 704. In this example, the uncorrupted ML model 704 may be configured as the text encoder ML model 206 from the unchanged text-to-image model 116. The uncorrupted ML model 704 is then passed to a corrupted model generation module 706 to generate a corrupted machine learning model (illustrated as a corrupted ML model 708) from the uncorrupted ML model 704. To this end, the corrupted model generation module 706 employs a Gaussian noise generation module 710 to generate Gaussian noise by applying noise to the activations of the nodes of the uncorrupted ML model 704.

[0079] Then, in this example, the recovery module 712 is utilized to generate a recovered machine learning model (illustrated as the recovered ML model 714) from the corrupted ML model 708. As part thereof, the replacement module 716 is utilized to replace the activations at one or more nodes and / or one or more layers of the corrupted ML model 708 with the activations from the uncorrupted ML model 704. For example, the replacement module 716 selects a layer within the uncorrupted ML model 704 and passes the activations from the layer as replacements for the activations at the corresponding layer of the corrupted ML model 708, thereby forming the recovered ML model 714.

[0080] The restored ML model 714 is then passed to a candidate digital image generation module 718 to generate a candidate digital image 720. For example, the candidate digital image generation module 718 utilizes a text input corresponding to the visual attribute in question (e.g., the visual attribute input 216), and generates the candidate digital image 720 thereby.

[0081] The knowledge detection module 722 is then utilized by the text encoder knowledge tracking module 212 to compare the candidate digital image 720 with the digital image generated by the uncorrupted ML model 704 (i.e., the text encoder ML model 206 in this example). The comparison is utilized to detect the similarity of the candidate digital image 720 with the digital image generated by the uncorrupted ML model 704, and thus detect whether the replaced activations of the nodes and / or layers from the uncorrupted ML model 704 cause a change. The comparison module 724 can be configured to perform the comparison in a variety of ways, examples of which include scoring techniques for measuring visual similarity, such as cosine similarity used as an indirect estimation effect as part of causal relationship determination, perceptual similarity (e.g., Dream-Sim-Score), etc.

[0082] These techniques may be performed over multiple iterations until a threshold similarity is reached (e.g., based on a scoring technique) to locate which nodes and / or layers of the uncorrupted ML model 704, when copied to the corrupted ML model 708 to form the restored ML model 714, give rise to the expression of the visual attribute. When this threshold amount of similarity is reached, then a model location indication 726 is output by the causal mediation analysis module 208 to the concept editing module 214 to serve as a basis for editing the text-to-image model 116.

[0083] Similar to Figure 6 , three states of the text encoder ML model 206 are described: (i) by “v γ ” represents the uncorrupted ML model 704 as the text encoder ML model 206, (ii) the corrupted ML model 708 where the word embeddings of the topic in a given caption “c” are corrupted, and (iii) the recovered ML model 714 It is similar to ML model 704 "v γ " is copied. Similar to the discussion of the previous figure, the task of the knowledge detection module 722 is to find The visual attributes are associated with the position, where For each of the layers to be probed in the text encoder ML model 206 :

[0084]

[0085] Evaluating as a recovered text encoder using no classifier bootstrapping The restored ML model 714 is used to generate the image “x 0 ”:

[0086]

[0087] For a given description "c", Note that Figure 6 The causal tracing of the encoder-decoder knowledge tracing module 210 is different because the text encoder ML model 206 is decoupled from the time steps, so in this example, the same recovered text encoder is used across each time step in the classifier-free guided time steps.

[0088] Figure 8 The system 800 is depicted in more detail in an example implementation showing the operation of the concept editing module 214 of the model editing system 122 in editing the text encoder ML model 206. As previously described, the model editing system 122 is configured to first understand how knowledge is implemented in the text-to-image model 116, and then use this information to utilize techniques to edit the text-to-image model 116. In the implementation example, it has been discovered that a single causal state, a first self-attention layer, is included in the text-to-image model 116. Therefore, in this example, techniques are described to update the weight matrix in the first self-attention layer in the text encoder ML model 206 independently of (e.g., without using) a digital image.

[0089] The concept editing module 214 supports various technical advantages. For example, the concept editing module 214 is data-free, indicating that the operations of the concept editing module 214 can be performed without using training digital images to update model weights. The concept editing module 214 is also approximately one thousand times faster than conventional techniques in testing to generate an edited text-to-image model 124, for example, to update or remove concepts in the text-to-image model 116. In one or more instances, the model editing system 122 can be configured to consume less than one second to update or remove a concept from a given generative model.

[0090] In the illustrated example, the concept editing module 214 includes a model input module 802, which in the illustrated example obtains the text encoder ML model 206 from the text-to-image model 116. Then, a matrix position module 804 is employed to obtain an updateable weight matrix 806 used by the text encoder ML model 206. Examples of updateable weight matrices 806 include a key weight matrix 808, a query weight matrix 810, a value weight matrix 812, and a projection weight matrix 814.

[0091] For example, the first self-attention layer of the text encoder ML model 206 includes four updateable weight matrices 806, where “W k , W q , W v” Correspondingly, there are key weight matrix 808, query weight matrix 810, and value weight matrix 812 for key embedding, query embedding, and value embedding. out The updateable weighting matrix of ” is the projection weighting matrix 814 before the final layer output.

[0092] The concept editing module 214 is configured to update the projection weighting matrix 814 "W out ” matrix. To this end, the caption pair generation module 816 generates a caption pair 818 “(k, v)”, where the key “k” is the initial caption and the value “v” is the caption to which “k” will be mapped.

[0093] For example, to remove the style of Van Gogh, the key "k" is "Van Gogh's style" and the value "v" is set to "generic watercolor style". To update the projection weighting matrix 814, the matrix update module 820 is configured to update the projection weighting matrix 814 "W out " to solve the following optimization problem:

[0094]

[0095] where “λ” is used to prevent the edited weights from deviating significantly from the original pre-trained weights “W′ out , N" regularizer, "N" represents the total number of caption pairs containing the final topic token embedding of the key and value. As shown in the following expression, due to the absence of any nonlinearity, a closed-form solution is supported. In particular, the edited projection weighting matrix 822 used to generate the edited text-to-image model 124 is configurable because the optimal projection weighting matrix "W out " can be expressed as:

[0096]

[0097] By editing the projection weight matrix 814 “W” corresponding to the first self-attention layer in the text encoder out ” Instead of editing the entire text encoder ML model 206, the updated weights can therefore be saved to the storage device as it consumes less than 0.06% of the total parameters present in the underlying text-to-image model 116.

[0098] When edited, the matrix update module 820 may also support the output of a verification user interface to verify the validity of the edits. Figure 4An example implementation 400 of the verification user interface 218 is depicted. The user interface 218 displays a first digital image 402 generated by the 116 for text input (eg, the visual attribute input 216). The verification user interface 218 also displays a second digital image 404 generated by the edited text-to-image model 124.

[0099] continue Figure 3 In the example of , the visual attribute input 216 includes the text of "picnic table". The first digital image 402 also includes an object 406 corresponding to the text (picnic table). The second digital image 404 removes the object 406. Other examples may also be considered, such as changing the picnic table to an object, (changing) the style of the table, etc. Thus, the verification user interface 218 supports the ability to view "how well" the edits to the edited text-to-image model 124 were performed.

[0100] A first option is included to "Approve" 408 the edits to the model, which stops processing by the model editing system 122. However, if the edited text-to-image model 124 does not perform as expected, a second option is also provided to "Try Again" 410 to cause the model editing system 122 to repeat the localization and editing operations above, which may include, for example, providing additional text input (e.g., returning to Figure 3 The functionality of the user interface 302) is further explained to further illustrate the visual attribute input 216.

[0101] Fig. 9 900 is a flowchart depicting a step-by-step process in an example implementation of operations that are executable to achieve a result of forming an edited text-to-image model based on locations identified as corresponding to visual attributes in the text-to-image model. To begin in this example, the causal mediation analysis module 208 of the model editing system 122 identifies locations within the text-to-image model 116. The locations are configured to generate visual attributes of a digital image (block 902).

[0102] The edited text-to-image model 124 is formed by the concept editing module 214 of the model editing system 122 by editing the positions of the text-to-image model 116. The editing causes the removal of the trained ability of the text-to-image model to generate visual attributes in subsequent digital images (block 904), an example of which is shown in FIG. Figure 4 A subsequent digital image is then generated by editing the text into the image model, where the visual attributes are removed (block 906).

[0103] Fig.101000 is a flowchart depicting a step-by-step process in an example implementation of operations executable for achieving a result of forming an edited text-to-image model based on an uncorrupted machine learning model, a corrupted machine learning model, and a restored machine learning model. The corrupted machine learning model 608, the corrupted machine learning model 708 is generated by the corrupted model generation module 606, the corrupted model generation module 706 based on the text-to-image model 116 (block 1002), for example, for a symmetric encoder-decoder ML model 202 and / or a text encoder ML model 206 using Gaussian noise.

[0104] The restoration modules 612, 712 are implemented to generate restored machine learning models 614, 714. The restoration modules 612, 712 do so by applying one or more activations from one or more nodes of the text-to-image model 116 to the corrupted machine learning model 608, 708 (block 1004).

[0105] Then, the candidate digital image generation module is implemented to generate candidate digital images 620, 720 using the restored machine learning model 614, 714 (block 1006). The candidate digital images 620, 720 are compared with corresponding digital images generated by the text-to-image model 116 by the knowledge detection module 622, 722 (block 1008), for example, for the same text input.

[0106] In this example, the change is detected as an edit to the visual attribute based on the comparison (block 1010). In response, a model position indication 626 is generated that indicates a position corresponding to the visual attribute within the text-to-image model 116. The position is based on one or more nodes (block 1012), for example, one or more nodes or layers from which the text-to-image model 116 is copied.

[0107] Fig.11 1 is a flowchart depicting an algorithm 1100 as a step-by-step process in an example implementation of operations executable to achieve a result of forming an edited text-to-image model by editing a text-to-image model based on a position configured to affect the generation of visual attributes. In this example, an indication of a position within the text-to-image model 116 is received (e.g., model position indication 626, model position indication 726). The position is configured to affect the generation of visual attributes as part of a digital image by the text-to-image model 116 (block 1102), for example, to form a text-to-image model such as Figure 3 The picnic table shown.

[0108] The edited text-to-image model 124 is formed by the model editing system 122 by editing the text-to-image model 116 based on the position. When a subsequent digital image is generated by the edited text-to-image model, the edits cause changes to the visual attributes (block 1104), e.g., the ability to remove the generated picnic table, the ability to change the picnic table to another style of table, etc. Various other examples are also contemplated.

[0109] Example Systems and Devices

[0110] Fig.12 An example system is illustrated at 1200 that generally includes an example computing device 1202, which represents one or more computing systems and / or devices that implement the various techniques described herein. This is illustrated by including a model editing system 122. The computing device 1202 can be configured as, for example, a server of a service provider, a device associated with a client (e.g., a client device), a system on a chip, and / or any other suitable computing device or computing system.

[0111] As shown, the example computing device 1202 includes a processing device 1204, one or more computer-readable media 1206, and one or more I / O interfaces 1208 that are coupled to each other in communication. Although not shown, the computing device 1202 also includes a system bus or other data and command delivery system that couples various components to each other. The system bus can include any bus structure or a combination of different bus structures in different bus structures, such as a memory bus or a memory controller, a peripheral bus, a universal serial bus, and / or a processor or a local bus that utilizes any architecture in a variety of bus architectures. Various other examples are also considered, such as control lines and data lines.

[0112] Processing device 1204 represents the function of performing one or more operations using hardware. Therefore, processing device 1204 is illustrated as including hardware elements 1210 that can be configured as processors, functional blocks, etc. This includes being implemented in hardware as application-specific integrated circuits or other logic devices formed using one or more semiconductors. Hardware elements 1210 are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, a processor can be configured as a semiconductor and / or a transistor (e.g., an electronic integrated circuit (IC)). In this context, processor executable instructions are electronic executable instructions.

[0113] Computer-readable storage media 1206 is illustrated as including memory / storage 1212 that stores instructions executable to cause processing device 1204 to perform operations. Memory / storage 1212 represents memory / storage capacity associated with one or more computer-readable media. Memory / storage 1212 includes volatile media (such as random access memory (RAM)) and / or non-volatile media (such as read-only memory (ROM), flash memory, optical disks, magnetic disks, etc.). Memory / storage 1212 includes fixed media (e.g., RAM, ROM, fixed hard drive, etc.) and removable media (e.g., flash memory, removable hard drive, optical disk, etc.). Computer-readable media 1206 can be configured in various other ways as further described below.

[0114] (Multiple) input / output interface(s) 1208 represent functionality that allows a user to enter commands and information into computing device 1202, and also allows information to be presented to the user and / or other components or devices using various input / output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone, a scanner, touch functionality (e.g., a capacitive sensor or other sensor configured to detect physical touch), a camera (e.g., using visible or invisible wavelengths such as infrared frequencies to identify movement as gestures that do not involve touch), etc. Examples of output devices include a display device (e.g., a monitor or projector), a speaker, a printer, a network card, a tactile response device, etc. Thus, computing device 1202 can be configured in various ways as further described below to support user interaction.

[0115] Various techniques are described herein in the general context of software, hardware elements, or program modules. Typically, such modules include routines, programs, objects, elements, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The terms "module," "function," and "component" as used herein generally represent software, firmware, hardware, or a combination thereof. The features of the techniques described herein are platform-independent, meaning that the techniques can be configured on various commercial computing platforms with various processors.

[0116] An implementation of the described modules and techniques is stored on or transmitted across some form of computer-readable media. Computer-readable media includes various media accessed by computing device 1202. By way of example and not limitation, computer-readable media include "computer-readable storage media" and "computer-readable signal media."

[0117] "Computer-readable storage media" refers to media and / or devices that implement persistent and / or non-transient information storage (e.g., instructions executable by a processing device are stored thereon), as opposed to mere signal transmission, carrier waves, or signals themselves. Thus, computer-readable storage media refers to non-signal bearing media. Computer-readable storage media include hardware (such as volatile and non-volatile, removable and non-removable media and / or storage devices) implemented in a method or technology suitable for storage of information (such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data). Examples of computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, hard disk, cassette, magnetic tape, disk storage or other magnetic storage, or other storage device, tangible media, or an article of manufacture suitable for storing desired information and accessible by a computer.

[0118] "Computer-readable signal media" refers to signal-bearing media that is configured to transmit instructions to the hardware of the computing device 1202 (such as via a network). Signal media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal (such as a carrier wave, a data signal, or other transmission mechanism). Signal media also include any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a way as to encode information in the signal. By way of example and not limitation, communication media include wired media (such as a wired network or a direct wired connection), and wireless media (such as acoustic, RF, infrared, and other wireless media).

[0119] As previously described, hardware elements 1210 and computer-readable media 1206 represent modules, programmable device logic, and / or fixed device logic implemented in hardware that are employed in some embodiments to implement at least some aspects of the techniques described herein (such as, executing one or more instructions). Hardware includes integrated circuits or systems on chips, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other components implemented in silicon or other hardware (forms). In this context, hardware operates as a processing device that performs program tasks defined by the instructions and / or logic embodied in the hardware, as well as hardware used to store instructions for execution (e.g., the computer-readable storage media described previously).

[0120] Combinations of the above are also employed to implement the various techniques described herein. Thus, software, hardware, or executable modules are implemented as one or more instructions and / or logic embodied on some form of computer-readable storage media and / or embodied by one or more hardware elements 1210. Computing device 1202 is configured to implement specific instructions and / or specific functions corresponding to software modules and / or hardware modules. Thus, the implementation of a module executable by computing device 1202 as software is at least partially implemented in hardware (in the form), for example, by using computer-readable storage media and / or using hardware elements 1210 of processing device 1204. Instructions and / or functions are executable / operable by one or more articles (e.g., one or more computing devices 1202 and / or processing devices 1204) to implement the techniques, modules, and examples described herein.

[0121] The techniques described herein are supported by various configurations of computing device 1202 and are not limited to the specific examples of the techniques described herein. The functionality may also be implemented in whole or in part through the use of a distributed system, such as on a "cloud" 1214 via a platform 1216 as described below.

[0122] The cloud 1214 includes and / or represents a platform 1216 for resources 1218. The platform 1216 abstracts the underlying functionality of the hardware (e.g., server) resources and software resources of the cloud 1214. The resources 1218 include applications and / or data that can be utilized when performing computer processing on a server that is remote from the computing device 1202. The resources 1218 can also include services provided over the Internet and / or over a user network (such as a cellular network or a Wi-Fi network).

[0123] The platform 1216 abstracts resources and functionality to connect the computing device 1202 with other computing devices. The platform 1216 is also used to abstract the scaling of resources to provide a scaling level corresponding to the demand encountered for the resources 1218 implemented via the platform 1216. Therefore, in an interconnected device embodiment, the implementation of the functionality described herein may be distributed throughout the system 1200. For example, the functionality may be implemented partially on the computing device 1202 and via the platform 1216 that abstracts the functionality of the cloud 1214.

[0124] In implementation, platform 1216 employs a "machine learning model" configured to implement the techniques described herein. A machine learning model refers to a computer representation that can be adjusted (e.g., trained and retrained) based on an input to approximate an unknown function. In particular, the term machine learning model can include a model that uses an algorithm to learn from known data and make predictions about known data by analyzing training data to learn and relearn to generate outputs that reflect the patterns and properties of the training data. Examples of machine learning models include neural networks, convolutional neural networks (CNNs), long short-term memory (LSTM) neural networks, decision trees, and the like.

[0125] Although the invention has been described in language specific to structural features and / or methodological acts, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed invention.

Claims

1. A method comprising: identifying, by a processing device, locations within a text-to-image model that support a trained capability of the text-to-image model to generate visual attributes in a digital image; forming, by the processing device, an edited text-to-image model by editing the locations of the text-to-image model, the editing causing removal of the trained capabilities of the text-to-image model to generate the visual attributes in a subsequent digital image; as well as The subsequent digital image is generated by the processing device through the edited text-to-image model, in which the visual attribute is removed.

2. The method of claim 1, wherein the identifying is performed using causal mediation analysis by analyzing causal inference of changes in the response variable of the visual attribute after intervention on an intermediate variable of interest in the text-to-image model.

3. The method of claim 1 , wherein the location is identified as corresponding to one or more nodes included in at least one layer of the text-to-image model, and editing the location comprises: One or more activations associated with the one or more nodes included in the at least one layer are edited.

4. The method according to claim 1, wherein the identification comprises: generating a corrupted machine learning model based on the text-to-image model; generating a restored machine learning model by applying activations from nodes included in at least one layer of the text-to-image model to nodes of the corrupted machine learning model; detecting that a change has been made to edit the visual attribute by comparing a candidate digital image generated by the restored machine learning model to a digital image generated by the text-to-image model; as well as A model position indication is generated, the position indication indicating the position within the text-to-image model as corresponding to the node activation from the at least one layer.

5. The method of claim 4, wherein the corrupted machine learning model is generated by applying Gaussian noise to the text-to-object model.

6. The method of claim 5, wherein the Gaussian noise is applied to: A symmetric encoder-decoder machine learning model of the text-to-image model; or The text encoder machine learning model of the text-to-image model.

7. The method of claim 1, wherein said editing said text to said location within an image model comprises: At least one weight matrix associated with a layer of the text-to-image model is edited.

8. The method of claim 7, wherein the weight matrix is ​​a projection matrix associated with a self-attention layer of a text encoder machine learning model of the text-to-image model.

9. The method of claim 8, wherein the self-attention layer is the first layer of the text encoder machine learning model.

10. The method of claim 1, further comprising receiving, via a user interface, a visual attribute input identifying the visual attribute, and wherein the identifying is performed in response to the receiving.

11. The method of claim 10, wherein the visual attribute input indicates an object, style, color, viewpoint, action, or concept.

12. The method of claim 1, wherein the text-to-image model is configured to generate the digital image with the visual attribute based on a text input, and the edited text-to-image model is configured to generate the subsequent digital image with a change to the visual attribute based on the text input.

13. A method according to claim 1, wherein the position is configured to affect the generation of the visual attribute by the text-to-image model, and the forming is based on the position and causes a change to the visual attribute when the subsequent digital image is generated by the edited text-to-image model.

14. A system comprising: A corrupted model generation module implemented by a processing device, the corrupted model generation module being used to generate a corrupted machine learning model based on a text-to-image model; a recovery module implemented by the processing device, the recovery module to generate a recovered machine learning model by applying one or more activations from one or more nodes of the text-to-image model to the corrupted machine learning model; a candidate digital image generation module implemented by the processing device, the candidate digital image generation module being configured to generate a candidate digital image using the restored machine learning model; as well as A knowledge detection module implemented by the processing device, the knowledge detection module is configured to indicate a location within the text-to-image model corresponding to a visual attribute based on the candidate digital image.

15. A system according to claim 14, wherein the corrupted model generation module is configured to generate the corrupted machine learning model by applying Gaussian noise to a symmetric encoder-decoder machine learning model of the text-to-image model or a text encoder machine learning model of the text-to-image model.

16. The system according to claim 14 also includes a concept editing module implemented by the processing device, and the concept editing module is used to form an edited text-to-image model by editing the positions within the text-to-image model, and the editing causes changes to the visual attributes when a subsequent digital image is generated by the edited text-to-image model.

17. The system of claim 16, wherein said editing said text to said location within an image model comprises: At least one weight matrix associated with a layer of the text-to-image model is edited.

18. The system of claim 17, wherein the weight matrix is ​​a projection matrix associated with a self-attention layer of a text encoder machine learning model of the text-to-image model.

19. One or more computer-readable media storing instructions that, in response to execution by a processing device, cause the processing device to perform operations comprising: receiving an indication of a location within a text-to-image model, the location configured to affect generation by the text-to-image model of a visual attribute that is part of a digital image; as well as An edited text-to-image model is formed by editing the text-to-image model based on the location, the editing causing a change to the visual attribute when a subsequent digital image is generated by the edited text-to-image model.

20. The one or more computer-readable media of claim 19, wherein said editing said text to said location within an image model comprises: At least one weight matrix associated with a layer of the text-to-image model is edited.

Images