Forgetting in pre-trained generative machine learning models

Abstract

A technique for forgetting concepts when using a pre-trained generative machine learning model is described. The process receives a description of a concept to be forgotten when using the pre-trained generative machine learning model. The pre-trained generative machine learning model is used to process negative and positive cues to generate associated activation volume maps. A set of conditions is identified to distinguish between activation volume maps associated with negative cues and activation volume maps associated with positive cues. A model adapter is generated to use a different set of model parameters when the processing of cues by the pre-trained generative machine learning model satisfies the set of conditions.

Description

Background Technology

[0001] Recent advances in machine learning have spurred the development of large, pre-trained generative models (sometimes called base models). These pre-trained models can serve as a starting point for task-specific customization. While initially trained on massive datasets using significant computational resources over hours, days, weeks, or even months, these pre-trained models have demonstrated exceptional performance. By fine-tuning these models, users can adapt the model's output to a given application with only a fraction of the initial training cost. Attached Figure Description

[0002] Various examples according to this disclosure will be described with reference to the following figures.

[0003] Figure 1 This is a diagram illustrating an example environment for a model unlearning training service, based on some examples.

[0004] Figure 2 This is a diagram illustrating an example model architecture, including a model adapter, based on some examples.

[0005] Figure 3 This is a diagram illustrating a sample graphical user interface for a model forgetting training service based on some examples.

[0006] Figure 4 This is a graph generated based on an exemplary training dataset of some examples.

[0007] Figure 5 The plot is shown to illustrate the fitting of an exemplary adapter detector based on some examples.

[0008] Figure 6A , Figure 6B and Figure 6C A diagram illustrating the parameter update strategy based on some exemplary models.

[0009] Figure 7 This is a flowchart illustrating the operations of a method for generating a pre-trained generative model adapter, based on some examples.

[0010] Figure 8 The example cloud provider's network environment is shown based on some examples.

[0011] Figure 9 This is a block diagram of an example cloud provider network that provides storage services and hardware virtualization services to customers, based on some examples.

[0012] Figure 10 A block diagram illustrating an example computing device that can be used in some examples. Detailed Implementation

[0013] This disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for forgetting in pre-trained generative machine learning models. According to some examples, a model forgetting training service generated by a cloud provider network generates an adapter that modifies the behavior of a pre-trained generative model during inference operations, causing the pre-trained model to "forget" or "discard" how to generate outputs that include specific concepts. More specifically, the adapter may include a detector that operates to enable or disable the application of updated model parameters that, when applied, perturb the model to prevent the generation of outputs that include the "forgotten" concepts. As used herein, "forgetting" (also referred to as "intentional forgetting") refers to modifying the parameters of a pre-trained generative AI model in a way that biases it to prevent the generation of concepts during inference operations.

[0014] Generative AI systems are typically used to create new content in one or more modalities from inputs received from a user across a variety of modalities (e.g., text, images, audio, video). Generative AI systems often rely on neural network techniques. Neural networks are powerful tools for modeling complex patterns and relationships in data. The "neuron" in a neural network is the basic unit of computation. Such a neuron typically includes inputs (one or more outputs from previous layers of the network), weights representing the strength of the connections between these inputs, biases that can shift the weighted sum of the inputs, and an activation function that introduces non-linearity between the inputs and output (e.g., if the weighted sum with bias is less than zero, the output is zero; otherwise, the output is the weighted sum with bias).

[0015] Pre-trained generative AI models typically consist of: a model definition, which specifies how the neural network is constructed; and model parameters, which specify the parameters the network learns during training (e.g., weights, biases, and other parameters) that enable the model to perform useful functions during inference—for example, prompting a generative image model to generate an image of a house, producing an image of a house, and not something else. Some generative AI systems employ built-in content moderation modules to block or filter inappropriate or offensive content after it has been generated. However, such functionality is often general (e.g., content moderation is inappropriate in the workplace) and does not meet the needs and definitions of content moderation for individual users.

[0016] As detailed in this article, forgetting causes previously trained neural networks to forget previously learned concepts by modifying how the model operates during the processing of cues in response to generated outputs. Here, "concept" refers to a modality-specific object that the pre-trained model was previously able to generate. For example, in language generation, a concept can be a name, place, word, phrase, slang, etc.; in image or video generation, a concept can be an object, object category, style, brand, trademark, etc.; or in audio generation, a concept can be a melody, instrument, speech, music genre, etc. It is important to note that a concept can represent multiple things, such as object categories, word sets, speech sets, etc.

[0017] The model forgetting training service allows users to easily implement customized content moderation within their generative AI systems. At a higher level, users can identify concepts to be forgotten, for example, by providing a text description, selecting from a list of various topics or keywords, or providing a labeled sample of output, with or without the concept. In some examples, the service allows users to customize the degree to which concepts are forgotten. Based on concept identification, the service can generate or expand sample datasets to train the model in various ways. The sample dataset can include inputs that enable the model to generate outputs that include, exclude, or include concepts similar to or adjacent to the unwanted concepts. The service then uses the sample dataset to train the model, capturing model behavior (e.g., neuronal activation) as it processes samples. The service can use the captured data to identify patterns or "fingerprints" of the model that statistically indicate when the model will generate concepts to be forgotten. Using this pattern, the service can define a detector that selectively applies model parameter updates. Applying parameter updates can guide the remaining forward pass operations to bypass concepts and pass through the model. Updated model parameters can be set or computed. For example, the weights for activations in downstream model layers can be set to zero, or adjusted to blur activations (e.g., scaled based on a noise function), or computations can be performed to bias the model toward another related concept. The service allows users to customize pre-trained generative models for a fraction of the computation time and cost that would otherwise be required when retraining the model using datasets that exclude concepts to be forgotten or negatively reinforce those concepts.

[0018] The techniques described in this article have a wide range of use cases. For example, advertisers customizing generative image models might want to remove logos from various companies in a highly competitive neighborhood. Such logos could inadvertently appear in the output of a baseline, pre-trained generative AI model because these companies have made their product representations in the training data used to initially train the pre-trained model. As another example, a medical school might want to generate anatomically accurate images of the heart, rather than emojis or emoticons that the pre-trained model may have learned. It's important to note that "negative cues" are techniques that can be used to bias generative models away from undesirable outputs. For example, the cue "Generate an image of a heart instead of a heart-shaped emoji" could be considered a negative cue. Therefore, the responsibility for using negative cues to circumvent the aforementioned concepts (which may be undesirable for individuals or organizations deploying models to a wider audience) falls on the end user.

[0019] Figure 1 This diagram illustrates an example environment for a model forgetting training service, based on some examples. In this example, the model forgetting training service 120 is provided as part of a machine learning service 110 of a cloud provider network. The machine learning service 110 may be implemented as hardware, a software application (e.g., a computer program), or a combination of both, utilizing the computing resources of the cloud provider network 100. Other machine learning services may include, for example, a model hosting service 130 and a model training service (not shown). Such model training and hosting services allow users or customers of the cloud provider network to train and host machine learning models using the resources of the cloud provider network.

[0020] Cloud provider network 100 (also referred to herein as provider network, service provider network, etc.) provides users with the ability to use one or more of various types of compute-related resources, such as compute resources (e.g., executing virtual machine (VM) instances and / or containers, executing batch jobs, executing code without provisioning servers), data / storage resources (e.g., object storage, block-level storage, data archive storage, databases and database tables, etc.), network-related resources (e.g., configuring virtual networks (including compute resource groups), content delivery networks (CDNs), domain name services (DNS)), application resources (e.g., databases, application build / deployment services), access policies or roles, identity policies or roles, machine images, routers, and other data processing resources, etc. These and other compute resources may be provided as services, such as compute services 150 that can execute compute instances (e.g., virtual machines, containers, etc.), storage services 160 that can provide object-level or block-level data services, databases, etc. Users (or “customers”) of cloud provider network 100 may use one or more user accounts associated with a customer account, but these terms may be used interchangeably to some extent depending on the use case. Cloud provider networks are sometimes “multi-tenant” because they can use the same physical computing infrastructure to serve multiple different customers.

[0021] Users can interact with cloud provider network 100 across one or more intermediate networks (e.g., the Internet) via one or more interfaces, such as through application programming interface (API) calls, via a console implemented as a website or application, etc. An API refers to an interface and / or communication protocol between a client and a server, such that if a client issues a request in a predefined format, the client should receive a response in a specific format or initiate a defined action. In a cloud provider network context, an API provides a gateway to enable customers to access cloud infrastructure by allowing customers to obtain data from or initiate actions within the cloud provider network, enabling the development of applications that interact with resources and services hosted within the cloud provider network. APIs can also enable different services within the cloud provider network to exchange data with each other. Interfaces can be part of the control plane of cloud provider network 100 or act as the front end of the control plane, which includes "back-end" services that support and implement services that can be provided more directly to customers.

[0022] Therefore, a cloud provider network (or simply "the cloud") generally refers to a large pool of accessible virtualized computing resources, such as compute, storage, and networking resources, applications, and services. The cloud provides convenient, on-demand network access to a shared pool of configurable computing resources that can be programmatically provisioned and released in response to client commands. These resources can be dynamically provisioned and reconfigured to adapt to variable loads. Thus, cloud computing can be viewed as applications delivered as a service over publicly accessible networks (e.g., the Internet, cellular networks) and the hardware and software in the cloud provider's data centers that provide these services.

[0023] To provide these and other computing resource services, cloud provider network 100 typically relies on virtualization technology. For example, virtualization technology can provide users with the ability to control or use computing resources (e.g., “computing instances,” such as: VMs using a guest operating system (O / S) that may or may not operate on top of the underlying host O / S; containers that may or may not operate within a VM; computing instances that can execute on “bare metal” hardware without an underlying hypervisor), where one or more computing resources can be implemented using a single electronic device. Thus, users can directly use computing resources hosted by the provider network (e.g., provided by a hardware virtualization service) to perform various computing tasks. Alternatively, users can indirectly use computing resources by submitting (e.g., via an on-demand code execution service) code for execution by the provider network, which then uses one or more computing resources to execute the code; typically, the user does not have any control or knowledge of the underlying computing instances involved.

[0024] As described herein, a type of service that a provider network can provide may be termed a "managed computing service," which executes code or provides computing resources to its users under a managed configuration. Examples of managed computing services include, for instance, hardware virtualization services, on-demand code execution services, container services, etc. Such services may be provided as computing service 150. Hardware virtualization services (referred to in various implementations as elastic computing services, virtual machine services, compute cloud services, compute engines, or cloud computing services) enable users of provider network 100 to provision and manage computing resources such as virtual machine instances.

[0025] As described herein, the Model Forget Training Service 120 allows users to leverage pre-trained generative AI / machine learning models (or simply pre-trained generative models) while providing them with additional control over the generated output. Specifically, the Model Forget Training Service 120 generates adapters to cause the pre-trained generative model to "forget" or "recall" concepts identified by the user. Compared to other fine-tuning techniques or retraining generative models when the data does not include unwanted concepts or using negative reinforcement for unwanted concepts, this type of adapter achieves finer control over the model output with relatively low computational overhead.

[0026] exist Figure 1 In the diagram, the circles marked (1) to (4) illustrate an example process in which a user utilizes the model forgetting training service 120 to customize a pre-trained generative model using the concept of forgetting.

[0027] As shown in circle (1), a user (not shown) operating electronic device 105 (which executes software applications such as web browsers, standalone applications, integrated development environments (IDEs), etc.) interacts with machine learning service 110. In particular, electronic device 105 can send a request to model forgetting training service 120 to initiate the creation of an adapter that allows a pre-trained generative model to forget or forget concepts.

[0028] In some examples, the software application may provide an interface (e.g., a web-based interface) for the user to use to invoke the creation of the adapter. For example, Figure 3 This diagram illustrates an example graphical user interface (GUI) for a model forgetting training service based on some examples. The graphical user interface (GUI) 300 can be provided through the front end of the machine learning service 110 of the cloud provider network 100, for example, via a console-like application or a standalone application provided through a browser.

[0029] GUI 300 includes several input fields that allow the user to invoke the model forgetting training service 120. GUI 300 includes field 305 to provide a selection of a base pre-trained generative model. Here, GUI 300 includes a drop-down menu from which a class of models (e.g., image generation, language generation, etc.) can be selected, as well as a subordinate drop-down menu from which available models within that class can be selected.

[0030] The GUI 300 includes a text input field 315 where the user can provide a textual description of a concept to be forgotten. And continuing here... Figure 4 The training dataset was generated, and the user has identified "dog" as a concept to be forgotten. In other examples, the GUI can provide the user with a list of topics or keywords from which to select concepts to be forgotten.

[0031] GUI 300 includes a column 325 where the user can identify existing training data (if any). In the image modality, the existing training data can be labeled images. Here, the user can provide negative samples (e.g., samples that include the concept to be forgotten) and positive samples (e.g., samples that do not include the concept to be forgotten).

[0032] GUI 300 includes column 335, in which the user can specify the computing resources used during the forgetting period, such as virtual machines (whether or not they have specific hardware acceleration) executed by the underlying host system.

[0033] The GUI 300 includes a selector 345 in which the user can specify the degree to which the model forgetting training service 120 should attempt to modify the pre-trained generative model to forget concepts. Since neural network-based models are largely probabilistic, the model forgetting training service 120 can provide such selections because different degrees of review can affect the model's performance relative to other neighboring concepts.

[0034] The GUI 300 includes a button 355 to allow the user to initiate adapter creation via the model forgetting training service 120. It should be noted that the electronic device displaying the GUI 300 can aggregate input specified by the GUI and send it as one or more messages to the model forgetting training service 120 to indicate a request.

[0035] In some examples, the GUI can provide user-provided feedback during the development of a training dataset for fitting a model adapter to a concept. Here, GUI 300 includes a box 365 labeled “Training Dataset Builder” in the lower section. The model forgetting training service 120 can return representative samples based on prompts developed to expand the concept to be forgotten, soliciting user feedback on whether the image generated by the prompts includes the concept to be forgotten. In some examples, the model forgetting training service 120 can utilize visual language models such as CLIP to provide more targeted questions. For example, using CLIP, the model forgetting training service 120 can identify a set of objects in an image and provide the user with the option to select one or more objects as the concept to be forgotten. In either case, the user can submit feedback, which the model forgetting training service 120 can use to classify certain prompts as negative or positive. In some examples, feedback can be performed using visual prompts instead of a “yes / no” selector as shown. The GUI can display the image generated by the prompts and allow the user to select a region in the image that includes the concept to be forgotten. For example, the model forgetting training service 120 can pre-identify (e.g., using CLIP) objects in an image, which can then be selected by the user. As another example, the user may be able to select a region of an image, and then the model forgetting training service 120 can identify (e.g., using CLIP) objects within the selected region to determine concepts to be forgotten.

[0036] Return to Figure 1 The model forgetting training service 120 includes a pre-trained generative model 121, training data 125, and a model adapter 126. Model 121 can be one of many other models provided via machine learning service 110. Similarly, adapter 126 can be one of many other adapters generated for the same model 121, for other models, for other concepts in the presence of the same model 121, or for the same or other concepts in the presence of other models, etc., and training data 125 can be a set of many training datasets. The model, training datasets, and adapters can be stored using storage service 160.

[0037] In some examples, the pre-trained generative model (such as model 121) includes model definition 122 and model parameters 123. The exemplary pre-trained model definition 122 can be specified as human-readable code (e.g., Python) or machine-readable code (e.g., compiled code) specifying the flow of data through the model. For example, an image generation model may include a front-end encoder to encode text-based cues into an embedding space to feed into a diffusion model that is fed to generate output. Model definition 122 may reference pre-trained parameters that have been learned during a previous training process (typically over days, weeks, months, etc.).

[0038] Figure 1 The lower part illustrates the different behaviors of the pre-trained generative model in generating anatomically accurate images of the heart with and without a trained adapter, using earlier examples. Initially, the pre-trained generative model 121 (including model definition 122 and model parameters 123) is executed by one or more computation instances 151. A user (who could be the user who submitted the earlier adapter creation request or another user) can submit prompts 166 to model 121. Assuming that model 121 is trained on a sufficiently diverse dataset (e.g., collected from the internet), the model may favor cartoonish images of the heart, thus producing output 167 without further refinement through prompts.

[0039] Upon receiving a request for a generative adapter, the model forgetting training service 120 generates a model adapter, as indicated by circle (2). In short, the generative adapter component includes constructing or expanding the training dataset (if sufficient samples are not provided in the request, if any), fitting the detector to the identified concept to be forgotten, and generating parameter updates to apply to the pre-trained model parameters (typically weight updates to apply to pre-trained weights). These parameter updates can then be applied during inference (e.g., added to existing pre-trained model parameters) or used to generate a separate set of model parameters used to adjust the inference data flow through individual layers or neurons. References Figure 4 Provides additional details regarding the generation of the dataset. References Figure 5 Additional details regarding the detector fitting are provided. Refer to Figure 6 for further details regarding the parameter update generation.

[0040] In some examples, model adapters such as model adapter 126 include model definition update 127 and model parameter update 128. Model definition update 127 may represent a change to model definition 122 to include detector operations and resulting conditional weight updates, and may be provided as one or more lines of code that can be incorporated into definition 122. Model definition update 127 may reference parameter 123 and update parameter 128, conditionally combining parameter 123 with update parameter 128 when a concept to be forgotten is detected. Therefore, parameter 123 may remain unchanged to enable subsequent hint processing.

[0041] In some examples, model adapters such as model adapter 126 include model definition update 127 and pre-computed model parameters for an alternative data stream to the original model definition. The pre-computed model parameters can be a set of parameters computed based on the original pre-trained model parameters. Upon detecting a concept to be forgotten, model definition update 127 can cause the intermediate output of the pre-trained model to use the pre-computed model parameters instead of the original pre-trained model parameters, effectively bypassing certain neurons or layers of the pre-trained model and routing data processing through alternative (or parallel) neurons or layers of the model adapter, then routing the outputs of these neurons or layers back to downstream neurons or layers of the pre-trained model. Whether dynamically updating the pre-trained model parameters or rerouting data processing through alternative neurons / layers with different parameters, the model adapter causes at least a portion of the processing of the cue to use a different set of parameters than the set of parameters of the pre-trained model.

[0042] In some examples, once the model forgetting training service 120 generates an adapter, it can return the adapter to the requester (in this case, the electronic device 105, as indicated by circle (3)). Providing the adapter allows the user who initiated the request to deploy the model with the adapter in other environments.

[0043] In some examples, machine learning service 110 may receive requests (not shown) to host an adapted pre-trained model via model hosting service 130. An adapted pre-trained model refers to a pre-trained model that includes an adapter generated for forgetting concepts. As shown in circle (4), the adapted pre-trained generative model 121A is executed by one or more computation instances 153. In this example, the adapted model 121A includes model definition 122, model parameters 123, model definition update 123, and model parameter update 128. Note that in practice, model definition update 123 can be applied to model definition 122, producing a composite model definition different from the original pre-trained model's model definition. Now, assuming the model forgetting training service 120 generates an adapter to forget a cartoonish heart shape, when a user submits a hint 166 to the adapted model 121A, the model can output an image 168 that is a reasonable anatomical approximation of the heart without further hint refinement. Reference Figure 2 Additional details of the exemplary architecture of the adapted, pre-trained model are shown and described.

[0044] It should be noted that although the operations used to generate model adapters are presented and described relative to Service 120, they can also be performed in other non-cloud or non-service environments.

[0045] Furthermore, it should be noted that although forgetting is presented here and in the following figures in the context of text-to-image generation, forgetting can be easily extended to the generation of other types of modalities (e.g., text, audio, etc.).

[0046] Figure 2 A diagram illustrating an example model architecture including a model adapter is provided. As shown, the pre-trained model 121 comprises multiple layers 290, including layers 291 and 292. During inference, inputs such as text cues are received and processed by the model to generate outputs such as images. When processing the input, the outputs from upstream layers are passed to one or more "downstream" layers (feedback loops, recursive mechanisms, or other nonlinearities may exist, not shown). The adapter includes a detector module that monitors the outputs (typically activations) of one or more layers and, based on these values, controls the use of different weights (or more generally, parameters) in the continuous forward pass-through processing of the cues.

[0047] In the illustrated example adapter 126, the model adapter 126 monitors activations from layer 291 via detector module 227A. When a pattern or "fingerprint" of the concept to be forgotten appears in the monitored activations, detector module 227A activates control signal 228 to cause the continuous forward pass processing of the input to use different parameters, thereby perturbing the downstream data flow and avoiding the concept. The detector can utilize various techniques to perform the detection. For example, the detector can evaluate a set of Boolean conditions to determine whether the activation levels of the monitored set of neurons match a pattern associated with the concept to be forgotten (e.g., neuron A is above a certain level, neuron B is below a certain level, and so on).

[0048] It should be noted that different parameters can be used to influence the forward pass processing in various ways. In the case of example adapter 126, control signal 228 causes weight update 229 to be summed to the pre-trained weights of layer 292. When the detector does not detect a pattern, control signal 228 remains inactive, and model 121 processes the cue normally (e.g., without applying parameter updates).

[0049] In other examples, parameter updates can be a different set of parameters, such that instead of adding the parameter updates to the original parameters of the pre-trained model and using the same layer to process the upstream output, control signals cause the output from a layer to be rerouted through parallel paths (e.g., neurons, layers, etc.) with alternative model parameters, thus bypassing the default model path with the original pre-trained parameters. This is illustrated by the dashed example model adapter 226. Model adapter 226 monitors activations from layer 291 via detector module 227B. As before, when a pattern of a concept to be forgotten appears in the monitored activations, detector module 227B activates control signal 228 to switch the continuous forward pass processing to layer 292A via switchers 231, 233. Layer 292A includes different parameters than layer 292, causing perturbation of the downstream data flow, bypassing the concept.

[0050] While the examples above use layers and weights, other examples can use different parameters (e.g., biases, parameterized activation functions, etc.) or granularity (e.g., a single neuron instead of a layer) to perturb the model and avoid concepts to be forgotten.

[0051] In some examples, detector module 227 may monitor activations from multiple layers (e.g., from layer 291 and one or more neighboring or non-neighboring upstream layers) as indicated. Similarly, weight updates 229 (or bypasses) may be applied to multiple layers (e.g., to pre-trained weights of layer 292 and one or more neighboring or non-neighboring downstream layers) as indicated. In some examples, one or more other layers may reside between the activations of the layers monitored by detector module 227 and the use of different weights in downstream layers.

[0052] It should be noted that multiple adapters can be applied to a single model for the same or different concepts to be forgotten. In some examples, a hierarchical approach can be used for a single concept, where the first detector observes activations in the model earlier than the second detector, and either the first or second detector can inject weight updates into downstream layers if it detects a representation of the concept to be forgotten during inference. In other examples, multiple adapters can be generated for different concepts and applied to the same model.

[0053] Figure 4 To illustrate the graphs generated from an exemplary training dataset of some examples, the model forgetting training service 120 includes a cue extension module 450 to generate a set of cues for processing with a pre-trained generative model. These cues can later be used to identify patterns representing concepts to be forgotten. Figure 4 In the diagram, circles labeled (1) through (3) illustrate an example process for generating the training dataset. It should be noted that the examples shown pertain to generating a training dataset for a text-to-image generative model. Other types of models may include different modal cues and outputs (e.g., image, audio samples, etc.). The development of the training dataset described herein can be tailored to these other modalities.

[0054] At circle (1), the model forgetting training service 120 receives an instruction for the concept to be forgotten (in this example, the text description "dog"). At circle (2), the prompting extension module 450 utilizes another pre-trained language model 510 executed by one or more computational instances 411. The pre-trained language model 510 (which may be a large language model (LLM)) can model semantic relationships between words. In the case of querying model 510, the prompting extension module 450 can identify additional dimensions across which it develops a set of prompts. For example, the prompting extension module 450 can query model 510 for variables such as other objects that look like the identified concept or settings in which the identified concept can be displayed. Such queries can be predefined for modalities such as related concepts, scenes, views, etc. The prompting extension module 450 can classify prompts as negative prompts 426 or positive prompts 427 (according to the convention used herein, a negative prompt is a sample that includes the concept to be forgotten or an output that may cause the concept to be forgotten). For example, the prompt extension module 450 can classify prompts based on whether they include a concept to be forgotten, thereby classifying prompts that mention dogs or dog-related objects as negative prompts 426.

[0055] In some examples, model 510 can return an indication of the strength of the relationship between the query and the returned results. The hint extension module 450 can use a threshold to determine whether to classify the received hint as negative or positive. For example, the model can return a value indicating that dogs are more correlated with wolves than with cats. The model forgetting training service 120 can classify hints with low relevance (e.g., below the threshold) as positive hints and hints with high relevance (e.g., above the threshold) as negative hints.

[0056] Although not shown, the training dataset generation may include a model forgetting training service 120 that uses sample cues generated by cue expansion to prompt a pre-trained generative model. The model forgetting training service 120 may then provide the generated output (e.g., via box 365 of GUI 300) to a user to solicit user feedback on whether the cue generated an image that includes or does not include the concept to be forgotten. Based on the user feedback, the model forgetting training service 120 may classify the cue associated with the presented image into its corresponding negative sample group 426 or positive sample group 427.

[0057] In some examples, the model forgetting training service 120 may select prompts for soliciting user feedback based on prompts from the response of model 510 (which have a very strong relationship to the concept (e.g., above a second higher threshold)). Continuing with the previous example, the model forgetting training service 120 may obtain images generated from such prompts to further refine the user request (e.g., using images to prompt the user, said images being generated using wolf-based prompts instead of cat-based prompts based on the former's relevance exceeding a threshold while the latter's relevance does not).

[0058] Figure 5 This is a diagram illustrating the fitting of an exemplary adapter detector based on some examples. The detector operates to predict whether the final model output will include the concept to be forgotten during cue processing via a pre-trained generative model. If so, the detector perturbs the model weights with the adapter to reduce the likelihood of the concept appearing in the output. To identify a set of conditions for identifying the concept to be forgotten during the intermediate processing phase of the model (and thus when to perturb or apply updated weights), the model forgetting training service 120 relies on statistical techniques to extract patterns or fingerprints of the concepts based on positive and negative samples.

[0059] exist Figure 5In the diagram, circles labeled (1) through (3) illustrate an example process for identifying detector strips within them. A machine learning model can generally be represented as an activation volume conceptually represented by activation layer 505. Different inputs can produce different activation volume maps. As used herein, an activation volume map refers to all or part of the model's activation volume captured during the processing of input cues. Activation volumes include values ​​from the outputs of one or more layers of a neural network.

[0060] At circle (1), the model forgetting training service 120 uses a pre-trained generative model to process training data 125 (negative and positive cues). In some examples, the entire activation volume can be captured for each cue, but in other cases, a subset can be stored, which is predetermined based on the model definition.

[0061] At circle (2), the model forgetting training service 120 captures an activation volume map 525 associated with each cue in the training data. Each map is associated with a cue in the training data 125, so each map represents either a positive cue or a negative cue. The process of evaluating cue and capturing activation volume maps can be performed by one or more computational instances (not shown).

[0062] At circle (3), the statistical profiling engine 530 identifies which neurons contain significant information for negative examples (concept presence) and (optionally) significant information for positive examples (concept absence) by comparing the distribution of neuron activation maps for positive samples with the distribution of neuron activation maps for negative samples. For example, the statistical profiling engine 530 can identify patterns of neuron activation and optionally identify levels that satisfy one or more detection requirements (e.g., false negative rate below a certain threshold, false positive rate below a certain threshold, true negative rate above a certain threshold, and / or true negative rate above a certain threshold). Various statistical techniques can be used to identify patterns. In some examples, dimensionality reduction techniques such as principal component analysis can be used to reduce the complexity of the analysis. For example, the activation distribution for a sample can be reduced to a lower dimension indicating neurons that primarily contribute to the appearance of the concept to be forgotten in negative samples rather than neurons that are fired from positive samples (e.g., from an activation volume of 10,000 activations to N activations).

[0063] Once a set of neuronal activations associated with the concept to be forgotten is identified, a set of conditions 535 for the indication pattern can be formalized for evaluation by the detector during subsequent passes of the adapted, pre-trained generative model (e.g., by detector 227 of model 121A). A set of example conditions may include tests evaluating true or false, where all conditions must be satisfied to infer a detection result (e.g., neuron activation A below 0.0, neuron activation B above 12, neuron activation C between 2 and 4, etc.). In some examples, activation values ​​can be normalized to "0" or "1" and evaluated using an XOR operation against a mask that has an associated "0" for neurons indicating the concept when inactive and an associated "1" for neurons indicating the concept when active. In some examples, the detector can be calibrated using a statistical profile of a set of neurons for the concept to be forgotten, such that the detector can induce the use of different parameters when subsequent forward passes produce data points within this profile.

[0064] In some examples, the search space for the statistical profiling engine 530 can be constrained using gradient basis methods. For instance, during the processing of cues in the training data 125, model output 575 can be stored. The model output can be evaluated by an object recognition model 580, which can identify and locate concepts, such as concepts to be forgotten, via bounding boxes. Based on the presence and location of concepts in the output, a gradient-weighted class activation map, or GradCAM engine 585, can compute gradients back through the model to find the activation volume regions most sensitive to the concepts. The GradCAM engine 585 can provide the statistical profiling engine 530 with the identification of activation volume regions, from which the statistical profiling engine can then exclude unidentified regions from its profiling.

[0065] Figure 6 is a diagram illustrating the exemplary model parameter update strategy based on some examples. Figures 6A to 6CEach of these illustrates a simple neuron structure of a neural network. A neuron comprises inputs from various upstream activations, which are scaled and summed by weights W, adjusted by a bias B, and then fed into another activation function. In this example, the summed topmost input is scaled by weight W1, the middle input by weight W2, and the bottom input (indicated as coming from a neuron signaling the presence of a concept) by weight W3, where weights W1, W2, and W3 are pre-trained weights associated with a generative model. Since the neuron activations used to identify patterns associated with a concept are strongly correlated with that concept, one way to update the weights is to update the weights applied to these activations associated with detection (here, the bottom input). For example, if activation of neuron X from layer Y triggers detection, the input from neuron X to neurons in layer Y+1 can be adjusted via changes to the corresponding weights.

[0066] Figure 6A This demonstrates how updating weights to have magnitudes equal to but opposite signs of the pre-trained model weights can zero out concept-associated activations. This zeroing out of weights prevents the propagation of concept-associated activations. Figure 6B This demonstrates how adding noise (e.g., random values ​​bounded by + / -X) can "blur" activations associated with a concept. In this way, randomizing the noise helps guide the model toward a different outcome (and avoid patterns that trigger the detector). Figure 6C The diagram demonstrates adjusting weights according to a mapping function that activates guidance toward profiling in unrelated concepts (e.g., cats instead of dogs). In such cases, patterns for permissible concepts can be developed via a statistical profiling engine 530. The mapping function can adjust the weights such that weight-scaled input values ​​produce inputs to neurons that match the inputs to permissible concepts. This mapping function can be predetermined during statistical profiling of negative and positive samples. For example, if the detected input is -0.8 and the permissible concept is 0.7, the mapping function can return a weight of -0.875 for the neuron (e.g., input -0.8 multiplied by -0.875 equals 0.7). Regardless of the chosen strategy, the model forgetting training service 120 can store the weight updates as weight updates 229 for use by the adapter. The example shown envisions updating existing weights W3. As indicated elsewhere in this document, other embodiments can reroute the data flow through the model so that existing weights remain unchanged.

[0067] It should be noted that when generating weight updates 229, the model forgetting training service 120 can scale the weight updates according to the degree of forgotten concepts specified by the user (e.g., via GUI element 345). For example, if the scaling factor is 0.5, then... Figure 6A The weight update in [the data] will be -0.5. W3.

[0068] Figure 7 The flowchart illustrates operation 700 of a method for generating a pre-trained generative model adapter according to some examples. Some or all of operation 700 (or other processes, variations, and / or combinations thereof described herein) execute under the control of one or more computing devices configured with executable instructions and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more application programs) that executes collectively on one or more processors. The code is stored on a computer-readable storage medium, for example, in the form of a computer program including instructions executable by one or more processors. The computer-readable storage medium is non-transitory. In some examples, one or more (or all) of operation 700 is performed by a machine learning service 110 (such as model forgetting training service 120 in other figures).

[0069] Operation 700 includes, at box 702, receiving a description of a concept to be forgotten when using a pre-trained generative machine learning model. The description can be provided based on the input modality from an untrained model. For example, a text description can be provided for a text-to-text model or a text-to-image model, an image description can be provided for an image-to-image model or an image-to-text model, and so on.

[0070] Operation 700 further includes, at box 704, using a pre-trained generative machine learning model to process negative and positive cues to generate associated activation volume maps. As described herein, activation volumes represent intermediate values ​​of the model during cue processing. These intermediate values ​​from the forward pass of these cue processing are stored as mappings (e.g., cue A has a corresponding mapping A, cue B has a corresponding mapping B, and so on).

[0071] Operation 700 further includes, at box 706, identifying a set of conditions to distinguish between activation volume maps associated with negative cues and activation volume maps associated with positive cues. As described herein, the set of conditions can be a set of outputs or activations to be monitored and associated activation levels (e.g., above or below a threshold). Statistical techniques can be used to identify a set of conditions to identify feature patterns in negative samples that can be distinguished from positive samples (within a certain confidence level).

[0072] Operation 700 also includes, at box 708, generating a model adapter to use a different set of model parameters when processing of the cues by the pre-trained generative machine learning model satisfies a set of conditions. As described herein, various techniques can be used to adapt the pre-trained generative machine learning model. Detectors can be used to generate model definition updates to modify one or more parts of the model definition (such as conditional statements in software), and different weights can be used based on the detection of patterns. Some examples may involve in-situ temporary updates to the pre-trained weights to perturb the output. Other examples may involve rerouting the data flow through the model to apply different weights.

[0073] Figure 8 An example provider network (or “service provider system”) environment is illustrated based on some examples. Provider network 800 may provide resource virtualization to customers via one or more virtualization services 810, which allow customers to purchase, lease, or otherwise obtain instances 812 of virtualized resources (including, but not limited to, compute and storage resources) implemented on devices within one or more provider networks in one or more data centers. A local Internet Protocol (IP) address 816 may be associated with resource instance 812; the local IP address is the internal network address of resource instance 812 on provider network 800. In some examples, provider network 800 may also provide customers with public IP addresses 814 and / or ranges of public IP addresses (e.g., Internet Protocol version 4 (IPv4) or Internet Protocol version 6 (IPv6) addresses) that can be obtained from provider 800.

[0074] Typically, provider network 800 can, via virtualization service 810, allow service provider customers (e.g., customers of one or more customer networks 850A to 850C (or “client networks”) whose operations include one or more customer devices 852) to dynamically associate at least some public IP addresses 814 assigned to the customer with specific resource instances 812 assigned to the customer. Provider network 800 can also allow customers to remap public IP addresses 814 previously mapped to one virtualized computing resource instance 812 assigned to the customer to another virtualized computing resource instance 812 also assigned to the customer. Using the virtualized computing resource instances 812 and public IP addresses 814 provided by the service provider, customers of the service provider (such as operators of customer networks 850A to 850C) can implement, for example, customer-specific applications and present these applications on an intermediate network 840 such as the Internet. Then, other network entities 820 on intermediate network 840 can generate traffic destined for public IP address 814 published by customer networks 850A to 850C; the traffic is routed to the service provider data center, and at the data center, it is routed via the network layer to the local IP address 816 of the virtualized computing resource instance 812 currently mapped to the destination public IP address 814. Similarly, response traffic from virtualized computing resource instance 812 can be routed back to source entity 820 on intermediate network 840 via the network layer.

[0075] As used herein, a local IP address refers to an internal or “private” network address of a resource instance, such as within a provider network. Local IP addresses may be within an address block reserved by the Internet Engineering Task Force (IETF) Request for Comments (RFC) 1918 and / or have the address format specified by IETF RFC 4193, and may vary within the provider network. Network traffic originating outside the provider network is not directly routed to a local IP address; instead, the traffic uses a public IP address that maps to the local IP address of the resource instance. A provider network may include networking devices or equipment that provide Network Address Translation (NAT) or similar functionality to perform mappings from public IP addresses to local IP addresses or vice versa.

[0076] A public IP address is a variable network address on the Internet assigned to a resource instance by a service provider or customer. For example, traffic routed to a public IP address via 1:1 NAT translation is used to forward the traffic to the appropriate local IP address of the resource instance.

[0077] Some public IP addresses may be assigned to specific resource instances by the provider's network infrastructure; these public IP addresses may be referred to as standard public IP addresses, or simply standard IP addresses. In some examples, the mapping of standard IP addresses to the local IP addresses of resource instances is the default startup configuration for all resource instance types.

[0078] At least some public IP addresses can be assigned to or obtained by customers of the provider network 800; customers can then assign their assigned public IP addresses to specific resource instances assigned to them. These public IP addresses may be referred to as customer public IP addresses, or simply customer IP addresses. Instead of being assigned to resource instances by the provider network 800 as in the case of standard IP addresses, customer IP addresses can be assigned to resource instances by the customer, for example, via an API provided by the service provider. Unlike standard IP addresses, customer IP addresses are assigned to customer accounts and can be remapped to other resource instances by the corresponding customer as needed or desired. Customer IP addresses are associated with customer accounts, not specific resource instances, and the customer controls this IP address until the customer chooses to release it. Unlike regular static IP addresses, customer IP addresses allow customers to mask resource instance or availability zone failures by remapping their public IP addresses to any resource instance associated with their customer account. For example, customer IP addresses enable customers to resolve resource instance or software issues by remapping their customer IP addresses to alternative resource instances.

[0079] Figure 9 This is a block diagram of an example provider network environment that provides storage services and hardware virtualization services to customers, based on some examples. Hardware virtualization service 920 provides customers with multiple computing resources 924 (e.g., computing instances 925, such as VMs). Computing resources 924 may be provided as a service to customers of provider network 900 (e.g., customers of implementing customer network 950). Each computing resource 924 may be configured with one or more local IP addresses. Provider network 900 may be configured to route packets from the local IP addresses of computing resources 924 to public internet destinations and to route packets from public internet sources to the local IP addresses of computing resources 924.

[0080] Provider network 900 can provide a client network 950, for example, coupled to intermediate network 940 via local network 956, with the ability to implement virtual computing systems 992 via hardware virtualization service 920 coupled to intermediate network 940 and provider network 900. In some examples, hardware virtualization service 920 can provide one or more APIs 902, such as web service interfaces, via which client network 950 can access the functionality provided by hardware virtualization service 920, for example, via console 994 of client device 990 (e.g., web-based applications, standalone applications, mobile applications, etc.). In some examples, at provider network 900, each virtual computing system 992 at client network 950 can correspond to computing resources 924 that are leased, rented, or otherwise provided to client network 950.

[0081] From instances of virtual computing system 992 and / or another client device 990 (e.g., via console 994), a client can access the functionality of storage service 910, for example, via one or more APIs 902, to access data from and store data in storage resources 918A to 918N of virtual data storage 916 (e.g., folders or "buckets," virtualized volumes, databases, etc.) provided by provider network 900. In some examples, a virtualized data storage gateway (not shown) may be located at client network 950, which may locally cache at least some data (e.g., frequently accessed data or critical data) and may communicate with storage service 910 via one or more communication channels to upload new or modified data from the local cache in order to maintain the main data storage area (virtualized data storage 916). In some examples, a user may install and access virtual data storage 916 volumes via storage service 910, which acts as a storage virtualization service, via virtual computing system 992 and / or another client device 990, and these volumes may appear to the user as local (virtualized) storage devices 998.

[0082] Although Figure 9 Although not shown, virtualization services can also be accessed from resource instances within provider network 900 via API 902. For example, a customer, equipment service provider, or other entity can access virtualization services from within a corresponding virtual network on provider network 900 via API 902 to request the allocation of one or more resource instances within that virtual network or another virtual network.

[0083] Explanatory System

[0084] In some examples, systems implementing some or all of the techniques described herein may include general-purpose computer systems, such as Figure 10 The illustrated computing device 1000 (also referred to as a computing system or electronic device) includes, or is configured to access one or more computer-accessible media. In the illustrated example, the computing device 1000 includes one or more processors 1010 coupled to system memory 1020 via an input / output (I / O) interface 1030. The computing device 1000 also includes a network interface 1040 coupled to the I / O interface 1030. Although Figure 10 The computing device 1000 is shown as a single computing device, but in various examples, the computing device 1000 may include a single computing device or any number of computing devices configured to work together as a single computing device 1000.

[0085] In various examples, computing device 1000 may be a single-processor system including one processor 1010 or a multiprocessor system including several processors 1010 (e.g., two, four, eight, or another suitable number). Processor 1010 may be any suitable processor capable of executing instructions. For example, in various examples, processor 1010 may be a general-purpose or embedded processor implementing any of a variety of instruction set architectures (ISAs) (such as x86, ARM, PowerPC, SPARC, or MIPS ISA or any other suitable ISA). In a multiprocessor system, each processor in processor 1010 may typically, but not necessarily, implement the same ISA.

[0086] System memory 1020 may contain instructions and data accessible by storage processor 1010. In various examples, system memory 1020 may be implemented using any suitable memory technology, such as random access memory (RAM), static RAM (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory. In the illustrated example, program instructions and data implementing one or more desired functions (such as the methods, techniques, and data described above) are shown stored within system memory 1020 as model forgetting training service code 1025 (e.g., executable to fully or partially implement model forgetting training service 120) and data 1026.

[0087] In some examples, I / O interface 1030 may be configured to coordinate I / O traffic between processor 1010, system memory 1020, and any peripheral devices within the device, including network interface 1040 and / or other peripheral interfaces (not shown). In some examples, I / O interface 1030 may perform any necessary protocol, timing, or other data transformations to convert data signals from one component (e.g., system memory 1020) into a format suitable for use by another component (e.g., processor 1010). In some examples, I / O interface 1030 may include support for devices attached via various types of peripheral buses, such as, for example, variants of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard. In some examples, the functionality of I / O interface 1030 may be split into two or more separate components, such as, for example, a northbridge and a southbridge. Additionally, in some examples, some or all of the functionality of I / O interface 1030 (such as an interface to system memory 1020) may be directly incorporated into processor 1010.

[0088] Network interface 1040 can be configured to allow data to pass between computing device 1000 and other computing devices 1060 attached to one or more networks 1050 (such as, for example, ...). Figure 1 Exchange between other computer systems or devices shown. In various examples, network interface 1040 can support communication via any suitable wired or wireless general-purpose data network (such as, for example, various types of Ethernet networks). Additionally, network interface 1040 can support communication via telecommunications / telephone networks (such as analog voice networks or digital fiber optic communication networks), via storage area networks (SANs) (such as Fibre Channel SANs), and / or via any other suitable type of network and / or protocol.

[0089] In some examples, computing device 1000 includes one or more offload cards 1070A or 1070B (including one or more processors 1075 and possibly one or more network interfaces 1040), which are connected using I / O interface 1030 (e.g., a version implementing the Peripheral Component Interconnect Fast (PCI-E) standard or a bus of another interconnect (such as a Fast Path Interconnect (QPI) or Hyper Path Interconnect (UPI)). For example, in some examples, computing device 1000 may act as a host electronic device hosting computing resources such as computing instances (e.g., operating as part of a hardware virtualization service), and one or more offload cards 1070A or 1070B act as a virtualization manager that can manage the computing instances running on the host electronic device. As an example, in some examples, offload card 1070A or 1070B may perform computing instance management operations, such as pausing and / or unpausing computing instances, starting and / or terminating computing instances, performing memory translation / copy operations, etc. In some examples, these management operations may be performed collaboratively by the offload card 1070A or 1070B and a hypervisor (e.g., upon request from the hypervisor) executed by other processors 1010A to 1010N of the computing device 1000. However, in some examples, the virtualization manager implemented by the offload card 1070A or 1070B may adapt to requests from other entities (e.g., from the computing instance itself) and may not collaborate with (or serve) any individual hypervisor.

[0090] In some examples, system memory 1020 may be one example of a computer-accessible medium configured to store program instructions and data as described above. However, in other examples, program instructions and / or data may be received, transmitted, or stored on different types of computer-accessible media. Generally, computer-accessible media may include any non-transitory storage medium or memory medium, such as magnetic or optical media, for example, a disk or DVD / CD coupled to computing device 1000 via I / O interface 1030. Non-transitory computer-accessible storage media may also include any volatile or non-volatile medium that may be included as system memory 1020 or another type of memory in some examples of computing device 1000, such as RAM (e.g., SDRAM, Double Data Rate (DDR) SDRAM, SRAM, etc.), read-only memory (ROM), etc. Furthermore, computer-accessible media may include transmission media or signals transmitted via communication media (such as networks and / or wireless links), such as electrical signals, electromagnetic signals, or digital signals, such as those implemented via network interface 1040.

[0091] The various examples discussed or presented herein can be implemented in a variety of operating environments, in some cases of which may include one or more user computers, computing devices, or processing devices that can be used to operate any of a variety of applications. User devices or client devices may include any of a variety of general-purpose personal computers, such as desktop or laptop computers running standard operating systems, and cellular, wireless, and handheld devices running mobile software and capable of supporting multiple networking and messaging protocols. Such systems may also include multiple workstations running a variety of commercially available operating systems and any of other known applications for purposes such as development and database management. These devices may also include other electronic devices, such as virtual terminals, thin clients, gaming systems, and / or other devices capable of communicating via a network.

[0092] Most examples use at least one network familiar to those skilled in the art to support communication using any of a variety of widely available protocols, such as Transmission Control Protocol / Internet Protocol (TCP / IP), File Transfer Protocol (FTP), Universal Plug and Play (UPnP), Network File System (NFS), Public Internet File System (CIFS), Extended Messaging and Field Protocol (XMPP), AppleTalk, etc. The network can include, for example, a Local Area Network (LAN), a Wide Area Network (WAN), a Virtual Private Network (VPN), the Internet, an intranet, an extranet, the Public Switched Telephone Network (PSTN), an infrared network, a wireless network, and any combination thereof.

[0093] In examples using web servers, the web server can run any of a variety of server or middleware applications, including HTTP servers, File Transfer Protocol (FTP) servers, Common Gateway Interface (CGI) servers, data servers, Java servers, business application servers, etc. The server may also be able to execute programs or scripts in response to requests from user devices, such as executing one or more web applications that can be implemented as one or more scripts or programs written in any programming language (such as Java®, C, C#, or C++) or any scripting language (such as Perl, Python, PHP, or TCL) and combinations thereof. The server may also include database servers, including but not limited to commercially available database servers from Oracle®, Microsoft®, Sybase®, IBM®, etc. The database server can be relational or non-relational (e.g., "NoSQL"), distributed or non-distributed, etc.

[0094] The environment disclosed herein may include a variety of data storage areas and other memories and storage media as discussed above. These may reside in a variety of locations, such as on storage media local to one or more computers (and / or residing within one or more computers), or on any or all computers remotely on a network. In a particular set of examples, information may reside in a storage area network (SAN) familiar to those skilled in the art. Similarly, any necessary files for performing functions belonging to a computer, server, or other network device may be stored locally and / or remotely, as appropriate. Where the system includes computerized devices, each such device may include hardware elements that can be electrically coupled via a bus, including, for example, at least one central processing unit (CPU), at least one input device (e.g., mouse, keyboard, controller, touchscreen, or keypad), and / or at least one output device (e.g., display device, printer, or speaker). Such systems may also include one or more storage devices, such as hard disk drives, optical storage devices, and solid-state storage devices (such as random access memory (RAM) or read-only memory (ROM)), as well as removable media devices, memory cards, flash memory cards, etc.

[0095] Such devices may also include computer-readable storage medium readers, communication devices (e.g., modems, network interface cards (wireless or wired), infrared communication devices, etc.), and working memory, as described above. A computer-readable storage medium reader may be connected to or configured to receive a computer-readable storage medium, which represents a remote, local, fixed, and / or removable storage device and storage medium for temporarily and / or more permanently accommodating, storing, transmitting, and retrieving computer-readable information. Systems and various devices will also typically include multiple software applications, modules, services, or other elements residing within at least one working memory device, including operating systems and applications such as client applications or web browsers. It should be understood that alternative examples may have many variations different from those described above. For example, custom hardware may also be used, and / or specific elements may be implemented in hardware, software (including portable software such as applets), or both. Furthermore, connectivity to other computing devices, such as network input / output devices, may be employed.

[0096] Storage media and computer-readable media used to contain code or code portions may include any suitable media known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile media, removable and non-removable media implemented in any way or technology to store and / or transmit information (such as computer-readable instructions, data structures, program modules or other data), including RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, optical disc-read-only memory (CD-ROM), digital universal disc (DVD) or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by system devices. Based on this disclosure and the teachings provided herein, those skilled in the art will appreciate other ways and / or methods for implementing the various examples.

[0097] In the foregoing description, various examples have been described. Specific configurations and details have been elaborated for illustrative purposes to provide a thorough understanding of the examples. However, it will also be apparent to those skilled in the art that the examples can be practiced without these specific details. Furthermore, well-known features may be omitted or simplified to avoid obscuring the described examples.

[0098] This document uses parenthesized text and boxes with dashed borders (e.g., long dashed lines, short dashed lines, dotted lines, and dots) to illustrate optional aspects for adding additional features to some examples. However, such notation should not be interpreted as implying that these are the only options or optional operations, and / or that in some examples, boxes with solid borders are not optional.

[0099] Reference numerals with suffix letters (e.g., 918A to 918N) can be used to indicate that one or more instances of the mentioned entity may exist in various examples, and when multiple instances exist, each instance need not be identical, but may share some general characteristics or function in a common form. Furthermore, the specific suffix used does not imply the existence of a specific number of entities unless specifically indicated otherwise. Therefore, in various examples, two entities using the same or different suffix letters may have or not have the same number of instances.

[0100] The use of terms like "an example" or "example" indicates that the described example may include a specific feature, structure, or characteristic, but each example may not necessarily include that specific feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same example. Additionally, when a specific feature, structure, or characteristic is described in conjunction with an example, it should be assumed that, whether explicitly described or not, implementing such a feature, structure, or characteristic in conjunction with other examples is within the knowledge of those skilled in the art.

[0101] Furthermore, in the various examples described above, unless otherwise specifically indicated, the intent of the disjunctive language, such as the phrase “at least one of A, B, or C,” is understood to mean A, B, or C, or any combination thereof (e.g., A, B, and / or C). Similarly, the intent of the language, such as “at least one or more of A, B, and C” (or “one or more of A, B, and C”), is understood to mean A, B, or C, or any combination thereof (e.g., A, B, and / or C). Therefore, the disjunctive language is neither intended nor should be understood to imply that a given example requires the existence of at least one of A, at least one of B, and at least one of C.

[0102] As used herein, the term "based on" (or similar) is an open-ended term used to describe one or more factors that influence a determination or other action. It should be understood that this term does not exclude additional factors that may influence a determination or action. For example, a determination may be based solely on the listed factors or on the stated factors and one or more additional factors. Therefore, if action A is "based on" B, it should be understood that B is a factor influencing action A, but this does not preclude the action from also being based on one or more other factors, such as factor C. However, in some cases, action A may be entirely based on B.

[0103] Unless otherwise expressly stated, articles such as "a / an" should generally be interpreted as including one or more of the described items. Therefore, phrases such as "a device configured to..." or "computing device" are intended to include one or more of the described devices. Such one or more described devices may be collectively configured to perform the described operations. For example, "a processor configured to perform operations A, B, and C" could include a first processor configured to perform operation A working in conjunction with a second processor configured to perform operations B and C.

[0104] Furthermore, the words “may” or “can” are used in a permissive sense (i.e., implying a possibility) rather than a mandatory sense (i.e., implying a requirement). The words “include,” “including,” and “includes” are used to indicate an open relationship and are therefore intended to include, but are not limited to, this. Similarly, the words “have,” “having,” and “has” also indicate an open relationship and are therefore intended to include, but are not limited to, this. Terms such as “first,” “second,” “third,” etc., as used herein, are used as markers for the nouns that follow them and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless otherwise explicitly indicated. Similarly, the values ​​of such numerical markers are generally not used to indicate the required quantity of a particular noun in a claim set forth herein, and therefore the element “fifth” generally does not imply the presence of four other elements unless those elements are explicitly included in the claim or their presence is otherwise sufficiently clear.

[0105] At least some embodiments of the disclosed technology may be described in light of the following terms: 1. A computer-implemented method, the computer-implemented method comprising: The machine learning service of the cloud provider network receives a request to forget a concept from a pre-trained generative machine learning model, the request including the identification of the pre-trained generative machine learning model and the description of the concept. Generate a set of negative prompts and a set of positive prompts, wherein the negative prompts include the concept and the positive prompts do not include the concept; Using the pre-trained generative machine learning model, positive and negative cues are processed to generate associated activation volume maps, wherein the activation volume map for a given cue includes the outputs from one or more layers of the pre-trained generative model. Identify a set of activation conditions to distinguish between activation volume maps associated with negative cues and activation volume maps associated with positive cues; Generate a model adapter to use a different set of model parameters when the processing of the cue by the pre-trained generative machine learning model satisfies the set of activation conditions.

[0106] 2. The computer-implemented method according to Clause 1, further comprising: The pre-trained generative machine learning model and the model adapter are executed using the computing resources of the cloud provider's network. Receive prompts submitted by the user; and The user-submitted prompt is processed using the pre-trained generative machine learning model, wherein the processing includes: The model adapter is used to determine whether the processing of the user-submitted prompt satisfies the set of activation conditions; and The user-submitted prompts are processed using the different set of model parameters.

[0107] 3. A computer-implemented method according to any one of Clauses 1 to 2, wherein the set of different model parameters is based on at least one of: zeroing a corresponding set of pre-trained model weights, adding random noise to the corresponding set of pre-trained model weights, or adding a predetermined set of weights to the corresponding set of pre-trained model weights, so as to bias the pre-trained generative machine learning model from the concept to another concept.

[0108] 4. A computer-implemented method, the computer-implemented method comprising: Receive a description of the concepts to be forgotten when using a pre-trained generative machine learning model; The pre-trained generative machine learning model is used to process negative and positive cues to generate associated activation volume maps. Identify a set of conditions to distinguish between activation volume maps associated with negative cues and activation volume maps associated with positive cues; and Generate a model adapter to use a different set of model parameters when the processing of the prompts by the pre-trained generative machine learning model satisfies the set of conditions.

[0109] 5. The computer-implemented method according to Clause 4, further comprising: The model adapter is used to execute the pre-trained generative machine learning model; Receive prompts submitted by the user; and The user-submitted prompt is processed using the pre-trained generative machine learning model, wherein the processing includes: The model adapter is used to determine whether the processing of the user-submitted prompt satisfies the set of conditions; and The user-submitted prompts are processed using the different set of model parameters.

[0110] 6. The computer-implemented method according to Clause 5, wherein processing the user-submitted prompt using the set of different model parameters includes at least one of: updating the pre-trained model parameters using a set of update parameters, or redirecting the output of a layer from the pre-trained generative machine learning model through different layers including the set of different model parameters.

[0111] 7. A computer-implemented method according to any one of Clauses 4 to 6, wherein the set of different model parameters is based on at least one of: zeroing a corresponding set of pre-trained model weights of the pre-trained generative machine learning model, adding random noise to the corresponding set of pre-trained model weights, or adding a predetermined set of weights to the corresponding set of pre-trained model weights to bias the pre-trained generative machine learning model from the concept to another concept.

[0112] 8. A computer-implemented method according to any one of clauses 4 to 6, the computer-implemented method further comprising receiving an indication of the degree of forgetting the concept, wherein at least a portion of the different set of model parameters is based on the degree of forgetting the concept.

[0113] 9. A computer-implemented method according to any one of claims 4 to 6, the computer-implemented method further comprising using gradient-based activation mapping to identify a first region of a model activation volume sensitive to a first negative cue and a second region of the model activation volume sensitive to a first positive or negative cue, wherein the set of conditions is limited to activations within the first and second regions of the model activation volume.

[0114] 10. A computer-implemented method according to any one of clauses 4 to 6, the computer-implemented method further comprising generating at least some of the negative prompts and at least some of the positive prompts, the negative prompts including the concept and the positive prompts not including the concept.

[0115] 11. The computer-implemented method according to Clause 10, further comprising: Receive instructions on related concepts and instructions on the correlation between the concept to be forgotten and the related concepts; Determine that the indication of the correlation meets a threshold; The pre-trained generative machine learning model is used to generate output based on cues including the relevant concepts. The user-submitted classifications received from the output are used as positive samples; and The prompts that include the aforementioned related concepts are classified as positive prompts.

[0116] 12. The computer-implemented method according to any one of Clauses 4 to 6, wherein the computer-implemented method is performed via machine learning services of a cloud provider network.

[0117] 13. A computer-implemented method according to any one of clauses 4 to 6, wherein the description of the concept to be forgotten is received from an electronic device, and the computer-implemented method further comprises: The model adapter is sent to the electronic device, thereby enabling the model adapter to be deployed together with the pre-trained generative machine learning model to reduce the likelihood of the concept to be forgotten appearing in the output of the pre-trained generative machine learning model.

[0118] 14. A system comprising: First or more computing devices, the first or more computing devices being configured to execute pre-trained generative machine learning models in a multi-tenant provider network; and A second or more computing device is configured to implement a model forgetting training service in the multi-tenant provider network, the model forgetting training service including instructions that, when executed, cause the model forgetting training service to: Receive a description of the concepts to be forgotten when using the pre-trained generative machine learning model; The pre-trained generative machine learning model is used to process negative and positive cues to generate associated activation volume maps. Identify a set of conditions to distinguish between activation volume maps associated with negative cues and activation volume maps associated with positive cues; and Generate a model adapter to use a different set of model parameters when the processing of the prompts by the pre-trained generative machine learning model satisfies the set of conditions.

[0119] 15. The system according to Clause 14, further comprising: A third or more computing devices, configured to execute the pre-trained generative machine learning model using the model adapter within the multi-tenant provider network, wherein the environment for executing the pre-trained generative machine learning model using the model adapter includes instructions for the following operations: Receive prompts submitted by the user; and The user-submitted prompt is processed using the pre-trained generative machine learning model, wherein the instructions for processing include instructions for the following operations: The model adapter is used to determine whether the processing of the user-submitted prompt satisfies the set of conditions; and The user-submitted prompts are processed using the different set of model parameters.

[0120] 16. The system of claim 15, wherein the instructions for processing the user-submitted prompt using the set of different model parameters include at least one of the following: instructions for updating the pre-trained model parameters using a set of update parameters, or instructions for redirecting the output of a layer from the pre-trained generative machine learning model through different layers including the set of different model parameters.

[0121] 17. The system according to any one of Clauses 14 to 16, wherein the different set of model parameters is based on at least one of: zeroing out a corresponding set of pre-trained model weights of the pre-trained generative machine learning model, adding random noise to the corresponding set of pre-trained model weights, or adding a predetermined set of weights to the corresponding set of pre-trained model weights to bias the pre-trained generative machine learning model from the concept to another concept.

[0122] 18. The system according to any one of Clauses 14 to 16, wherein the model forgetting training service includes additional instructions that, when executed, cause the model forgetting training service to receive an indication of the degree to which the concept is forgotten, wherein at least a portion of the different set of model parameters is based on the degree to which the concept is forgotten.

[0123] 19. The system according to any one of Clauses 14 to 16, wherein the model forgetting training service includes additional instructions that, when executed, cause the model forgetting training service to use gradient-based activation mapping to identify a first region of the model activation volume sensitive to a first negative cue and a second region of the model activation volume sensitive to a first positive or negative cue, and wherein the set of conditions is limited to activations within the first and second regions of the model activation volume.

[0124] 20. The system according to any one of Clauses 14 to 16, wherein the model forgetting training service includes additional instructions that, when executed, cause the model forgetting training service to generate at least some of the negative prompts and at least some of the positive prompts, the negative prompts including the concept and the positive prompts not including the concept.

[0125] The specification and drawings should therefore be considered illustrative rather than restrictive. However, it will be apparent that various modifications and alterations may be made therein without departing from the broader scope of this disclosure as set forth in the claims.

Claims

1. A computer-implemented method, the computer-implemented method comprising: Receive a description of the concepts to be forgotten when using a pre-trained generative machine learning model; The pre-trained generative machine learning model is used to process negative and positive cues to generate associated activation volume maps. Identify a set of conditions to distinguish between activation volume maps associated with negative cues and activation volume maps associated with positive cues; and Generate a model adapter to use a different set of model parameters when the processing of the prompts by the pre-trained generative machine learning model satisfies the set of conditions.

2. The computer-implemented method according to claim 1, further comprising: The model adapter is used to execute the pre-trained generative machine learning model; Receive prompts submitted by users; as well as The user-submitted prompt is processed using the pre-trained generative machine learning model, wherein the processing includes: The model adapter is used to determine whether the processing of the prompt submitted by the user satisfies the set of conditions; and The user-submitted prompts are processed using the different set of model parameters.

3. The computer-implemented method of claim 2, wherein processing the user-submitted prompt using the set of different model parameters comprises at least one of: updating the pre-trained model parameters using a set of update parameters, or redirecting the output of a layer from the pre-trained generative machine learning model through different layers including the set of different model parameters.

4. The computer-implemented method according to any one of claims 1 to 3, wherein the set of different model parameters is based on at least one of: zeroing a corresponding set of pre-trained model weights of the pre-trained generative machine learning model, adding random noise to the corresponding set of pre-trained model weights, or adding a predetermined set of weights to the corresponding set of pre-trained model weights, so that the pre-trained generative machine learning model is biased from the concept to another concept.

5. The computer-implemented method according to any one of claims 1 to 3, the computer-implemented method further comprising receiving an indication of the degree of forgetting the concept, wherein at least a portion of the different set of model parameters is based on the degree of forgetting the concept.

6. The computer-implemented method according to any one of claims 1 to 3, the computer-implemented method further comprising using gradient-based activation mapping to identify a first region of the model activation volume sensitive to a first negative cue and a second region of the model activation volume sensitive to a first positive or negative cue, wherein the set of conditions is limited to activations within the first and second regions of the model activation volume.

7. The computer-implemented method according to any one of claims 1 to 3, further comprising generating at least some of the negative prompts and at least some of the positive prompts, wherein the negative prompts include the concept and the positive prompts do not include the concept.

8. The computer-implemented method according to claim 7, further comprising: Receive instructions on related concepts and instructions on the correlation between the concept to be forgotten and the related concepts; Determine that the indication of the correlation meets a threshold; The pre-trained generative machine learning model is used to generate output based on cues including the relevant concepts. The user-submitted classifications that receive the output are taken as positive samples; as well as The prompts that include the aforementioned related concepts are classified as positive prompts.

9. The computer-implemented method according to any one of claims 1 to 3, wherein the description of the concept to be forgotten is received from an electronic device, and the computer-implemented method further comprises: The model adapter is sent to the electronic device, thereby enabling the model adapter to be deployed together with the pre-trained generative machine learning model to reduce the likelihood of the concept to be forgotten appearing in the output of the pre-trained generative machine learning model.

10. A system comprising: A first or more computing devices are configured to execute a pre-trained generative machine learning model in a multi-tenant provider network. as well as A second or more computing device is configured to implement a model forgetting training service in the multi-tenant provider network, the model forgetting training service including instructions that, when executed, cause the model forgetting training service to: Receive a description of the concepts to be forgotten when using the pre-trained generative machine learning model; The pre-trained generative machine learning model is used to process negative and positive cues to generate associated activation volume maps. Identify a set of conditions to distinguish between activation volume maps associated with negative cues and activation volume maps associated with positive cues; and Generate a model adapter to use a different set of model parameters when the processing of the prompts by the pre-trained generative machine learning model satisfies the set of conditions.

11. The system of claim 10, further comprising: A third or more computing devices, configured to execute the pre-trained generative machine learning model using the model adapter within the multi-tenant provider network, wherein the environment for executing the pre-trained generative machine learning model using the model adapter includes instructions for the following operations: Receive prompts submitted by the user; and The user-submitted prompt is processed using the pre-trained generative machine learning model, wherein the instructions for processing include instructions for the following operations: The model adapter is used to determine whether the processing of the user-submitted prompt satisfies the set of conditions; and The user-submitted prompts are processed using the different set of model parameters.

12. The system of claim 11, wherein the instructions for processing the user-submitted prompt using the set of different model parameters include at least one of the following: instructions for updating pre-trained model parameters using a set of update parameters, or instructions for redirecting the output of a layer from the pre-trained generative machine learning model through different layers including the set of different model parameters.

13. The system according to any one of claims 10 to 12, wherein the set of different model parameters is based on at least one of: zeroing a corresponding set of pre-trained model weights of the pre-trained generative machine learning model, adding random noise to the corresponding set of pre-trained model weights, or adding a predetermined set of weights to the corresponding set of pre-trained model weights to bias the pre-trained generative machine learning model from the concept to another concept.

14. The system according to any one of claims 10 to 12, wherein the model forgetting training service includes additional instructions that, when executed, cause the model forgetting training service to receive an indication of the degree to which the concept is forgotten, wherein at least a portion of the different set of model parameters is based on the degree to which the concept is forgotten.

15. The system according to any one of claims 10 to 12, wherein the model forgetting training service includes additional instructions that, when executed, cause the model forgetting training service to use gradient-based activation mapping to identify a first region of the model activation volume sensitive to a first negative cue and a second region of the model activation volume sensitive to a first positive or negative cue, and wherein the set of conditions is limited to activations within the first and second regions of the model activation volume.

Images