Personalized generative neural networks

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating personalized output data items using a generative neural network. In one aspect, one of the method includes maintaining a respective set of adaptation parameters of a generative neural network for each of a plurality of predetermined content providers; obtaining a new input associated with a new content provider; using a classifier model to generate a respective score for each of the plurality of predetermined content providers; generating a new set of adaptation parameters based on determining a weighted combination of the respective set of adaptation parameters for each of a subset of the plurality of predetermined content providers; and generating, by the generative neural network and based on the new set of adaptation parameters and the new input, a new personalized output data item.

Description

[0001] PERSONALIZED GENERATIVE NEURAL NETWORKS

[0002] BACKGROUND

[0003] This specification relates to generating data items using neural networks. For example, the data items can include textual data items, audio data items, image data items, and so on.

[0004] Neural networks are machine learning models that employ one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from a received input in accordance with current values of a respective set of parameters.

[0005] SUMMARY

[0006] This specification describes a personalized data generation system implemented as computer programs on one or more computers in one or more locations that maintains a plurality of sets of adaptation parameters for use with a generative neural network having a set of base neural network parameters and that generates personalized output data items based on one or more sets of the plurality of sets of adaptation parameters and the set of base neural network parameters.

[0007] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages. Performance of a generative neural network on many content generation applications, e.g., conditional text, image, or audio generation tasks, can be improved when the generative neural network is able to generate personalized output data items, e.g., an output sequence of text that adheres to a user’s language style, an image that adheres to a user's visual style, or an audio signal that adheres to a user’s audio style. This saves computational and memory resources that are otherwise required by, e.g., the execution of multiple runs of the generative neural network to generate multiple output data items that are generic, i.e., not personalized, until an output data item that satisfies the needs of a user is generated; or the further processing, e.g., modification, of such generic output data items to obtain the personalized output data items. This also improves user experience with these content generation applications.

[0008] In order to be able to generate personalized output data items for different users, a corresponding set of adaptation parameters would need to be learned for each different user. Learning this set of adaptation parameters could have been needed each time that a new user of the personalized data generation system requests the generation of a personalized output data item. However, repeatedly learning and storing a set of adaptation parameters for each new user consumes a significant amount of computational and memory resources when the user base is large, e.g., when there are one million, two million, ten million, or more users. In fact, learning a set of adaptation parameters for each user becomes practically infeasible when the user base is large.

[0009] By leveraging a classifier model to dynamically compose a new set of adaptation parameters for each new user based on the plurality of sets of adaptation parameters that are already stored in a personalization database, the personalized data generation system reduces the amount of computational and memory resources consumed by the adaptation process because learning and storing a set of adaptation parameters for each new user is no longer required. Instead, a new set of adaptation parameters can be dynamically generated for any new user based on a relatively small number of existing sets of adaptation parameters. Thus, the personalized data generation system can enhance the performance of the generative neural network to generate personalized output data items that will more likely satisfy the needs of a user with reduced consumption of computational and memory resources.

[0010] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

[0011] BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG. 1 shows an example personalized data generation system.

[0013] FIG. 2 shows an example training system.

[0014] FIG. 3 is a flow diagram of an example process for generating a new personalized output.

[0015] FIG. 4 is a flow diagram of an example process for generating a new set of adaptation parameters for a new user.

[0016] Like reference numbers and designations in the various drawings indicate like elements.

[0017] DETAILED DESCRIPTION FIG. 1 is a diagram of an example personalized data generation system 100. The personalized data generation system 100 is an example of a system implemented as computer programs on one or more computers in one or more locations, in which the systems, components, and techniques described below can be implemented.

[0018] The personalized data generation system 100 is a system that generates personalized output data items 152 in response to received requests 102. An output data item is referred to as ‘‘personalized’7because the output data item includes attributes that are specific to a user that submitted the request 102. The personalized data generation system 100 can generate any kind of personalized output data items 152, e.g., textual data items, image data items, audio data items, and so on.

[0019] In some cases, the personalized data generation system 100 can be a text generation system that generates text sequences, i.e., each personalized output data item 152 generated by the system is an output sequence of text that includes a sequence of text tokens from a vocabulary of text tokens that includes, e.g., one or more of characters, sub-words, words, punctuation marks, numbers, or other symbols that appear in a natural language or a computer language. For example, the system can generate text sequences in response to a request 102 submitted by a user of the system and provide the text sequences for presentation to the user which submitted the request 102.

[0020] In some of these cases, the personalized data generation system 100 can receive a conditioning input as part of, or associated with, the request 102 and generate a personalized output data item 152 that is a response to the conditioning input.

[0021] For example, the conditioning input can be an input sequence of text and the output sequence is another sequence of text, e.g., a translation of the input sequence of text, a completion of the input sequence of text, a paraphrase of the input sequence of text, a response to a question posed in the input sequence, or a sequence of text that is about a topic specified by the input sequence of text. As another example, the conditioning input can be an input other than text, e.g., an image, and the output sequence can be text that describes the input.

[0022] In these cases, a user can have a preference for the output sequence of text, e.g., that the output sequence of text adheres to the user’s language style. For example, if the input sequence of text is a sequence of text in one language, and the output sequence of text is a piece of text in another language that is a predicted proper translation of the input sequence of text into the other language, the user can have a preference that the translation be proper in the other language while aligning with the specific manner in which the user communicates in the language of the input sequence of text. As another example, if the conditioning input includes an image, and the output sequence of text is a piece of text in another language that is a caption of the image or an answer to a question posed about the image, the user can have specific language style preferences or characteristics for captioning or the format of the answer.

[0023] As a particular example, the personalized data generation system 100 can be part of a dialog system and the conditioning input can include audio or text from the most recent conversational turn submitted by a user of the dialog system during the dialog while the output sequence of text is the next turn in the conversation, e.g., either text or audio that is a response to the most recent conversational turn. Optionally, the conditioning input can also include one or more historical conversational turns that occurred earlier in the conversation. In this example, a user may prefer that the output sequence of text aligns with a specific manner in which the user communicates in a more casual setting, e.g., includes specific slang, specific idioms, specific abbreviations, specific punctuations, etc.

[0024] As another particular example, the personalized data generation system 100 can be part of a computer code generation system and the conditioning input can be a text description of a desired piece of code or a snippet of computer code in a programming language and the output sequence of text can be computer code, e.g., a snippet of code that is described by the conditioning input or a snippet of code that follows the conditioning input in a computer program. In this example, a user may prefer that the output sequence of text aligns with a specific manner in which the user writes code, e.g., includes specific indentation structure, specific variable names, etc.

[0025] In some cases, the personalized data generation system 100 can be an image or video generation system that generates images or videos that each have multiple frames (where each frame is an image) as sequences of pixels, i.e., each personalized output data item 152 generated by the system includes a sequence of color values for pixels in an output image arranged according to a specified order.

[0026] In these cases, a user can have a preference for the images generated by the system, e.g., that the images adhere to the user's visual style that is represented by one or more specific visual attributes. For example, the user can have a preference that an image includes a specific color, a specific texture, a specific brightness, etc. As another example, the user can have a preference that an image includes a depiction of a specific object, a depiction of a specific watermark, a depiction of a specific logo, a depiction of a specific symbol, etc. In some cases, the personalized data generation system 100 can be an audio generation system that generates audio signals, e.g.. each personalized output data item 152 is an output audio example that includes a sample of an audio wave at each of a sequence of output time steps that span a specified time window. For example, the output time steps can be arranged at regular intervals within the specified time window. The audio sample at a given output time step can be an amplitude value of the audio wave or an amplitude value that has been compressed, companded, or both. For example, the audio sample can be a raw amplitude value or a mu-law companded representation of the amplitude value.

[0027] In these cases, a user can have a preference for the audio signals generated by the system, e.g., that the audio signals adhere to the user’s audio style that is represented by one or more specific audio attributes. For example, the user can have a preference that an output audio example has a specific acoustic property, e.g., a specific speaker identity, a specific recording condition (such as a specific level of reverberation, distortion, or background noise, etc. As another example, the user can have a preference that an output audio example has a specific semantic property, e.g., a specific linguistic content (when the output audio example represents speech), or a specific melody a specific rhythm (when the output audio example represents music).

[0028] In particular, the personalized data generation system 100 receives a request 102 for a personalized output data item 152 and, in response, generates the personalized output data item 152 using a generative neural network 110 that has a set of base neural network parameters 1 15.

[0029] The generative neural network 110 can be any appropriate generative neural network that has been trained, e.g., by the personalized data generation 100 or another training system, on one or more generative modeling tasks, e.g., a task that requires predicting, given a current sequence of tokens, the next token that follows the current sequence in the training data.

[0030] In some implementations, the generative neural network 110 can have any of a variety of Transformer-based neural network architectures, e.g., encoder-only Transformer architectures, encoder-decoder Transformer architectures, decoder-only Transformer architectures, other attention-based architectures, and so on. In implementations the generative neural network 110 can be configured as, or include, a generative (large) language model or a multi-modal model, e.g., a visual and language model.

[0031] Examples of such architectures include those described in Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang. Michael Matena, Yanqi Zhou. Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv: 1910. 10683, 2019; Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall. Noah Fiedel. Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. Towards a human-like opendomain chatbot. CoRR, abs / 2001.09977, 2020; Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell. et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165. 2020; Aakanksha Chowdhery. et al. PaLM: Scaling Language Modeling with Pathways, arXiv preprint arXiv:2204.02311; and Rohan Anil, et al. Palm 2 technical report. arXiv preprint arXiv:2305. 10403, 2023.

[0032] For example, the generative neural network 110 can have been trained on a large training dataset based on optimizing a maximum-likelihood objective to determine the trained values of the set of base neural network parameters 115. In various cases, such a training dataset can be a large dataset of text in one or more natural languages, e.g., text that is publicly available from the Internet or another text corpus, a large dataset of computer code in one or more programming languages, e.g., computer code that is publicly available from the Internet or another code repository, a large dataset of audio samples, e.g.. audio recordings or waveforms that represent the audio recordings, a large dataset of images where each image includes an array of pixels, a large dataset of videos w here each video includes a temporal sequence of frames, or a large multi-modal dataset that includes a combination of two or more of these datasets.

[0033] The personalized data generation system 100 includes or accesses a personalization database 141. The personalization database 141 stores a plurality of sets of adaptation parameters 140A-N that correspond (or map) respectively to a plurality of predetermined users. These predetermined users can include a group of existing users of the personalized data generation system 100 or a group of default, placeholder users. FIG. 1 thus illustrates that the personalization database 141 stores a first set of adaptation parameters 140A that corresponds to a first predetermined user, a second set of adaptation parameters 140B that corresponds to a second predetermined user, a third set of adaptation parameters 140C that corresponds to a third predetermined user, and so on. In some implementations, the personalized data generation system 100 does not store, or otherwise has access to, any other information that characterizes these users.

[0034] A user may be an individual. Alternatively, a user may be a “content provider” entity, e.g., a named group of people, an organization, a website, or a company, that creates, manages, maintains, and / or updates digital content. For example, a content provider can create, maintain, remove, modify, and / or update electronic documents. An electronic document is data that presents a set of digital content. Examples of electronic documents include webpages, word processing documents, spreadsheets, portable document format (PDF) documents, images, videos, audios, and feed sources.

[0035] Each set of adaptation parameters 140A-N included in the personalization database 141 is an additional set of parameters that can be used together with the set of base neural network parameters 115 to adapt the generative neural network 110 to generate the personalized output data items 152.

[0036] To generate a personalized output data item 152 in response to a request 102 submitted by a user of the system, the personalized data generation system 100 can first identify, from among the plurality of sets of adaptation parameters 140A-N, a set of adaptation parameters that corresponds to the user, and then processes a new input using both (i) the set of base neural network parameters 115 and (ii) the identified set of adaptation parameters to generate the personalized output data item 152. For example, the new input can include the conditioning input when it is received as part of, or associated with, the request 102. For example, the identified set of adaptation parameters can represent an additional set of parameters that will be used together with the set of base neural network parameters 115 to process the new input to generate the personalized output data item 152; or a soft prompt embedding that will be processed alongside with the new input by the set of base neural network parameters 115 to generate the personalized output data item 152.

[0037] As will be described further below with reference to FIG. 2, each set of adaptation parameters 140A-N can be learned using any appropriate neural network adaptation technique, e.g., a fine-tuning adaptation technique or a prompt tuning technique, on a set of training data that is specific to a predetermined user that correspond to the set of adaptation parameters. Thus, learning different sets of adaptation parameters 140A-N will generally require different sets of training data.

[0038] The plurality of sets of adaptation parameters 140A-N that correspond respectively to a plurality of predetermined users can have been learned independently of the set of base neural network parameters 1 15 of the generative neural network 110. That is, the plurality of sets of adaptation parameters 140A-N generally were not used during the training of the generative neural network 110, during which the set of base neural network parameters 115 were updated.

[0039] In order to be able to generate personalized output data items 152 for different users, a corresponding set of adaptation parameters would need to be learned for each different user. Learning this set of adaptation parameters could have been needed each time that a new user of the personalized data generation system 100 requests the generation of a personalized output data item 152. However, repeatedly learning and storing a set of adaptation parameters for each new user consumes a significant amount of computational and memory resources when the user base is large, e.g., when there are one million, two million, ten million, or more users. In fact, learning a set of adaptation parameters for each user becomes practically infeasible when the user base is large.

[0040] To accommodate the scenarios where the user base is large, the personalized data generation system 100 adopts techniques that leverage a classifier model to dynamically compose a new set of adaptation parameters for each new user based on the plurality of sets of adaptation parameters 140A-N that are already stored in the personalization database 141. Such dynamic adaptation parameter composition techniques allow the system to generate personalized output data items 152 that accurately align with the preferences of each of a very7large, possibly infinitely large, number of different users with minimal additional computational and memory overhead.

[0041] FIG. 1 thus illustrates that the personalized data generation system 100 also includes a classifier model 130. The classifier model 130 is configured to process a model input that includes the request 102, data derived from the request 102, or both to generate a classification output that specifies a predicted classification of the user which submits the request 102, with reference to the plurality of predetermined users (which correspond respectively to the plurality of sets of adaptation parameters 140A-N).

[0042] The classification output can include a respective score for each of the plurality' of predetermined users. A score for a particular predetermined user represents a respective likelihood that the user which submitted the request 102 is the particular predetermined user. Generally, a higher score indicates a greater likelihood. For example, when the classification output computed by the classifier model 130 for a given request 102 includes a higher score for a first predetermined user and a lower score for a second predetermined user, it means there is a greater likelihood that the user which submitted the given request 102 is the first predetermined user than the second predetermined user.

[0043] The classifier model 130 may be, but need not be, a neural network. For example, the classifier model 130 can be one of: a decision tree model, a random forest model, a gradient boosting model, a linear regression model, a logistical regression model, a support vector machine (SVM) model, and so on. Alternatively, when configured as a neural network, the classifier model 130 can have any appropriate neural network architecture. That is, the classifier model 130 can include any appropriate types of neural network layers (e.g., embedding layers, fully connected layers, and so forth) in any appropriate number (e.g., 2 layers, or 5 layers, or 10 layers) and connected in any appropriate configuration (e.g., as a directed graph of layers). As a particular example, the classifier model 130 can be configured as a feed-forward neural network, e.g., a Transformer neural network, or a recurrent neural network, e.g., a long short-term memory neural network.

[0044] Having generated the classification output, the personalized data generation system 100 can then use the classification output to generate a new set of adaptation parameters 120 for the new user which submits the request 102. The new set of adaptation parameters 120 is generated based on determining a weighted combination of the respective set of adaptation parameters for each of at least a subset of the plurality of predetermined users. In the weighted combination, each set of adaptation parameters is weighted by a weight that is dependent on the respective score for each of the plurality of predetermined users.

[0045] FIG. 2 shows an example training system 200. The training system 200 is an example of a system implemented as computer programs on one or more computers in one or more locations that can train the plurality of sets of adaptation parameters 140A-N of FIG. 1 and that can train the classifier model 130 of FIG. 1. Each set of adaptation parameters 140A-N corresponds (or maps) to a respective one of the plurality of predetermined users of the personalized data generation system 100 of FIG. 1.

[0046] The training system 200 obtains generative neural network adaptation data 170. The generative neural network adaptation data 170 includes, for each of the plurality of predetermined users, one or more training pairs. Each training pair includes a training input and a target output. Generally, each training input includes the same type(s) of data as the conditioning input. Each target output includes the same type(s) of data as the personalized output data items 152 which the personalized data generation system 100 is configured to generate.

[0047] For example, when the personalized output data item 152 is a sequence of text, the training input can be a first sequence of text that has a first plurality of text tokens, and the target output can be a second sequence of text that has a second plurality of text tokens. As another example, when the personalized output data item 152 is an image, the training input can be a first image that has a first plurality of pixels, and the target output can be a second image that has a second plurality of pixels. As another example, when the personalized output data item 152 is an audio signal, the training input can be a waveform or spectrogram of a first audio signal, and the target output can be a waveform or spectrogram of a second audio signal.

[0048] For each training pair that corresponds to a predetermined user, the target output within the training pair is typically associated with the predetermined user. For example, the target output can be a sequence of text, an image, or an audio signal provided by the predetermined user. As another example, the target output can be a sequence of text, an image, or an audio signal that has writing / visual / audio styles that mimic those of the predetermined user, e.g., that has the attributes that are specific to the predetermined user, e.g., as described in a preference profile of the predetermined user.

[0049] In some cases, the training input within the training pair is also associated with the predetermined user. That is, the training input and the target output within each training pair are associated with the same user. In other cases, the training input within the training pair is not associated with the predetermined user. That is, the training input and the target output within each training pair are associated with different users.

[0050] In some cases, the training input and the target output within the training pair can have about equal length. For example, they include about the same number of text tokens or about the same number of image pixels. In other cases, the training input and the target output within the training pair can have different lengths. For example, the training input can be longer than the target output.

[0051] As a particular example, the training input within a training pair can be a first sequence of text in a natural language, e.g., it is a body of an email or content of a book, and the target output within the training pair is a second sequence of text that is shorter than the first sequence of text but summarizes the first sequence of text, i.e., represents the most important or relevant information within the first sequence of text, e.g.. it is a subject line of the email or a title / abstract of the book, and that w ould be provided by a predetermined user that corresponds to the training pair based on the first sequence of text.

[0052] The training system 200 can receive the generative neural network adaptation data 170 in any of a variety of ways. For example, the training system 200 can receive generative neural network adaptation data 170 as an upload from a user of the system over a data communication network, e.g., using an application programming interface (API) or a graphical user interface (GUI) made available by the training system 200. As another example, the training system 200 can receive an input from a user specifying which data that is already maintained by the system or another system that is accessible by the training system 200 should be used as the generative neural network adaptation data 170. The training system 200 uses the generative neural network adaptation data 170 to learn the trained values of the plurality’ of sets of adaptation parameters 140A-N that correspond respectively to the plurality of predetermined users.

[0053] The training system 200 trains the plurality of sets of adaptation parameters 140A-N together with the generative neural network 110 to adapt the generative neural network 110 to generate output data items that are close in style to the target outputs included in the training pairs, e.g., to generate sequences of text, images, or audio signals that have writing / visual / audio styles that mimic those of the target outputs. Any of a variety of training techniques can be used.

[0054] In some implementations, the training system 200 uses a fine-tuning adaptation technique. Each set of adaptation parameters 140A-N learned as a result of fine-tuning adaptation generally represents an additional set of parameters of the generative neural network 110, e.g., that represent additional weights and, optionally, additional biases of one or more layers of the generative neural network 110, that can be used together with the set of base neural network parameters 115 to process the new inputs to generate personalized output data items.

[0055] More specifically, for each of the plurality of predetermined users, the training system 200 can do this by, for each of the one of more training pairs included in the generative neural network adaptation data 170 that corresponds to the predetermined user: processing the training input in the training pair using (i) the set of base neural network parameters 115 of the generative neural network 1 10 and (ii) a respective set of adaptation parameters that corresponds to the predetermined user to generate a predicted output; and then adjusting the respective set of adaptation parameters based at least in part on a difference between the predicted output and the target output included in the training pair.

[0056] The training system 200 can do this by computing, for each training pair, respective gradients of a fine-tuning objective function, e.g., a cross-entropy loss function, that evaluates such a difference w ith respect to the respective set of adaptation parameters through backpropagation. The training system 200 can then determine the updates by applying an update rule, e.g.. an Adam update rule, an Rmsprop update rule, or a stochastic gradient descent (SGD) update rule, to the respective gradients.

[0057] In these implementations, the training system 200 can incorporate any number of techniques to improve the speed, the effectiveness, or both of the fine-tuning adaptation process. For example, the training system 200 can hold the set of base neural network parameters 115 of the generative neural network 110 fixed to their trained values, such that they are not adjusted during the fine-turning adaptation process.

[0058] As another example, to reduce the total number of trainable parameters and therefore preserve computing and memory resource consumption of the fine-tuning adaptation process, the training system 200 can use a low-rank approximation (LoRA) technique as described in Edward Hu, et al. "Lora: Low-rank adaptation of large language models." arXiv preprint arXiv:2106.09685 (2021), or another rank-decomposition-based approximation technique. Low-rank approximation refers to breaking up a matrix containing the adaptation parameters to be learned into a product of two smaller matrices that when multiplied together can (approximately) recover the adaptation parameters.

[0059] In some other implementations, the training system 200 uses a prompt tuning technique. Each set of adaptation parameters 140A-N learned as a result of prompt tuning generally represents an embedding (a “soft prompt embedding”) that can be processed alongside the new inputs by the generative neural network 110 in accordance with the set of base neural network parameters 115 to generate personalized output data items. As embedding refers to an ordered collection of numerical values, e.g.. a vector, matrix, or other tensor of numerical values. Suitable prompt tuning techniques are described in Brian Lester et al., “The power of scale for parameter-efficient prompt tuning.” arXiv preprint arXiv:2104.08691 (2021), but other prompt tuning techniques can also be used.

[0060] More specifically, for each of the plurality of predetermined users, the training system 200 can do this by, for each of the one of more training pairs included in the generative neural network adaptation data 170 that corresponds to the predetermined user: processing (i) the training input in the training pair and (ii) a soft prompt embedding using the set of base neural network parameters 115 of the generative neural network 110 to generate a predicted output; and then adjusting the soft prompt embedding based at least in part on a difference between the predicted output and the target output.

[0061] The training system 200 can do this by computing, for each training pair, respective gradients of a prompt tuning objective function, e.g., a cross-entropy loss function, that evaluates such a difference with respect to numerical values included in the soft prompt embedding through backpropagation. The training system 200 can then determine the updates by applying an update rule, e.g., an Adam update rule, an Rmsprop update rule, or a stochastic gradient descent (SGD) update rule, to the respective gradients. Like in the finetuning adaptation, the training system 200 can hold the set of base neural network parameters 115 of the generative neural network 110 fixed to their trained values, such that they are not adjusted during the prompt tuning process.

[0062] The training system 200 also obtains classifier model training data 180. The classifier model training data 180 includes, for each of the plurality of predetermined users, one or more training inputs that is associated with the predetermined user. For example, each training input can be a sequence of text, an image, or an audio signal provided by the predetermined user. As another example, each training input can be a sequence of text, an image, or an audio signal that has writing / visual / audio sty les that mimic those of the predetermined user, e.g., that has the attributes that are specific to the predetermined user, e.g., as described in a preference profile of the predetermined user.

[0063] Despite FIG. 2 illustrates the classifier model training data 180 as being separate from the generative neural network adaptation data 170, in some cases the classifier model training data 180 is included as part of the generative neural network adaptation data 170. That is, in some cases, the training system 200 need not obtain additional training data; instead, it will utilize a portion of the generative neural network adaptation data 170 as the classifier model training data 180 to train the classifier model 130. For example, the classifier model training data 180 can include the target outputs included in the training pairs.

[0064] The training system 200 uses the classifier model training data 180 to learn the trained values of the parameters of the classifier model 130 such that it can accurately classify new users.

[0065] More specifically, for each of the plurality of predetermined users, the training system 200 can do this by, for each of the one of more training inputs included in the classifier model training data 180 that corresponds to the predetermined user: processing the training input using the classifier model to generate a score for each of the plurality of predetermined content providers, with each score representing a predicted likelihood that the training input is provided by the predetermined content provider; and then adjusting the parameters of the classifier model 130 based at least in part on the scores generated by the classifier model 130.

[0066] The training system 200 can adjust the parameters of the classifier model 130 by minimizing a classification objective function, e.g., a cross-entropy loss function or a log likelihood loss function, that measures a difference between the scores generated by the classifier model 130 and the ground truth scores for the plurality of predetermined content providers (e.g., a score of 1.0 for a particular predetermined user, and a score of 0.0 for each remaining user in the plurality of predetermined users). For example, when the classifier model 130 is a neural network, the training system 200 can train the classifier model 130 to minimize the classification objective function using gradient descent with backpropagation.

[0067] FIG. 3 is a flow diagram of an example process 300 for generating a new personalized output. For convenience, the process 300 will be described as being performed by a system of one or more computers located in one or more locations. For example, a personalized data generation system, e.g., the personalized data generation system 100 of FIG. 1, appropriately programmed, can perform the process 300.

[0068] The system maintains a respective set of adaptation parameters of a generative neural network (step 302). Each set of adaptation parameters corresponds to a respective one of a plurality of predetermined users of the system. Each set of adaptation parameters can have been learned, e.g., independently of the plurality of generative neural network parameters, by using a fine-tuning adaptation technique or a prompt tuning technique on a set of adaptation data that is specific to a corresponding predetermined user, as described above.

[0069] The system obtains a new input associated with a new user that is not in the plurality of predetermined users (step 304). That is. the new user is different from any one of plurality of predetermined users of the system. The new input can be obtained in any of a variety of ways. For example, the system can receive a conditioning input as part of, or associated with, a request for a personalized output data item from the new user. Such a conditioning input provides context for the personalized output data item. The system can then use the conditioning input to generate the new input.

[0070] As another example, the system can obtain metadata associated with a user session between the new user and the system. Such metadata includes information that characterizes the new user, e.g., information about the new user’s identity, the new user’s social network, social actions, or activities, profession, the new user’s preferences, or the new user’s cunent geographic location. The system can then use the metadata to generate the new input.

[0071] In this example, a user may be provided with controls (e.g., user interface elements with which a user can interact) allowing the user to make an election as to both if and when systems, programs, or features described herein may enable collection of user information (e.g.. information about a user’s social network, social actions, or activities, profession, a user’s preferences, or a user’s current location), and if the user is sent content or communications from a server. In addition, certain data may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user’s identity may be treated so that no personally identifiable information can be determined for the user, or a user’s geographic location may be generalized where location information is obtained (such as to a city, postal code, or state level), so that a particular location of a user cannot be determined. Thus, the user may have control over what information is collected about the user, how that information is used, and what information is provided to the user.

[0072] The system processes a model input using a classifier model to generate a classification output for the new input (step 306). In some cases, the model input is the same as the new input while in other cases, the model input is different from the new input, e.g.. it is a portion of the new input or it includes additional data that is not included in the new input. The classification output can include a respective score for each of the plurality of predetermined users. A score for a particular predetermined user represents a respective likelihood (as computed by the classifier model) that the user which submitted the request is the particular predetermined user.

[0073] The system generates a new set of adaptation parameters based on the classification output and on the respective set of adaptation parameters for each of at least a subset of the plurality of predetermined users (step 308). In some implementations, the new set of adaptation parameters is determined by computing a weighted combination of the respective set of adaptation parameters for each of the plurality of predetermined users. In the weighted combination, each set of adaptation parameters is weighted by a weight that is dependent on the respective score for each of the plurality of predetermined users.

[0074] A specific example of computing a new set of adaptation parameters will now be described. In the following example, the weight for each set of adaptation parameters is equal to the score included in the classification output. That is, each adaptation parameter included in a particular set of adaptation parameters are multiplied by the score included in the classification output for the particular set of adaptation parameters. However, in other examples, the score may be scaled or divided (e.g., by a constant), square rooted, squared (or exponentiated by a number other than 2), normalized, or otherwise modified, and the modified score will be used as the weight.

[0075] In this example, for a new input associated with a new user, the classification output includes the following scores for each of a total of three predetermined users: 0.2 for a first predetermined user (indicating that the likelihood of the new user being the first predetermined user is 0.2), 0.5 for a second predetermined user, and 0.3 for a third predetermined user. Using these scores included the classification output and a respective set of adaptation parameters for each of the three predetermined users, the new set of adaptation parameters for the new user can be computed as: Wnew= 0.2Wuserl+ 0.5WZuserl+ 0.3VIZuser3, where Wuserlrepresents the respective set of adaptation parameters for the first predetermined user.

[0076] The system generates, by the generative neural network and based on the new set of adaptation parameters and the new input, a new personalized output data item (step 310). In some implementations, the new set of adaptation parameters represents an additional set of parameters of the generative neural network, and the generative neural network processes the new input in accordance with (i) the set of base neural network parameters and (ii) the new set of adaptation parameters to generate the new personalized output data item.

[0077] In some implementations, the new set of adaptation parameters represents a soft prompt embedding, and the generative neural network processes (i) the new input and (ii) the soft prompt embedding in accordance with the set of base neural network parameters to generate the new personalized output data item.

[0078] In either implementation, because of the use of the new set of adaptation parameters, the generative neural network is able to generate output data items that are personalized, i.e., includes attributes that are specific to the new user.

[0079] By repeatedly performing iterations of the process 300, the system can generate a new set of adaptation parameters for every new user of the system, and then generating the one or more new personalized outputs for the new users based on the new sets of adaptation parameters. In particular, the system can generate a very large, possibly infinitely large, number of sets of adaptation parameters that correspond respectively to a very large number of different new users of the system based on a fixed number of sets of adaptation parameters that have been maintained by the system.

[0080] The new personalized outputs generated by the system generated as a result of performing the iterations of the process 300 can be used in any of a variety of ways. For example, the system can provide the new personalized outputs for presentation to the user, e.g., as playback, on a client device. As another example, the system can provide the new personalized outputs to another component in the system for further processing. As yet another example, the system can store the new personalized outputs in a data repository for some future purpose.

[0081] FIG. 4 is a flow diagram of an example process 400 for generating a new set of adaptation parameters for a new user. For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a personalized data generation system, e.g., the personalized data generation system 100 of FIG. 1 , appropriately programmed, can perform the process 400. The system maintains a respective set of adaptation parameters of a generative neural network (step 402). Each set of adaptation parameters corresponds to a respective one of a plurality of predetermined users of the system. Each set of adaptation parameters can have been learned, e.g., independently of the plurality of generative neural network parameters, by using a fine-tuning adaptation technique or a prompt tuning technique on a set of adaptation data that is specific to a corresponding predetermined user, as described above.

[0082] The system trains a classifier model on classifier model training data such that the classifier model can generate classification outputs that accurately classify new users (step 404). A classification output can include a respective score for each of the plurality of predetermined users. A score for a particular predetermined user represents a respective likelihood (as computed by the classifier model) that a new user is the particular predetermined user.

[0083] The system uses the classifier model to generate a new set of adaptation parameters for a new user (step 406). Generally, the system can repeatedly perform step 406 to generate different sets of adaptation parameters for different users of the system.

[0084] To do this, the system uses the classifier model to generate a classification output for the new user. The classification output can include a respective score for each of the plurality of predetermined users. In some implementations, for the new user, the system generates the new set of adaptation parameters by computing a weighted combination of the respective set of adaptation parameters for each of the plurality of predetermined users. In the weighted combination, each set of adaptation parameters is w eighted by a weight that is dependent on the respective score for each of the plurality of predetermined users.

[0085] This specification uses the term “configured” in connection with systems and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0086] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine- readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

[0087] The term “data processing apparatus" refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0088] A computer program, which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

[0089] In this specification, the term “database” is used broadly to refer to any collection of data: the data does not need to be structured in any particular way, or structured at all, and it can be stored on storage devices in one or more locations. Thus, for example, the index database can include multiple collections of data, each of which may be organized and accessed differently.

[0090] Similarly, in this specification the term “engine’’ is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Generally, an engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines can be installed and running on the same computer or computers.

[0091] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.

[0092] Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0093] Computer readable media suitable for storing computer program instructions and data include all forms of non volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks.

[0094] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditor}' feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone that is running a messaging application, and receiving responsive messages from the user in return.

[0095] Data processing apparatus for implementing machine learning models can also include, for example, special-purpose hardware accelerator units for processing common and compute-intensive parts of machine learning training or production, i.e., inference, workloads.

[0096] Machine learning models can be implemented and deployed using a machine learning framework, e.g., a TensorFlow framework or a Jax framework.

[0097] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g., the Internet.

[0098] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data, e.g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client. Data generated at the user device, e.g.. a result of the user interaction, can be received at the server from the device.

[0099] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0100] Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0101] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

[0102] What is claimed is:

Claims

CLAIMS1. A method performed by one or more data processing apparatus, the method comprising: maintaining a respective set of adaptation parameters of a generative neural network for each of a plurality of predetermined content providers; obtaining a new input associated with a new content provider that is not in the plurality7of predetermined content providers; using a classifier model to generate a respective score for each of the plurality7of predetermined content providers; generating a new set of adaptation parameters based on determining a weighted combination of the respective set of adaptation parameters for each of a subset of the plurality of predetermined content providers, wherein each set of adaptation parameters is weighted in the weighted combination by7a weight that is dependent on the respective score for each of the plurality of predetermined content providers; and generating, by the generative neural network and based on the new set of adaptation parameters and the new input, a new personalized output data item.

2. The method of claim 1, further comprising: obtaining generative neural network adaptation data comprising, for each of the plurality7of predetermined content providers, one or more training pairs, wherein each training pair comprises a training input and a target output, wherein the target output is associated with the predetermined content provider; and using the generative neural network adaptation data to train the generative neural network to learn the respective set of adaptation parameters for each of plurality7of predetermined content providers, wherein the training comprises, for each of the plurality7of predetermined content providers, and for each of the one of more training pairs: processing the training input using the generative neural network to generate a predicted output; and adjusting the respective set of adaptation parameters based at least in part on a difference between the predicted output and the target output.

3. The method of any one of claims 1-2, further comprising: obtaining classifier model training data comprising training inputs associated with each of the plurality of predetermined content providers; and using the classifier model training data to train the classifier model having a plurality of parameters, wherein the training comprises, for each training input: processing the training input using the classifier model to generate a score for each of the plurality of predetermined content providers, with each score representing a predicted likelihood that the training input is provided by the predetermined content provider; and adjusting the plurality of parameters based at least in part on the scores generated by the classifier model.

4. The method of any one of claims 2-3, wherein for each training pair included in the language model training data, the training input and the target output are associated with different content providers.

5. The method of any one of claims 2-3, wherein for each training pair included in the generative neural network training data, the training input and the target output are associated with a same content provider but have a different length than each other.

6. The method of any one of claims 2-5, wherein the generative neural network adaptation data includes the classifier model training data.

7. The method of any one of claims 1-6, wherein the new input associated with the new content provider comprises text provided by the new content provider.

8. The method of any one of claims 1-6, wherein the new input associated with the new content provider comprises text that characterizes the new content provider.

9. The method of any one of claims 1-8, wherein the generative neural network comprises a set of base neural network parameters, and wherein using the generative neural network adaptation data to train the generative neural network comprises holding the set of base neural network parameters fixed during the training.

10. A system comprising: one or more computers; and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising: maintaining a respective set of adaptation parameters of a generative neural network for each of a plurality of predetermined content providers; obtaining a new input associated with a new content provider that is not in the plurality of predetermined content providers; using a classifier model to generate a respective score for each of the plurality of predetermined content providers; generating a new set of adaptation parameters based on determining a weighted combination of the respective set of adaptation parameters for each of a subset of the plurality of predetermined content providers, wherein each set of adaptation parameters is weighted in the weighted combination by a weight that is dependent on the respective score for each of the plurality of predetermined content providers; and generating, by the generative neural network and based on the new set of adaptation parameters and the new input, a new personalized output data item.

11. The system of claim 10, wherein the operations further comprise: obtaining generative neural network adaptation data comprising, for each of the plurality of predetermined content providers, one or more training pairs, wherein each training pair comprises a training input and a target output, wherein the target output is associated with the predetermined content provider; and using the generative neural network adaptation data to train the generative neural network to learn the respective set of adaptation parameters for each of plurality of predetermined content providers, wherein the training comprises, for each of the plurality of predetermined content providers, and for each of the one of more training pairs: processing the training input using the generative neural network to generate a predicted output; and adjusting the respective set of adaptation parameters based at least in part on a difference between the predicted output and the target output.

12. The system of any one of claims 10-11, wherein the operations further comprise: obtaining classifier model training data comprising training inputs associated with each of the plurality of predetermined content providers; and using the classifier model training data to train the classifier model having a plurality of parameters, wherein the training comprises, for each training input: processing the training input using the classifier model to generate a score for each of the plurality of predetermined content providers, with each score representing a predicted likelihood that the training input is provided by the predetermined content provider; and adjusting the plurality of parameters based at least in part on the scores generated by the classifier model.

13. The system of any one of claims 11-12, wherein for each training pair included in the language model training data, the training input and the target output are associated with different content providers.

14. The system of any one of claims 11-12, wherein for each training pair included in the generative neural network training data, the training input and the target output are associated with a same content provider but have a different length than each other.

15. The system of any one of claims 11-14, wherein the generative neural network adaptation data includes the classifier model training data.

16. The system of any one of claims 10-15, wherein the new input associated with the new content provider comprises text provided by the new content provider.

17. The system of any one of claims 10-15, wherein the new input associated with the new content provider comprises text that characterizes the new content provider.

18. The system of any one of claims 10-17, wherein the generative neural network comprises a set of base neural network parameters, and wherein using the generative neural network adaptation data to train the generative neural network comprises holding the set of base neural network parameters fixed during the training.

19. One or more computer storage media storing instructions that when executed by one or more computers cause the one more computers to perform operations comprising: maintaining a respective set of adaptation parameters of a generative neural network for each of a plurality of predetermined content providers; obtaining a new input associated with a new content provider that is not in the plurality of predetermined content providers; using a classifier model to generate a respective score for each of the plurality of predetermined content providers; generating a new set of adaptation parameters based on determining a weighted combination of the respective set of adaptation parameters for each of a subset of the plurality of predetermined content providers, wherein each set of adaptation parameters is weighted in the weighted combination by a weight that is dependent on the respective score for each of the plurality of predetermined content providers; and generating, by the generative neural network and based on the new set of adaptation parameters and the new input, a new personalized output data item.

20. The computer storage media of claim 19, wherein the operations further comprise: obtaining generative neural network adaptation data comprising, for each of the plurality of predetermined content providers, one or more training pairs, wherein each training pair comprises a training input and a target output, wherein the target output is associated with the predetermined content provider; and using the generative neural network adaptation data to train the generative neural network to learn the respective set of adaptation parameters for each of plurality of predetermined content providers, wherein the training comprises, for each of the plurality of predetermined content providers, and for each of the one of more training pairs: processing the training input using the generative neural network to generate a predicted output; and adjusting the respective set of adaptation parameters based at least in part on a difference between the predicted output and the target output.

Images