Adapting generative models to new contextual data using compressed representations

Abstract

The invention relates to adapting a generative model to new contextual data using compressed representations. Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing tasks. In one aspect, a method includes receiving a query of a task to be performed; receiving a plurality of contextual content items for the task; for each content item of the plurality of content items, processing an input comprising a representation of the content item using a trained compression model to generate a compressed representation of the content item, the compressed representation comprising one or more vectors having a fixed size; using the compressed representations to generate an aggregated compressed representation comprising one or more vectors, the aggregated compressed representation representing the plurality of content items; and processing an input comprising (i) the query and (ii) the aggregated compressed representation using a generative neural network to generate a response to the query.

Description

Technical Field

[0001] This manual relates to using machine learning models to process data. Background Technology

[0002] Machine learning models receive input and generate outputs, such as predicted outputs, based on the received input. Some machine learning models are parametric models, and they generate outputs based on the received input and the values ​​of the model parameters.

[0003] Some machine learning models are deep models, which use multiple layers to generate outputs from received inputs. For example, a deep neural network is a type of deep machine learning model that includes an output layer and one or more hidden layers, each of which applies a non-linear transformation to the received input to generate an output. Summary of the Invention

[0004] This specification generally describes a system implemented as a computer program on one or more computers at one or more locations, which can use a compression model to generate an aggregate compressed representation of context data items (e.g., text documents, images, audio files, videos, etc.) associated with a task (e.g., a retrieval task, a generation task, etc.) and then use that compressed representation to perform the task.

[0005] For example, the system can receive new contextual data items for the task after the generative model has been trained.

[0006] For example, the task could be a personalization task that requires personalizing the output of a generative model with new user data. This personalization task could be, for example, a personalized skills learning task, where the generative model generates responses that match the user's style or preferences (e.g., generating images personalized based on the user's image, generating text documents personalized based on text snippets written by the user, generating audio files personalized based on audio recordings of the user's voice, generating videos personalized based on videos depicting the user, generating videos personalized based on videos created by the user, etc.). As another example, the personalization task could be a knowledge-intensive question-answering task, where the generative model generates responses based on the user's specific knowledge (e.g., generating responses to questions about the user's relationship with another individual, personalized based on images depicting the user and another individual; generating responses to academic questions about the user, personalized based on text documents describing the academic courses the user has taken; generating responses to questions about the user's friendships, personalized based on audio files depicting interviews with the user; etc.). New contextual data items could be, for example, text snippets generated by the new user, audio recordings of the new user's voice, documents describing the new user's memories, etc. Incorporating new contextual data items into the context of the generative model enables the generative model to generate more accurate and personalized responses.

[0007] In one aspect, a method executed by one or more computers is provided, the method comprising: receiving a query for a task to be performed; receiving a plurality of contextual content items for the task; for each of the plurality of content items, processing an input including the content item using a trained compression model to generate a compressed representation of the content item, the compressed representation including one or more vectors of a fixed size; using the compressed representations to generate an aggregate compressed representation including one or more vectors, the aggregate compressed representation representing the plurality of content items; and using a generative neural network to process an input including (i) the query and (ii) the aggregate compressed representation to generate a response to the query.

[0008] When the system receives a new contextual content item relevant to the task (i.e., for personalization or due to an incomplete dataset), the system can use a trained compression model to generate a new compressed representation of the new content item. The system can then use this new compressed representation to update the aggregated compressed representation.

[0009] The subject matter described in this specification can be implemented in specific embodiments to achieve one or more of the following advantages.

[0010] By using a trained compression model to compress task-related context data items into an aggregated, compact representation of all context data items, the system described in this specification enables generative models to adapt to new context data. Using the aggregated, compact representation of context data allows for efficient adaptation of the generative model to context data without requiring retraining or fine-tuning. Furthermore, using the "compressed" representation allows for computationally efficient inference, as the input to the generative model only needs to include the compressed representation and not the original context data items.

[0011] When the system receives a new context item, it can process only the new context data item to incorporate it into the aggregated compact representation, without reprocessing the context data items already represented by the aggregated compact representation. The generative model then uses the updated aggregated compact representation, which represents both the previous and new context data, to incorporate the new context data during task execution.

[0012] A common approach to adapting generative models to new data is to integrate the new context data through prompts—that is, by including the new context data as part of a sequence of prompts that the generative model processes as input. Integrating new context data through prompts incurs high inference costs because incorporating the new context data significantly increases the length of the prompts, and latency and computational costs increase with the length of the prompts. In contrast, this specification describes a technique for adapting generative models to context data using an aggregated, compact representation of the context data. This method reduces the length of the prompts for the generative model, thereby reducing computational costs during inference and enabling resources to be efficiently adapted to new context data. Specifically, during inference, latency can be a major problem in generative neural network inference services such as LLM services. By significantly reducing the length of the prompts that need to be encoded, the described technique significantly reduces the latency required for generative neural network inference. Furthermore, the smaller compressed representation can be stored along with the generative neural network, further reducing the additional latency required to incorporate additional context from context items.

[0013] Another common approach to adapting generative models to new data is to modify the model, i.e., by updating its existing weights or introducing new trainable weights. Modifying generative models incurs high training costs. In contrast, this specification describes a technique for adapting generative models to contextual data using aggregated compact representations of the contextual data. This method eliminates the need for a specialized training process for each new contextual data item, thus reducing training costs.

[0014] By using a merging process to combine the representations of each context data item received by the system, the system can generate an aggregated compact representation. The merging process can include order-independent aggregation operations, such as mean pooling, to eliminate position-dependent biases related to the context data items. The shape of the aggregated compact representation is independent of the number of context data items it represents, thereby reducing the computational cost of incorporating new context data items. The merging process enables the system to efficiently generate and update aggregated compact representations as new context data items are added.

[0015] The system can add new context data items to the aggregated compact representation without changing the shape of the compact representation. This allows the system to reduce inference costs while adapting the generative model to the new context data items. The system can maintain a consistent cue length regardless of the number of context items.

[0016] Details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of this subject matter will become apparent from the description, drawings, and claims. Attached Figure Description

[0017] Figure 1 This is a block diagram of an example response generation system.

[0018] Figure 2 This is a flowchart of an example process for performing response generation.

[0019] Figure 3 This is a flowchart of an example process for generating a compressed representation of a context data item.

[0020] Figure 4 This is a flowchart of an example process for training a compressed model for a response generation system.

[0021] Figure 5 An example architecture for a response generation system is shown.

[0022] Figure 6 An example architecture of a compressed neural network for a compressed model is shown.

[0023] Figure 7 An example trade-off between cost and quality in response generation tasks is shown.

[0024] The same reference numerals and names in the various figures indicate the same elements. Detailed Implementation

[0025] Figure 1An example response generation system 100 is shown. Response generation system 100 is an example of a system implemented as a computer program on one or more computers at one or more locations, in which the systems, components and techniques described below are implemented.

[0026] The response generation system 100 includes a compression model 106, an aggregator 110, and a generative neural network 116. The response generation system 100 is configured to process an input query 102 and one or more context data items 104a-c to generate a response 122.

[0027] The response generation system 100 can receive queries for tasks to be performed. Query 102 can be, for example, a request for a response that matches the user's style or preferences or a knowledge-intensive question. For example, the query could be a request for a text description or summary of a content item or a request for an answer to a question based on a provided context item.

[0028] In some examples, the task can be a personalization task that requires personalizing the output of the generative model with specific user data. This personalization task could be, for example, a personalized skill learning task, where the generative model generates responses that match the user's style or preferences (e.g., generating images personalized based on the user's image, generating text documents personalized based on text snippets written by the user, generating audio files personalized based on audio recordings of the user's speech, generating videos personalized based on videos depicting the user, generating videos personalized based on videos created by the user, etc.).

[0029] In some examples, the personalization task can be a knowledge-intensive question-answering task, where the generative model generates responses based on the user's knowledge (e.g., generating responses to questions about the user's relationship with another individual, personalized based on images depicting the user and the other individual; generating responses to academic questions about the user, personalized based on text documents describing the academic courses the user has taken; generating responses to questions about the user's friendships, personalized based on audio files depicting interviews with the user; etc.).

[0030] The response generation system 100 can receive one or more contextual content items 104a-c for the task. Content items 104a-c can be, for example, images, text documents, audio files, videos, or combinations thereof.

[0031] For example, in personalized skills learning tasks—such as paraphrasing documents to match a user's writing style—content items 104a-c could be documents previously written by the user. For instance, to generate a summary of content items in a specific author's style, the content item could be an article-title pair previously created by that author. Content items 104a-c can provide examples that can be used to personalize tasks by inferring user style or preferences from the user's past work.

[0032] In some examples, for knowledge-intensive question-answering tasks, content items 104a-c can be personalized knowledge bases tailored to a specific user. For instance, if a user queries about the relationship between two individuals, content items 104a-c could be text documents or images describing the interactions between those two individuals. System 100 can determine specific information from content items 104a-c, including, for example, verbatim details, to generate an accurate response based on the knowledge contained in those content items.

[0033] For each content item 104a-c, compression model 106 can process the input including that content item to generate a compressed representation 108a-c for that content item. The compressed representation 108a-c can represent the content item in a compact form designed for subsequent processing. This compressed representation can summarize relevant information from the original content item into a fixed-size format.

[0034] The compressed representation of content items 108a-c may include one or more vectors of a fixed size. The vectors can be compact representations of the content items. In some implementations, the vectors may have a fixed size corresponding to the number of embeddings, each with the same fixed dimension, for example, 4, 8, 16, 32, 64, or 128 embeddings, each with the same dimension, for example, all having 256, 512, or 1024 entries. Regardless of size, each content item can be mapped to the same number of embeddings. This allows for consistent processing regardless of the length of the original content items. By maintaining a uniform size, these vectors facilitate efficient aggregation and subsequent processing by other components of system 100.

[0035] Compression model 106 may include a compressed generative neural network. In some examples, compression model 106 may include a corresponding compressed generative neural network for each type of content item 104a-c. For example, compression model 106 may include a compressed generative neural network for processing text and a compressed generative neural network for processing images. Compression model 106 may be a recurrent neural network, a self-attention neural network, or a neural network including both recurrent layers and self-attention layers.

[0036] A neural network layer may include one or more self-attention layers. A self-attention layer may be configured to process its input by computing attention scores. These attention scores represent the correlation between different parts of the input. The neural network can use these attention scores to compute a weighted sum of the input representations. This allows the layer to focus on information associated with higher attention scores, such as more relevant information. The generative neural network is described in more detail below. To generate compressed representations 108a-c of content items 104a-c, a compression model 106 may attach one or more compressed embeddings to the lexical representations of the content items. A response generation system 100 may generate lexicalized versions of content items 104a-c. The response generation system 100 may lexicalize content items 104a-c into sequences of lexical units.

[0037] In some implementations, lexical units can represent text in natural or computer language, such as words, phrases, or characters. For example, text can be received as a series of encoded characters (e.g., UTF-8 encoded characters); such "characters" can include Chinese characters and other similar characters, as well as logograms, syllabograms, etc. A text encoder (i.e., a lexical analyzer) can process the text sequence to represent the text as a series of text lexical units in a vocabulary, where each text lexical unit represents a word, phrase, or character in natural or computer language. The computer language can be any form of language used for communicating with a computer, such as a markup language, or a command or configuration language, or a data exchange language (such as JSON), or a programming language. For example, a lexical analyzer can implement BPE (byte-pair encoding) or phrase lexicalization. Optionally, text can be obtained from audio data representing speech; the output lexical units can be converted to audio data.

[0038] Alternatively or concurrently, lexical units can represent images. For example, a set (sequence) of input or output lexical units can represent an image. Each image lexical unit can include a block encoding of the values ​​of pixels in different regions of the image, which maps the set of pixel values ​​to the corresponding image lexical unit. The block encoder can include, for example, a neural network with one or more (self)attention layers, such as a Transformer neural network.

[0039] Alternatively, lexical units can represent audio waveforms. For example, a set (sequence) of input or output lexical units can represent audio data that represents waveforms, such as instantaneous audio amplitude values ​​or time-frequency audio data. Each image lexical unit can include block encoding of the audio waveform at different time intervals of the audio, which maps the set of values ​​representing the audio waveform to the corresponding image lexical unit. The block encoder can include, for example, a neural network with one or more (self)attention layers, such as a Transformer neural network. In a multimodal system, audio data or images can be labeled with audio start lexical units or image start lexical units.

[0040] The compressed neural network can process a sequence of lexical terms representing content items 104a-c and one or more compressed embeddings. These compressed embeddings can be special compressed lexical terms attached to the input sequence of lexical terms representing the content items. A combined sequence of both compressed embeddings and content item lexical terms can be provided as input to the compressed neural network.

[0041] A compressed neural network can use one or more compressed embeddings generated by the last neural network layer in a compressed generative neural network as a compressed representation of a content item. After processing by the neural network layers, the compressed embeddings encapsulate information from the content item. The final layer representation of the compressed embeddings can be extracted as the output of the compressor. The system can then use the extracted representation of the embeddings as a compressed representation of the content item.

[0042] This allows the network to insert salient information from content items into the representation associated with the compressed embedding. The final layer representation of the compressed lemmas can encapsulate the semantic information of the content item. By focusing the processing power of the neural network on these specific compressed lemmas, the system 100 can generate concise and representative vectors for each content item.

[0043] Compressed embeddings can be learned during the training process. See below for reference. Figure 4 The training of the compressed model 106 is described in more detail.

[0044] Aggregator 110 can use compressed representations 108a-c to generate aggregated compressed representation 112. Aggregated compressed representation 112 can include one or more vectors representing content items 104a-c. The aggregated compressed representation can represent the combined information from all content items 104a-c in a single, compact format. This aggregated representation can be created by combining the individual compressed representations 108a-c of each content item. Aggregated compressed representation 112 can encapsulate relevant information from the entire set of content items 104a-c. The resulting single representation can then be processed efficiently.

[0045] In some examples, aggregator 110 can use order-independent aggregation operations to generate an aggregated compressed representation 112. Aggregator 110 can apply order-independent aggregation operations to the compressed representations 108a-c of the content items. Order-independent aggregation operations can be, for example, mean pooling. Mean pooling calculates a single aggregate vector by averaging the corresponding components of each compressed representation vector. By using an order-independent method, the resulting aggregated compressed representation 112 is not biased by the position or order of the processed content items. The aggregated compressed representation 112 remains consistent regardless of the arrangement of the input content items.

[0046] The aggregated compressed representation 112 can have a fixed size, for example, the same size as the compressed representations 108a-c. This is because the shape of the aggregated compressed representation is independent of the number of content items from which it is generated. Since order-independent aggregation operations produce representations of a consistent shape regardless of the number of input representations, the shape of the aggregated compressed representation can be maintained even when new compressed representations are incorporated. For example, the aggregated compressed representation can have a fixed size corresponding to 4, 8, 16, 32, 64, or 128 embeddings. Regardless of the number of content items represented by the aggregated compressed representation, it can be mapped to a fixed number of embeddings. For example, the aggregated compressed representation 112 representing two content items can have the same number of embeddings as the aggregated representation representing five content items.

[0047] The generative neural network 116 can process input 114 to generate a response 122 to query 102. The generative neural network can be configured to perform a generative task to generate data as output, including, for example, text data, image data, video data, audio data, or multimodal data including data in two or more different modalities.

[0048] In some cases, a generative neural network can be called an autoregressive neural network when it regressively generates a sequence of output words as its output. More specifically, the output generated by autoregression is created by generating each specific word in the output sequence under the conditions of the current input sequence, which includes the input sequence included in the network input and any words in the output sequence preceding the specific word—that is, any words that have been previously generated for any position in the output sequence preceding the specific word.

[0049] Generative neural networks can, for example, be any of a variety of Transformer-based neural network architectures, such as encoder-only Transformer architecture, encoder-decoder Transformer architecture, decoder-only Transformer architecture, diffusion Transformer architecture, other attention-based architectures, and so on.

[0050] Examples of such Transformer-based neural network architectures include those described in the following literature: Colin Raffel, et al., Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683, 2019; Daniel Adiwardana, et al., Towards a human-like open-domain chatbot. CoRR, abs / 2001.09977, 2020; Aakanksha Chowdhery, et al., PaLM: Scaling Language Modeling with Pathways. arXiv preprint arXiv:2204.02311 (Aakanksha Chowdhery et al., "PaLM: Modeling with Path Extension Language", arXiv preprint arXiv:2204.02311); Rohan Anil, et al., Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023 (Rohan Anil et al., "Palm 2 Technical Report", arXiv preprint arXiv:2305.10403, 2023); and Gemini Team, et al., Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023) (GeminiTeam et al., "Gemini: A Family of Highly Capable Multimodal Models", arXiv preprint arXiv:2312.11805 (2023)).

[0051] For example, a generative neural network can be a generative language model neural network. Examples of generative language model neural networks include Sparrow (Glaese et al., arXiv:2209.14375), Chinchilla (Hoffmann et al., arXiv:2203.15556), and PaLM 2 (Anil et al., arXiv:2305:10403). As another example, a generative neural network can be a multimodal model neural network, such as a visual language model (VLM) neural network. Examples of multimodal neural networks include Flamingo (Alayrac et al., arXiv:2204.14198), PaLI (Chen et al., arXiv:2209.06794), and PaLI-X (Chen et al., arXiv:2305.18565 Top of Form).

[0052] As another example, a generative neural network can be a base model neural network. A base model neural network is a large-scale machine learning model trained on a broad dataset that can be adapted and fine-tuned for a variety of applications and downstream tasks. Examples of base model neural networks include Imagen (Saharia et al., arXiv:2205.11487) and Parti (Yu et al., arXiv:2206.10789).

[0053] When a generative neural network is implemented using one of the architectures described below or other known architectures, some examples of machine learning tasks—including generative tasks—that it can be configured to perform are as follows.

[0054] In some cases, a generative neural network is a neural network configured to perform image processing tasks, i.e., receiving network input including an image and processing the image to generate a network output for the input image. For example, the task could be image classification, and the output generated by the neural network for a given image could be a score for each category in a set of object categories, where each score represents an estimated probability that the image contains an object belonging to that category. As another example, the task could be image embedding generation, and the output generated by the neural network could be a digital embedding of the input image. As yet another example, the task could be object detection, and the output generated by the neural network could identify the location of objects depicting a specific type in the input image. As yet another example, the task could be image segmentation, and the output generated by the neural network could assign each pixel of the input image to a category in a set of categories. In some other cases, a neural network is a neural network configured to perform image generation tasks, where the input is a conditional input and the output is an input sequence of pixel intensity values ​​of the image.

[0055] As an example, the task could be a neural machine translation task. For instance, if the network input to a generative neural network is a sequence of text in one language—such as a sequence of words, phrases, characters, or word fragments—the network output generated by the neural network could be a translation of the text sequence into another language; that is, a sequence of text in another language that serves as the translation of the input text sequence. The vocabulary of input lexical units could be words, word fragments, or characters in the first language, while the vocabulary of output lexical units could be words, word fragments, or characters in another language. As a specific example, the task could be a multilingual machine translation task, where a single neural network is configured to translate between multiple different source-target language pairs. In this example, the source language text could be augmented with an identifier that instructs the neural network to translate the source language text into the target language.

[0056] Some implementations can be used for automatic code generation. For example, input lexical units can represent words, phrases, or characters in a first natural language, and output lexical units can represent instructions in a computer programming or markup language, or instructions used to control an application to perform tasks—such as building data items like images or web pages.

[0057] As another example, the task could be an audio processing task. For instance, if the input to a generative neural network is a sequence representing spoken utterances, the output generated by the network could be a score for each text segment in a set of text segments, each score representing the estimated probability that the text segment is a correct transcription of the utterance. As another example, if the input to a neural network is a sequence representing spoken utterances, the output generated by the network could indicate whether a particular word or phrase (“hot word”) was spoken in that utterance. As yet another example, if the input to a generative neural network is a sequence representing spoken utterances, the output generated by the network could be a classification of the spoken utterances into one of several categories, such as the recognition of the natural language of spoken utterances.

[0058] As another example, a task can be a natural language processing or understanding task that operates on a sequence of texts in a natural language, such as entailment tasks, paraphrasing tasks, text similarity tasks, sentiment tasks, sentence completion tasks, grammar tasks, etc.

[0059] As another example, the task could be a text-to-speech task, where the network input is text in natural language or features of text in natural language, and the network output is a spectrogram, waveform, or other data that defines the audio of the text spoken in natural language.

[0060] As another example, the task could be a health status prediction task, where the network input is a sequence derived from a patient's electronic health record data, and the network output is a prediction related to the patient's future health status, such as a predicted treatment to be used for the patient, the likelihood of the patient experiencing an adverse health event, or a predicted diagnosis for the patient. For example, such electronic health data could include one or more sequences of physiological data obtained from a patient, where the output is a corresponding prediction associated with those data sequences. Examples of physiological data and corresponding predictions include: blood glucose measurements, where the prediction is a predicted future blood glucose measurement or a prediction of hyperglycemia or hypoglycemia events; heart rate, where the prediction is the presence or absence of heart disease or future cardiac events; blood pressure measurements, where the prediction is the risk of future heart disease; and so on.

[0061] As another example, the task could be a text generation task, where the network input is a sequence of text and the network output is another sequence of text, such as completion of the input text sequence, a response to a question posed in the input sequence, or a sequence of text about a topic specified by a first text sequence. As yet another example, the network input for a text generation task could be input other than text, such as an image, and the network output sequence could be text describing the network input.

[0062] In some implementations, the input sequence represents the data to be compressed, such as image data, text data, audio data, or any other type of data; and the output sequence is the compressed version of the data. Input and output terms can each include any representation of the data to be compressed / compressed, such as symbols or embeddings generated / decoded by the corresponding neural network.

[0063] As another example, the task can be an agent-controlled task, where the network input is a sequence of observations or other data characterizing the state of the environment, and the network output defines the action to be performed by the agent in response to the latest data in the sequence. The agent can be, for example, a real-world or simulated robot, a control system for an industrial facility, or a control system that controls different kinds of agents. Observations can include sensor data captured by sensors associated with the agent (e.g., a part of the agent), such as visual data, LiDAR (Light Detection and Ranging) data, sonar data, agent configuration data (e.g., joint angles), agent orientation data, etc.

[0064] In some implementations, the environment is a real-world environment, and the agent is a mechanical (or electromechanical) agent that interacts with the real-world environment. Examples include robots that operate or navigate through the environment, or autonomous or semi-autonomous land, air, or sea vehicles. The actions are performed by the mechanical agent within the real-world environment to carry out the task. For instance, the agent could be a robot that interacts with the environment to complete a specific task (e.g., locating or manipulating an object of interest within the environment, moving the object of interest to a designated location within the environment, or navigating to a designated destination within the environment).

[0065] In these implementations, observation can include one or more of the following: images, object position data, and sensor data, to capture observations as the agent interacts with its environment, such as sensor data from images, distance or position sensors, or from actuators. For example, in the case of a robot, observation can include data characterizing the robot's current state, such as one or more of the following: joint positions, joint velocities, joint forces, torque or acceleration (e.g., gravity-compensated torque feedback), and the global or relative pose of an object held by the robot. In the case of a robot or other mechanical agent or vehicle, observation can similarly include one or more of the following: the position, linear or angular velocity, force, torque or acceleration, and global or relative pose of one or more parts of the agent. Observation can be defined in one, two, or three dimensions and can be absolute and / or relative. Observation can also include, for example, sensed electronic signals, such as motor current or temperature signals; and / or image or video data, such as data from sensors of the agent or from sensors located separately from the agent in its environment.

[0066] In these implementations, actions can be control signals for controlling a robot or other mechanical agent, such as torques or higher-level control commands for the robot's joints; or control signals for controlling autonomous or semi-autonomous land, air, or sea vehicles, such as torques or higher-level control commands for controlling surfaces or other control elements (e.g., steering control elements of the vehicle). Control signals can include, for example, position, velocity, or force / torque / acceleration data for one or more joints of a robot or parts of another mechanical agent. Control signals can also, or alternatively, include electronic control data, such as motor control data, or more generally, data for controlling one or more electronic devices within an environment, the control of which affects the observed state of the environment. For example, in the case of autonomous or semi-autonomous land, air, or sea vehicles, control signals can define actions for controlling navigation (e.g., steering) and movement (e.g., braking and / or acceleration of the vehicle).

[0067] In some implementations, the environment is a simulation of the aforementioned real-world environment, and the agent is implemented as one or more computers interacting with the simulated environment. For example, a system implementing a generative neural network can be used to select actions in the simulated environment during the system's training or evaluation, and after training or evaluation, or both, the action selection strategy can be deployed to control a real-world agent in a specific real-world environment that is the object of the simulation. This avoids unnecessary depletion and damage to the real-world environment or the real-world agent, and allows the control neural network to be trained and evaluated in situations where it is rare, difficult, or unsafe to recreate in the real-world environment. For example, the system can be partially trained using a simulation of a mechanical agent in a simulation of a specific real-world environment, and then deployed to control a real mechanical agent in that specific real-world environment. Thus, in such cases, the observation of the simulated environment is related to the real-world environment, and the selected actions in the simulated environment are related to the actions to be performed by the mechanical agent in the real-world environment.

[0068] In some implementations, as mentioned above, the intelligent agent may not include a human (e.g., it is a robot). Instead, in some implementations, the intelligent agent includes a human user—a digital assistant—such as a smart speaker, smart display, or other device. Information defining the task can then be obtained from the digital assistant, and the digital assistant can be used to guide the user based on the task.

[0069] For example, a system implementing a generative neural network can output instructions to a human user via a digital assistant, instructing the user on actions to be performed at each of multiple time steps. For instance, the instructions can be generated in natural language (as audio and / or text transmitted on screen) based on actions selected by the system. The system selects actions such that they contribute to the performance of the task. A monitoring system (e.g., a camera system) can be provided to monitor the actions actually performed by the user at each time step (if any), in case (e.g., due to human error) differ from the actions the system instructed the user to perform. Using the monitoring system, the system can determine whether the task has been completed. The system can identify actions that the user has incorrectly performed with a certain probability. If so, the system can warn the user to be careful when instructing the user to perform such identified actions. Alternatively or additionally, the system can learn not to instruct the user to perform identified actions, i.e., actions that the user is likely to perform incorrectly.

[0070] More generally, a digital assistant guiding a user may include receiving an assistance request from the user at the digital assistant, and in response to that request, determining a series of tasks the user needs to perform, such as steps or subtasks of an overall task. Then, for one or more tasks in this series, such as for each task up to the final task in the series, the digital assistant may output instructions to the user regarding the task to be performed—e.g., steps or subtasks. This can be done using natural language, for example, on a display and / or using the digital assistant's speech synthesis subsystem. Visual (e.g., video) and / or audio observations of the user performing the task may be captured, for example, using the digital assistant. The system described above can then be used to determine whether the user has successfully completed the task—e.g., a step or subtask, i.e., based on the answer as previously described. If there are further tasks to complete, in response, the digital assistant may advance to the next task in the series (if any), for example, by outputting instructions for the next task to be performed. In this way, the user can be guided step-by-step through a series of tasks to perform the overall task. During the training of a generative neural network, training rewards can be generated, for example, from video data representing examples of the overall task (if a corpus of such data is available) or from simulations of the overall task.

[0071] In a further aspect, a digital assistant device including the system described above is provided. The digital assistant may also include a user interface enabling the user to request assistance and output information. In an implementation, this is a natural language user interface and may include a keyboard, a voice input / output subsystem, and / or a display. The digital assistant may further include an auxiliary subsystem configured to determine a series of tasks to be performed by the user in response to a request. In an implementation, this may include generating a (large-scale) language model, particularly for dialogue, such as a conversational agent like Sparrow (Glaese et al., arXiv:2209.14375) or Chinchilla (Hoffmann et al., arXiv:2203.15556). The digital assistant may have an observation capture subsystem for capturing visual and / or audio observations of the user performing tasks; and an interface for the aforementioned language model neural network (which may be implemented locally or remotely). The digital assistant may also have an auxiliary control subsystem configured to assist the user. The auxiliary control subsystem may be configured to perform the above steps for, for example, one or more tasks in a series of tasks—e.g., up to the final task in the series. More specifically, the auxiliary control subsystem outputs instructions to the user regarding the task to be performed, and the observation capture subsystem captures the user's visual or audio observations of the task performance, determining from the above results whether the user has successfully completed the task. In response, the digital assistant can move on to the next task in the series and / or control the digital assistant, for example, to stop capturing observations.

[0072] As another example, the task could be a genomics task, where the network input is a sequence representing a fragment of a DNA sequence or other molecular sequence, and the network output is an embedding of a fragment for use in downstream tasks (e.g., by using unsupervised learning techniques on a dataset of DNA sequence fragments) or an output specific to a downstream task. Examples of downstream tasks include promoter site prediction, methylation analysis, prediction of the functional effects of non-code processing variants, and so on.

[0073] In some cases, a machine learning task is a combination of multiple individual machine learning tasks; that is, the system is configured to perform multiple different individual machine learning tasks, such as two or more of the machine learning tasks mentioned above. For example, a system may be configured to perform multiple individual natural language understanding tasks, where the network input includes identifiers of the individual natural language understanding tasks to be performed on the network input.

[0074] In some cases, machine learning tasks are multimodal processing tasks that require handling multimodal data. Generally, multimodal data is a combination of two or more different types of data—for example, two or more of audio data, image data, text data, or graphical data. As an example, multimodal data can include audiovisual data, comprising a combination of pixels from an image or video and audio data representing values ​​of a digitized audio waveform. As another example, multimodal data can include a combination of i) text data representing text in natural language, and ii) pixels from an image or video or audio data representing values ​​of an audio waveform. Optionally, but not necessarily, different types of data can be represented using different modalities (types) to represent the same or overlapping objects, and when processing multimodal data, the data can be mapped to a common embedding space.

[0075] As a specific example, the task is a multimodal processing task that requires handling both text input and image input, such that the neural network includes both computer vision neural networks and text processing neural networks. That is, the target output generated by the computer vision neural network for a given image depends on one or more outputs generated by the text processing neural network for one or more corresponding text inputs (and vice versa). Examples of such tasks include open-vocabulary image classification, open-vocabulary object detection, image captioning, text-based image search, image-based retrieval, etc.

[0076] More generally, a multimodal processing task can correspond to any task previously described for any type of data that constitutes a multimodal combination. For example, when a task is applied to multimodal data that combines data for which a task has been previously described with another type of data, the accuracy of the previously described task can be improved. For example, when processing multiple different types (modalities) of data, the detection or classification of objects or events can be improved.

[0077] Specifically, generative neural networks can receive network inputs and generate network outputs for a variety of different machine learning tasks. Generally, two machine learning tasks are considered different if they have different expected outputs for the inputs received for that task. For example, if each image classification task requires classifying an input image into different object categories, the two tasks can be different. As another example, if two robot learning tasks require generating outputs that define the actions a robot should perform to achieve two different goals, then the two robot learning tasks can be different.

[0078] In fact, for any of these examples, the task to be performed by the neural network can be defined by at least a portion of the network input received by the neural network, such as in the form of prompts or requests. In other words, when the appropriate prompts or requests are received, the neural network will be able to perform any of these tasks.

[0079] The generative neural network 116 can be a large language model (LLM). In some examples, the generative neural network 116 can be a pre-trained model whose weights remain fixed during the training of the compressed model 106. The generative neural network can be configured to process an aggregated compressed representation in the form of soft cues to generate a response 122, which is conditioned on both the query and the aggregated compressed representation 112 of the content item. The generative neural network 116 can have the same underlying architecture as one or more compressed neural networks of the compressed model 106, and in some configurations, it can share weights with one or more compressed neural networks to optimize resource efficiency.

[0080] Input 114 may include query 102 and aggregated compressed representation 112. Response generation system 100 can form input 114 by concatenating the aggregated compressed representation 112 with the embedding of query 102. Response generation system 100 may lexicalize the query to generate an embedding of query 102. The embedding of the query may include a corresponding embedding for each lexical in query 102. For example, a lexical may represent a word in the query. This combined input 114 is then fed into generative neural network 116. Generative neural network 116 then produces a response 122 based on this combined input 114.

[0081] The response 122 can be any form of data generated by the generative neural network 116 based on the task defined by query 102 and the context provided by the aggregated compressed representation 112. For example, the response 122 can be an audio file, video, text document, or image.

[0082] For example, in a personalized skills learning task, the response could be a text document, image, audio file, or video that reflects the user's style or preferences determined from content item 104a-c. For instance, if the task is to paraphrase a social media post, response 122 would be a paraphrase of the post that matches the writing style shown in a previous post provided as a content item.

[0083] For knowledge-intensive question-answering tasks, response 122 can be the answer to query 102, formulated based on information within content items 104a-c. For example, if the query inquires about the relationship between two individuals, and the content item is a document describing their interaction, then response 122 could be a textual answer summarizing that relationship. In some examples, response 122 is not limited to text and can take other forms, such as images of a specific style given as a set of example images as a context item, or a segment of audio mimicking a specific voice from an audio sample provided as a context.

[0084] In some examples, the response generation system 100 can receive new context items for the task. For example, new context items can become available after the initial set of context items has been processed. In personalization tasks, this can happen when a user generates new content, such as writing a new social media post, creating a new document, or capturing or uploading a new image. In knowledge-intensive question-answering tasks, new context items can be added to the knowledge base to provide updated or more complete information. The response generation system 100 can incorporate such new items without reprocessing previously received context items. This allows for efficient updating of the context information used by the generative neural network 116.

[0085] Compression model 106 can process input including new content item 118 to generate a new compressed representation 120 of the new content item. The new compressed representation 120 can include one or more vectors. The vectors can have a fixed size, for example, the same size as the previously generated compressed representations 108a-c. The new compressed representation 120 can represent the new content item in a compact form suitable for subsequent processing.

[0086] The response generation system 100 can update the aggregated compressed representation 112 using the new compressed representation 120. The aggregator 110 can incorporate the new compressed representation 120 into the aggregated compressed representation 112. For example, in an implementation using mean pooling, the aggregator 110 can update the aggregated compressed representation by recalculating the mean based on the previous and new representations without reprocessing the original content items. The updated representation can be calculated using a weighted average of the previous aggregated representation and the new compressed representation, based on the corresponding counts of the content items represented by these representations.

[0087] Once aggregator 110 updates aggregated compressed representation 112 with the new compressed representation 120, generative neural network 116 can process the new input to generate a new response to query 102. The new input may include query 102 and the updated aggregated compressed representation. The new response can be generated based on the context from both the original content items 104a-c and the new content item 118. This allows response generation system 100 to efficiently adapt to new information without reprocessing the entire set of content items.

[0088] Figure 2 This is a flowchart of an example process for performing response generation. For convenience, process 200 will be described as being performed by a system of one or more computers located in one or more locations. For example, a commentary generation system appropriately programmed according to this specification—e.g. Figure 1 The response generation system 100 — executable process 200.

[0089] The system can receive queries for tasks to be performed (step 202). For example, the query can specify a personalized task or a knowledge-intensive question-answering task, where the personalized task involves customizing the output based on user data. In some examples, the query can be a query to request the generation of a text description or summary of a content item. In some examples, the query can be a query to request answers to questions based on provided context items.

[0090] The system can receive multiple contextual content items for the task (step 204). Content items can be, for example, images, text documents, audio files, videos, or combinations thereof. Content items can be associated with a specific user. In some cases, content can be associated with a group of users, including the specific user.

[0091] For each of the multiple content items, the system can use a trained compression model to process the input to generate a compressed representation of that content item (step 206). The input may include a representation of the content item. The representation of a content item may be a sequence of terms representing the content item. The compressed representation may include one or more vectors of a fixed size. For example, the vectors may have a fixed size corresponding to the number of embeddings, such as 4, 8, 16, 32, 64, or 128 embeddings. Regardless of the size of the content item, the compressed representation can include the same number of embeddings. See below for reference. Figure 4 A more detailed description of training the compressed model.

[0092] The system can use compressed representations to generate aggregate compressed representations (step 208). An aggregate compressed representation may include one or more vectors representing content items. The aggregate compressed representation may have a fixed size. For example, an aggregate compressed representation may have a fixed size corresponding to 4, 8, 16, 32, 64, or 128 embeddings.

[0093] In some examples, to generate aggregated compressed representations, the system can apply order-independent aggregation operations to the compressed representations of content items. Order-independent aggregation operations can be, for example, mean pooling. Mean pooling computes a single aggregated vector by averaging the corresponding components of each compressed representation vector.

[0094] The system can use a generative neural network to process the input to generate a response to the query (step 210). The input may include a representation of the query and an aggregated compressed representation. The representation of the query may be a sequence of terms representing the query. The generated response may be data of any form, such as a text document, image, audio file, or video.

[0095] In some examples, the system can receive new content items for a task. The system can process input including the new content items to generate a new compressed representation of those items. The system can use the new compressed representation to update the aggregate compressed representation. The system can use the updated aggregate compressed representation to process input to generate a response to a query. The system can incorporate new content items into the aggregate compressed representation without reprocessing the original context content items.

[0096] Figure 3 This is a flowchart of an example process 300 for generating a compressed representation of a context data item. For convenience, process 300 will be described as being executed by a system of one or more computers located in one or more locations. For example, a response generation system appropriately programmed according to this specification—for example, Figure 1 The response generation system 100 — executable process 300.

[0097] The system can attach one or more compressed embeddings to the representation of a content item (step 302). The compressed embedding can be a special lexical appended to the input sequence representing the content item. The system can attach compressed embeddings to the lexical sequence representing the content item. See below for reference. Figure 4 The training process, described in more detail, involves learning compressed embeddings.

[0098] The attached compressed embeddings and content items can be provided as input to the compressed neural network. This allows the network to insert salient information from the content items into the representation associated with the compressed embeddings.

[0099] The system can use a compressed generative neural network to process content items and one or more compressed embeddings (step 304). The compressed generative neural network can be, for example, a pre-trained large language model (LLM). The compressed generative neural network can include multiple neural network layers. In some examples, at least one of the neural network layers can include one or more self-attention layers.

[0100] The system can use one or more compressed embeddings generated by the last neural network layer in the compressed generative neural network as a compressed representation of the content item (step 306). The compressed embeddings generated by the last neural network layer can represent the semantic information of the content item.

[0101] A compressed neural network can use one or more compressed embeddings generated by the last neural network layer in a compressed generative neural network as a compressed representation of a content item. After processing by the neural network layers, the compressed embeddings encapsulate information from the content item. The final layer representation of the compressed embeddings can be extracted as the output of the compressed neural network. The system can then use the extracted representation of the embeddings as a compressed representation of the content item.

[0102] Figure 4 This is a flowchart of an example process 400 for training a compressed model of a response generation system. For convenience, process 400 will be described as being executed by a system of one or more computers located in one or more locations. For example, a response generation system appropriately programmed according to this specification—for example, Figure 1 The response generation system 100 — the training system executable process 400.

[0103] The system receives training data (step 402). The training data may include multiple training examples and multiple training context content items. Each training example may include a training query and a training response to that training query. Each training example may further include multiple training context content items.

[0104] For example, the training query can specify a personalized task or a knowledge-intensive question-answering task, where the personalized task involves customizing the output based on user data. The training response can be, for example, a text document, an image, an audio file, or a video. The training response for the training examples can specify the target output for the training examples.

[0105] Training context content items can be, for example, images, text documents, audio files, videos, or combinations thereof. Training context content items can be associated with a specific user. Training context items can be associated with a task specified by the training query. For example, for a knowledge-intensive question-answering task, a training context item could be a text document describing a specific user's previous interactions. For example, for an image-based personalization task, a training context item could be an image of a specific user.

[0106] The system can use the training data to train a compressed model to minimize the objective function (step 404). For each of the plurality of training content items, the system can use the compressed model to process the input to generate a compressed representation of the training content item. The input may include the training content item and one or more compressed embeddings. The compressed representation may include one or more vectors. The vectors may have a fixed size. For example, the vectors may have a fixed size corresponding to the number of embeddings, such as 4, 8, 16, 32, 64, or 128 embeddings.

[0107] This system can use compressed representations to generate compressed training aggregate representations. The compressed training aggregate representation can include one or more vectors representing multiple training content items. The compressed training aggregate representation can have a fixed size. For example, the compressed training aggregate representation can have a fixed size corresponding to 4, 8, 16, 32, 64, or 128 embeddings.

[0108] The system can train a compressed model to minimize the objective function using training examples and a training aggregate compressed representation. For each training example, the system can use a generative neural network—for example… Figure 1 A generative neural network 116 is used to process the training query and train the aggregated compressed representation to generate a response to the training query. In some examples, the compressed model and the generative neural network can be a single neural network. The objective function can measure the quality of the generated response to the training query relative to the training response to the training query. For example, a lower value of the objective function may indicate that the generated response is closer to the training response in semantic content or style, while a higher value may indicate a greater difference.

[0109] The objective function may include, for example, a cross-entropy loss term. In some implementations, the cross-entropy loss term can be computed using the probability distribution over the vocabulary predicted by the generative neural network for each position in the generated response. The loss can be computed by comparing the predicted probability distribution with the target distribution. By minimizing the cross-entropy loss term, the system can adjust the weights of the compressed model to increase the likelihood that the generative neural network will produce a sequence of lexical terms that matches the training response.

[0110] In some implementations, the compressed model may include pre-trained weights that remain fixed during training and trainable weights that are updated during training. The trainable weights may be LoRA weights. In some examples, the pre-trained weights may be, for example, the weights of the underlying generative neural network model based on an LLM. During the training of the compressed model, the pre-trained weights may be frozen. Keeping these weights fixed allows the use of the language understanding capabilities of the underlying model without incurring the high computational costs associated with fully fine-tuning the entire model.

[0111] In some implementations, the compression model can be pre-trained to improve performance. Pre-training can involve training the model on an unsupervised task, such as an autoencoding task designed for a multi-document setup. For example, during pre-training, multiple documents can be provided as input, and queries can ask for the content of a specific document in the set. The pre-training task can be the task of reconstructing text from selected documents. This allows the model to be trained to efficiently compress and retrieve information from multiple sources.

[0112] Updating only the trainable weights in a compressed model allows the training process to focus on a smaller, more manageable set of parameters. This can lead to improved efficiency during training. Trainable weights can be a relatively small subset of the total parameters. Trainable weights allow the compressed model to be adapted to the specific task of generating compact representations of content items.

[0113] For example, parameter-efficient fine-tuning techniques such as Low-Rank Adaptive (LoRA) can be used to integrate trainable weights into a compressed model. LoRA weights can be introduced as trainable low-rank matrices into layers of a pre-trained model. During training, these LoRA weights can be updated while the original pre-trained weights of the base model remain unchanged. This allows the compressed model to adapt to the compression task in a parameter-efficient manner, reducing computational resources used for training and storage. By tuning these specific weights, the compressed model can learn to produce weights that can be generated by, for example... Figure 1 The generative neural network 116 efficiently processes compressed representations.

[0114] Training a compressed model can involve propagating gradients to one or more compressed embeddings. During training, the system can determine the gradient of the objective function with respect to the trainable parameters of the compressed model. The system can then use the gradients to update the trainable parameters, for example, using a backpropagation algorithm. Because the compressed embeddings are trainable parameters, the system can propagate the gradients back to one or more compressed embeddings to adjust their values. This allows the embeddings to learn to effectively summarize representations of the content items of the input to the generating neural network.

[0115] Figure 5 Examples are shown Figure 1 An example architecture 500 of the response generation system 100. Architecture 500 may include a compression level 502, a merging level 504, and a response level 506.

[0116] Compression level 502 includes a compression generative neural network 510. The compression generative neural network 510 can process content items 104a-c to generate corresponding compressed representations 108a-c for each content item. For example, content item 104a could be the text "I'm 36.", content item 104b could be the text "I like bouldering in gyms", and content item 104c could be the text "I work at Company A".

[0117] The compressed generative neural network 510 can be built using a pre-trained base model, such as a large language model. During training of the compressed model, the weights of this base model can be kept fixed or "frozen." The compressed neural network 510 can, for example, use... Figure 4 The process 400 is trained to generate compressed representations of content items.

[0118] The compressed generative neural network 510 can process the lexicalized representations 508a-c of content items 104a-c. Lexicalized representations can be compact representations of content items that can be processed by a neural network. For example, content items as text documents can be lexicalized by converting their words into sequences of numeric identifiers. These identifiers can then be mapped to embedding vectors to form lexicalized representations.

[0119] The merging level can include aggregators, for example, Figure 1Aggregator 110. The aggregator can merge compressed representations 108a-c to generate an aggregated compressed representation 112 of content items 104a-c. The aggregator can use a mean pooling operation to merge compressed representations 108a-c. The mean pooling operation can compute an element-wise average of the vectors from each compressed representation to generate a single aggregated representation. This process can be order-independent, meaning the final aggregated representation is independent of the order in which the individual compressed representations are processed. The resulting aggregated representation can have the same shape as the individual compressed representations. For example, if each compressed representation 108a-c is as follows: Figure 5 The example shown includes two vectors, so the aggregate compression representation 112 can also include two vectors.

[0120] The response level may include a generative neural network 116. The generative neural network 116 may be a large, pre-trained language model. The generative neural network 116 can process query 102 to generate a response. The query may provide a specific information request that guides the generative neural network to formulate an answer. For example, query 102 may be read as "In which office can I find a gym?".

[0121] The generative neural network can process input 114 to generate a response, which includes an aggregated compressed representation 112 and a lexicalized version of the query. To form input 114, the generative neural network can generate a lexicalized representation of query 102 and concatenate it with the aggregated compressed representation 112.

[0122] The generative neural network 116 can generate a response based on input 114, which includes combined information from query 102 and content items 104a-c. Figure 5 In the example shown, the response to the query could provide relevant and personalized answers based on the user's age, workplace, and interests. For instance, the response could be a suggestion for a gym located within the office space associated with Company A.

[0123] Figure 6 Examples are shown Figure 1 The example architecture 600 of the compressed neural network in the compressed model 106 is shown. The compressed neural network may include a frozen generative neural network 608. The frozen generative neural network 608 may be, for example, a frozen large language model (LLM).

[0124] A compressed neural network can attach a lexical representation 508c of content item 104c to trainable compressed embeddings 602a-b to generate the input to the frozen generative neural network 608. The lexical representation 508c of content item 104c may include embeddings 604a-d representing words in the content item. Content item 104c may include, for example, the text "I work at Company A". Content item 104c is first converted into a lexical representation 508c. The lexical embedding may be an embedding sequence 604a-d representing the content of the item. The lexical representation is then combined with one or more trainable compressed embeddings 602a-b. In some implementations, compressed embeddings 602a-b may be attached to embedding sequences 604a-d to form a combined input embedding sequence.

[0125] The frozen generative neural network 608 can process the input to generate a compressed representation 108c of the content item 104c. The frozen generative neural network 608 can be a pre-trained large language model (LLM), and its weights can remain fixed during the training of the compressed model. The frozen generative neural network 608 can process the entire input sequence 508c through multiple neural network layers. During this process, the representations of the compressed embeddings 602a-b can be influenced by the content of the lexicalized representation 508c. This allows information from the content item to be injected into the final representation of the compressed embedding.

[0126] To allow the frozen generative neural network 608 to adapt to the compression task without modifying its core weights, trainable parameters can be introduced into the compressed neural network. For example, a small number of trainable weights can be added to the layers of the frozen generative neural network 608 using the low-rank adaptive (LoRA) technique. During the training of the compressed neural network 510, updates can be applied only to these trainable weights and the trainable compressed embeddings 602a-b. (See above reference) Figure 4 A more detailed description of training the compressed neural network.

[0127] The compressed representation 108c of content item 104c can be the compressed embedding representation 606a-b of the final layer of the frozen generative neural network 608. The compressed representation 108c can be formed by extracting the final layer representation of the trainable compressed embeddings. The number of compressed embeddings can be fixed. Regardless of the length of the original content item 104c, the resulting compressed representation 108c has a fixed size. The final representations 606a-b can represent the salient information from content item 104c in a compact format. This compact format can be suitable for subsequent merging and processing by the generative neural network.

[0128] Figure 7Example 700 illustrates an example tradeoff between cost and quality in a response generation task. Example 700 includes four graphs 702, 704, 706, and 708, which respectively illustrate example tradeoffs between cost and quality for systems using cue tuning and systems using compressed representations in a personalized skill learning task. Cost can be represented by the number of lexical units in the cue, and quality can be represented by perplexity or ROUGE-L score. The curves in the graphs can represent models trained with different numbers of embeddings—e.g., 4, 8, 16, 32, 64, or 128 embeddings.

[0129] Figure 702 shows the relationship between the perplexity of the personalized social media post paraphrasing task and the number of lemmas in the prompt. Figure 704 shows the relationship between the ROUGE-L score of the personalized social media post paraphrasing task and the number of lemmas in the prompt. Figure 706 shows the relationship between the perplexity of the personalized news headline generation task and the number of lemmas in the prompt. Figure 708 shows the relationship between the ROUGE-L score of the personalized news headline generation task and the number of lemmas in the prompt.

[0130] As shown in figures 702, 704, 706, and 708, for both systems that prompt tuning and use compressed representation, for example... Figure 1 The response generation system 100, which uses compressed aggregate representations, can achieve higher quality results with fewer resources compared to cue tuning. For example, as shown in Figure 702, for a personalized tweet rewriting task within a specific lexical budget—e.g., a maximum of 128 lexicals—the system using the compressed representations described in this specification can achieve results with higher perplexity—e.g., higher quality—while using fewer lexicals in the cue, compared to cue tuning.

[0131] This specification uses the term "configured" in conjunction with system and computer program components. For configuring one or more computer systems to perform a specific operation or action, it means that software, firmware, hardware, or a combination thereof are installed on the system that causes the system to perform that operation or action during operation. For one or more computer programs configured to perform a specific operation or action, it means that the one or more programs include instructions that, when executed by a data processing device, cause that device to perform that operation or action.

[0132] Embodiments of the subject matter and functional operation described in this specification may be implemented in digital electronic circuit systems, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their equivalents), or in one or more combinations thereof. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination thereof. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals (e.g., machine-generated electrical, optical, or electromagnetic signals) that are generated to encode information for transmission to a suitable receiver device for execution by the data processing device.

[0133] The term "data processing device" refers to data processing hardware and encompasses all kinds of devices, apparatuses, and machines used for processing data, including, for example, programmable processors, computers, or multiple processors or computers. The device may also be or further include special-purpose logic circuit systems, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the device may optionally include code that creates an execution environment for computer programs, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, or combinations thereof.

[0134] A computer program, which may also be referred to or described as a program, software, software application, app, module, software module, script, or code, can be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages); and it can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but does not necessarily, correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions). A computer program can be deployed to execute on a single computer or on multiple computers located at a site or distributed across multiple sites and interconnected via a data communication network.

[0135] In this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process programmed to perform one or more specific functions. Typically, an engine will be implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines may be installed and run on the same one or more computers.

[0136] The processes and logic flows described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processes and logic flows can also be executed by a dedicated logic circuit system, such as an FPGA or ASIC, or by a combination of a dedicated logic circuit system and one or more programmed computers.

[0137] A computer suitable for executing computer programs can be based on a general-purpose or special-purpose microprocessor, or both, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are the central processing unit for executing instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by or incorporated into a special-purpose logic circuit system. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or will be operatively coupled to receive data from or transfer data to such mass storage devices, or both. However, a computer need not have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0138] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0139] To provide interaction with the user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Additionally, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a web page to a web browser in response to a request received from a web browser on the user's device. Furthermore, the computer can interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone running a messaging application) and receiving responsive messages from the user in response.

[0140] Data processing devices used to implement machine learning models may also include, for example, dedicated hardware accelerator units for handling the general and computationally intensive portions of machine learning training or production (i.e., inference, workloads).

[0141] Machine learning frameworks (such as the TensorFlow framework) can be used to implement and deploy machine learning models.

[0142] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes back-end components (e.g., as a data server), or middleware components (e.g., an application server), or front-end components (e.g., a client computer having a graphical user interface, web browser, or app that a user can interact with through an implementation of the subject matter described in this specification), or any combination of one or more such back-end components, middleware components, or front-end components. Components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0143] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship is established by computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data (e.g., HTML pages) to a user device, for purposes such as displaying data to a user interacting with the device acting as a client and receiving user input from that user. Data generated at the user device, such as the results of user interactions, may be received at the server from the device.

[0144] While this specification contains numerous details of specific implementations, these details should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features described in this specification in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed in this way, one or more features from a claimed combination may, in some cases, be removed from that combination, and the claimed combination may involve sub-combinations or variations thereof.

[0145] Similarly, although operations are depicted in the accompanying drawings and described in a specific order in the claims, this should not be construed as requiring such operations to be performed in the specific order shown or in sequential order, or requiring all shown operations to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0146] Specific embodiments of this subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired result. As an example, the processes depicted in the drawings do not necessarily require a specific order or sequence to achieve the desired result. In some cases, multitasking and parallel processing can be advantageous.

Claims

1. A method executed by one or more computers, the method comprising: Receive queries for tasks to be executed; Receive multiple contextual content items for the task; For each of the plurality of contextual content items, a trained compression model is used to process the input including the representation of the content item to generate a compressed representation of the content item, the compressed representation including one or more vectors having a first fixed size; The compressed representations of the plurality of context content items are used to generate an aggregated compressed representation comprising one or more vectors, the aggregated compressed representation representing the plurality of context content items; as well as A first generative neural network is used to process the input, including (i) the query and (ii) the aggregated compressed representation, to generate a response to the query.

2. The method of claim 1, wherein using a trained compression model to process the input including the representation of the content item comprises: One or more compressed embeddings are attached to the representation of the content item; The representation of the content item and the one or more compressed embeddings are processed using a compressed generative neural network comprising multiple neural network layers; as well as The compressed representation of the content item is the one or more compressed embeddings generated by the last neural network layer in the compressed generative neural network.

3. The method of claim 2, wherein the one or more compressed embeddings are learned during training.

4. The method of claim 2, wherein the plurality of neural network layers includes one or more self-attention layers.

5. The method of claim 1, wherein generating the aggregated compressed representation comprises applying order-independent aggregation operations to the compressed representation of the plurality of context content items.

6. The method of claim 5, wherein the order-independent aggregation operation is an average pooling operation.

7. The method of claim 1, wherein the aggregate compression represents having the first fixed size.

8. The method of claim 1, further comprising: Receive new content items for the task; The trained compression model is used to process the input, which includes a representation of the new content item, to generate a new compressed representation of the new content item, the new compressed representation comprising one or more vectors of a second fixed size; and The aggregated compressed representation is updated using the new compressed representation.

9. The method of claim 2, further comprising training the compression model.

10. The method of claim 9, wherein training the compression model comprises: Receive multiple training examples, each training example including (i) a training query and (ii) a training response to the training query; Receive multiple training environment content items for the task; For each of the plurality of training context content items, the compression model is used to process the input including the representation of the training content item to generate a compressed representation of the training content item, the compressed representation including one or more vectors having a third fixed size; The compressed representations of the plurality of training context content items are used to generate a training aggregate compressed representation comprising one or more vectors, the aggregate compressed representation representing the plurality of training context content items; as well as The compressed model is trained using the plurality of training examples and the training aggregate compressed representation to minimize the objective function.

11. The method of claim 10, wherein the objective function includes a cross-entropy loss term.

12. The method of claim 10, wherein training the compressed model to minimize the objective function using the plurality of training examples and the training aggregate compressed representation includes, for each training example: The first generative neural network is used to process the training query and the training aggregate compressed representation to generate a response to the training query, wherein the objective function measures the quality of the response to the training query relative to the training response to the training query.

13. The method of any one of claims 10 to 12, wherein training the compressed model comprises propagating gradients to the one or more compressed embeddings.

14. The method of any one of claims 10 to 12, wherein the compressed model has (i) pre-trained weights that remain fixed during the training and (ii) trainable weights that are updated during the training.

15. The method of claim 14, wherein the trainable weights are low-rank adaptive LoRA weights.

16. The method of any one of claims 1 to 12, wherein the plurality of scene content items are images.

17. The method of any one of claims 1 to 12, wherein the plurality of context content items are text documents.

18. The method of any one of claims 1 to 12, wherein the plurality of scene content items are audio files.

19. The method of any one of claims 1 to 12, wherein the plurality of scene content items are videos.

20. A system comprising: One or more computers; as well as One or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations, the operations including: Receive queries for tasks to be executed; Receive multiple contextual content items for the task; For each of the plurality of contextual content items, a trained compression model is used to process the input, which includes a representation of the content item, to generate a compressed representation of the content item, the compressed representation comprising one or more vectors of a fixed size; The compressed representations of the plurality of context content items are used to generate an aggregated compressed representation comprising one or more vectors, the aggregated compressed representation representing the plurality of context content items; and A generative neural network is used to process the input, including (i) the query and (ii) the aggregated compressed representation, to generate a response to the query.

21. A non-transitory computer storage medium storing one or more instructions, said instructions causing said one or more computers to perform operations when executed, said operations including: Receive queries for tasks to be executed; Receive multiple contextual content items for the task; For each of the plurality of contextual content items, a trained compression model is used to process the input, which includes a representation of the content item, to generate a compressed representation of the content item, the compressed representation comprising one or more vectors of a fixed size; The compressed representations of the plurality of context content items are used to generate an aggregated compressed representation comprising one or more vectors, the aggregated compressed representation representing the plurality of context content items; as well as A generative neural network is used to process the input, including (i) the query and (ii) the aggregated compressed representation, to generate a response to the query.

Images