Generative model control for performing tasks using multiple generative models

Abstract

Methods, systems, and devices for controlling engagement of generative models in multi-agent systems, including computer programs encoded on computer storage media. One of the methods includes receiving an input for a task to be processed by a set of generative models to generate a final output for the task and processing the input by the set of generative models across a plurality of steps, processing the input including, for each intermediate step, obtaining context data at the intermediate step, generating control data for a target generative model in the set based on the context data, providing the context data and the control data to the target generative model in the set, obtaining an output from the target generative model generated in response to the context data and the control data, and updating the context data for the task based on the output from the target generative model.

Description

Background Technology

[0001] This specification relates to using a set of generative models to process requests.

[0002] Generative models have demonstrated state-of-the-art performance across a wide range of tasks, including text generation (e.g., writing, summarizing, translating, encoding), image generation, and audio generation. Generative models utilize very large neural network models trained on massive amounts of data. For example, a large language model (LLM) can include a transformer-based neural network model with self-attention capabilities and can achieve general language understanding and generation in response to queries. Therefore, generative models are being deployed in various applications, such as as coding assistants, email writing assistants, and for generating images in presentations.

[0003] Generative agents are software programs that use generative models to generate generative outputs in response to inputs, such as prompts in natural language that request the generative agent to perform a specific task. Summary of the Invention

[0004] This specification describes systems and techniques for managing and controlling the participation of two or more generative models in a multi-agent system using controlled generative models.

[0005] In a multi-agent system, a set of generative models processes task-specific inputs to generate task-specific outputs. In other words, two or more generative models in a multi-agent system interact with each other to solve the task.

[0006] In some cases, users can invoke multiple generative models to generate outputs specific to a task.

[0007] In some cases, a user can invoke a first generative model to generate task-specific output, and the first generative model can invoke one or more other generative models to collaboratively generate task-specific output.

[0008] Typically, this group of generative models processes task-specific inputs across multiple steps. At each step, the corresponding generative model in the group can generate a generative output.

[0009] However, when multiple generative models interact with each other, for example through dialogue, to generate task-specific outputs, there is a risk that several generative models may fail to complete the task within the amount of computational resources allocated to it, or may consume excessive computational resources while attempting to complete the task. For example, this could be because one or more generative models might generate an excessive amount of output, generate task-irrelevant output, or generate too much detail about a topic without making progress toward solving the task. Unlike human interaction, generative models may generate outputs that do not contribute to completing the task, but may incur significant computational costs and ultimately be counterproductive. For example, the outputs might be irrelevant, or the models could be configured by malicious actors to prevent other models from completing the task.

[0010] In some cases, chatroom systems offer basic control by removing specific generative language models from the conversation (e.g., disabling or muting specific generative language models). However, these approaches unnecessarily restrict the contribution of specific generative language models to the conversation without considering that, given appropriate cues, these models could provide useful outputs later in the conversation.

[0011] This specification describes systems and techniques for managing and controlling task-specific outputs generated from a set of multiple generative models using controlled generative models.

[0012] In some implementations, the control generative model can generate control data that defines the computational budget for the target generative model, which defines the amount of computational resources consumed by the target generative model in each step.

[0013] In some implementations, the control generative model can generate semantic control data that enables the target generative model to generate outputs focused on one aspect of the task.

[0014] In some implementations, controlling the generative model can monitor the collective progress of multiple generative models in solving the task and determine which of the multiple generative models should generate the output in the next step.

[0015] Generally, an innovative aspect of the subject matter described in this specification can be embodied in a method comprising the following actions: receiving task-specific input to be processed by a set of generative models to generate a final output for the task; processing the input across multiple steps by the set of generative models, wherein a corresponding generative model in the set generates a generative output at each of the multiple steps, and processing the input includes, for each intermediate step in the multiple steps, obtaining task-specific context data at that intermediate step by a control generative model; generating control data for a target generative model in the set based on the task-specific context data at the intermediate step by the control generative model; providing the task-specific context data and the control data at the intermediate step to the target generative model in the set by the control generative model; obtaining an output generated in response to the context data and the control data from the target generative model by the control generative model; and updating the task-specific context data by the control generative model based on the output from the target generative model. Other embodiments in this regard include corresponding computer systems, devices, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of these methods. For a system of one or more computers to be configured to perform a specific operation or action, this means that the system has software, firmware, hardware, or a combination thereof installed thereon that causes the system to perform those operations or actions in operation. For one or more computer programs configured to perform a specific operation or action, this means that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform the operation or action.

[0016] The foregoing and other embodiments may each optionally include one or more of the following features, individually or in combination. Specifically, one embodiment includes all of the following features in combination. The control data includes a computational budget for the target generative model, and the computational budget defines the amount of computational resources consumed by the target generative model in the intermediate step. The control data defines the number of lexical units in the output generated by the target generative model at the intermediate step. The control data limits the number of lexical units in the output generated by the target generative model at the intermediate step, and wherein limiting the number of lexical units in the output generated by the target generative model at the intermediate step does not reduce the final progress score of the task, which is a measure of how much the task has been completed after the input has been processed across the multiple steps to generate the final output for the task. The control data includes semantic control data that causes the target generative model to generate output focused on one aspect of the task. The control data causes the target generative model to generate output at the intermediate step indicating that the context data has not changed. The action includes, for each intermediate step, the controlling generative model selecting the target generative model from the group of generative models based on contextual data for the task at that intermediate step. The contextual data includes corresponding previous outputs generated by one or more generative models in the group at one or more previous steps preceding the intermediate step. The contextual data includes a progress score indicating the level of progress made in the direction of task completion. The contextual data includes the importance level of the task. The contextual data includes the remaining computational budget of the group of generative models at the intermediate steps for completing the task. The contextual data includes metadata about the group of generative models. The action includes: receiving the final output for the task generated by the group of generative models at the final step of the plurality of steps; generating training data for training the controlling generative model based on the contextual data for the task at each of the plurality of steps; and training the controlling generative model using the training data. The action includes, after receiving the final output from the generative model at the final step, the controlling generative model determining that the task has been completed.

[0017] Specific embodiments of the subject matter described in this specification may be implemented in order to achieve one or more of the following advantages.

[0018] The systems and methods described in this specification can reduce the computational resources (e.g., memory and processor cycles) consumed by multiple generative models generating the final output for a task without reducing overall progress toward task completion. Instead of controlling one or more generative models to prevent them from generating excessive output, irrelevant output, or too much detail on a topic without making progress toward task resolution, this is achieved by monitoring the collective progress of multiple generative models in solving the task, determining which of the multiple target generative models should generate output in the next step, and limiting the number of lexical units in the output from the target generative models.

[0019] This system and method can improve the efficiency of providing a final output for a task from multiple generative models. For example, given a number of lexical units for one or more of the generative models, the controlling generative model can generate the final output in fewer steps because it can use semantic control data to guide the dialogue between the multiple generative models, enabling them to generate outputs focused on one aspect of the task. In some implementations, the system and method can reduce the overhead of user interaction with multiple generative models because the user-visible dialogue is less verbose.

[0020] Details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of this subject matter will become apparent from the description, drawings, and claims. Attached Figure Description

[0021] Figure 1 This is a diagram of an example multi-agent system.

[0022] Figure 2 This is a sample user interface that displays a dialogue generated by multiple generative models controlled by a control generative model.

[0023] Figure 3A This is a flowchart of an example process for controlling the participation of two or more generative models.

[0024] Figure 3B This is a flowchart of an example process for controlling the participation of two or more generative models, executed at each intermediate step in a multi-step process.

[0025] Figure 4 This is a flowchart of an example process for training a control generative model.

[0026] In the various figures, the same reference numerals and names indicate the same elements. Detailed Implementation

[0027] Figure 1 This is a diagram of an example multi-agent system 100.

[0028] System 100 is an example of a system implemented as a computer program on one or more computers 116 in one or more locations, in which the systems, components, and technologies described herein are implemented. The one or more computers 116 may include personal computers, mobile communication devices, servers, and other devices capable of sending and receiving data over a network. The one or more computers 116 may be implemented via a network connection such as a local area network (“LAN”), a wide area network (“WAN”), the Internet, or combinations thereof (not shown). The system may use a single computer or multiple computers operating in combination with each other, including, for example, a set of remote computers deployed as a cloud computing service.

[0029] System 100 receives task-specific input 102 and generates task-specific output 114. To perform the task, the system uses a set of generative models 110 to process the input. This set of generative models 110 processes the input 102 across multiple steps.

[0030] At each step, the corresponding generative model in the group generates generative output.

[0031] For example, at each step, the generative model receives input and generates output. The input may include one or more of the following: raw input for the task, output generated at a previous step, output generated at more than one previous step, or a hint unrelated to the input.

[0032] The multi-agent system 100 includes multiple generative models that collaboratively solve tasks.

[0033] For example, in Figure 1 In one example, system 100 includes a first generative model 110 (1) and a second generative model 110 (2), collectively referred to as generative model 110. In other examples, multi-agent system 100 may have three, four, five or more generative models 110.

[0034] Multiple generative models 110 can be models with different sizes, different neural network architectures, different capabilities, or different specializations.

[0035] For example, multiple generative models can include one neural network model with fewer parameters and another neural network model with (relatively) more parameters.

[0036] Multiple generative models may include: a first neural network model trained to perform a wide range of tasks; and a second neural network model trained to perform a specific task in a specific task domain.

[0037] Multiple generative models can use natural language as a common interface for communication with each other.

[0038] Multiple generative models may include one or more models deployed on a local device and one or more other models deployed in the cloud. In some implementations, multiple generative models may include relatively small, general-purpose neural networks deployed on user devices, and one or more other relatively large or specialized neural networks deployed on servers. In some implementations, all generative models in the multi-agent system 100 may be implemented in the cloud. In some implementations, one or more generative models may have access to private user data, while other generative models may not have access to private user data.

[0039] In some implementations, if a user believes that combining multiple generative models can solve a task better than a single generative model, the user's device can invoke these generative models to solve the task. In other implementations, the user's device can invoke generative models, which in turn can invoke one or more other generative models to delegate aspects of the task to improve productivity.

[0040] For example, a user can query two different generative models to gain a more balanced perspective on a topic. As another example, a user can query generative models with different capabilities. For instance, a user can query a generative model that can book hotels, and another generative model that can book flights.

[0041] Generative models are machine learning (ML) models that generate content based on input, including text, images, audio, or other synthetic data. During inference, in response to query input, generative model 110 can generate generative output, such as content of a certain type. In some implementations, generative model 110 can generate multimodal output, such as an image and corresponding text describing that image.

[0042] In some implementations, the generative model 110 can be configured to process an input lexical sequence to generate an output lexical sequence. Lexical units can represent any suitable type of content, such as text, images, videos, audio, or some combination thereof. For example, the generative model may be a Large Language Model (LLM) and can be configured to process an input lexical sequence from a vocabulary of text lexical units to generate an output lexical sequence from the vocabulary.

[0043] More generally, the generative model 110 can be any suitable neural network that receives an input sequence including text words and autoregressively generates an output sequence including text words. For example, the generative model 110 can be a Transformer-based language model neural network or a recurrent neural network-based language model neural network.

[0044] In some cases, when a neural network used to implement a language model regressively generates a sequence of output lexical units, the generative model 110 may be referred to as an autoregressive neural network. More specifically, the output generated by autoregression is created by generating each specific lexical unit in the output sequence conditioned on the current input sequence, which includes any lexical units in the output sequence preceding that specific lexical unit (i.e., any lexical units that have been generated for any previous position in the output sequence preceding that specific lexical unit), and contextual input that provides context for the output sequence.

[0045] For example, when generating a lexical at any given position in the output sequence, the current input sequence may include lexicals in the input sequence and any preceding positions in the output sequence before that given position. As a specific example, the current input sequence may include the input sequence, followed by any preceding positions in the output sequence before that given position. Optionally, the input sequence and the current output sequence may be separated by one or more predetermined lexicals within the current input sequence.

[0046] More specifically, to generate a specific lexical at a specific position within the output sequence, the generative model 110 can process the current input sequence to generate a score distribution (e.g., a probability distribution) that assigns a corresponding score, e.g., a corresponding probability, to each lexical in the lexical vocabulary. The language model's neural network can then use this score distribution to select a lexical from the vocabulary as that specific lexical. For example, the language model's neural network can greedily select the lexical with the highest score, or it can sample lexicals from the distribution, for example, using kernel sampling or another sampling technique.

[0047] As a specific example, generative model 110 may be a neural network based on an autoregressive Transformer, which includes (i) multiple attention blocks, each of which applies a self-attention operation; and (ii) an output subnetwork that processes the output of the last attention block to generate a score distribution.

[0048] Generative model 110 can have any Transformer-based neural network architecture among various Transformer-based neural network architectures. Examples of such architectures include those described in the following literature: J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, LA Hendricks, J. Welbl, A. Clark et al., Training compute-optimal large language models, arXiv preprint arXiv:2203.15556, 2022; JWRae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, HF Song, J. Aslanides, S. Henderson, R. Ring, S. Young, E. Rutherford, T. Hennigan, J. Menick, A. Cassirer, R. Powell, G. van den Driessche, LA Hendricks, M. Rauh, P. Huang, A. Glaese, J. Welbl, S. Dathathri, S. Huang, J. Uesato, J. Mellor, I. Higgins, A. Creswell, N.McAleese, A.Wu, E. Elsen, SM Jayakumar, E. Buchatskaya, D. Budden, E.Sutherland, K. Simonyan, M. Paganini, L. Sifre, L. Martens, XL Li, A. Kuncoro, A.Nematzadeh, E. Gribovskaya,D. Donato, A. Lazaridou, A. Mensch, J. Lespiau, M.Tsimpoukelli, N. Grigorev, D. Fritz, T. Sottiaux, M. Pajarskas, T. Pohlen, Z. Gong, D. Toyama, C. de Masson d'Autume, Y. Li, T. Terzi, V. Mikulik, I. Babuschkin, A. Clark, D. de Las Casas, A. Guy, C. Jones,J.Bradbury, M. Johnson, BA Hechtman, L. Weidinger, I. Gabriel, WS Isaac, E. Lockhart, S. Osindero, L. Rimell, C. Dyer, O. Vinyals, K. Ayoub, J. Stanway, L. Bennett, D. Hassabis, K. Kavukcuoglu, and G. Irving. Scaling language models: Methods, analysis & insights from traininggopher, CoRR, abs / 2112.11446, 2021; Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, arXiv preprint arXiv:1910.10683, 2019; Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le, Towards a human-like open-domain chatbot (Towards Human-like Open Domain Chatbots), CoRR, abs / 2001.09977, 2020; and Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al., Language models are few-shot learners, arXiv preprint arXiv:2005.14165, 2020.

[0049] In some implementations, the generative model 110 may use a decoder-only architecture, which comprises multiple decoder blocks without an encoder. Each decoder block may include a self-attention layer and a feedforward neural network. The transformer-based generative model is an example of a generative model to which the system and techniques described in this paper are applicable.

[0050] More generally, the systems and techniques described herein are applicable to other types of generative models. One example of generative model 110 is a diffusion model. As another example, generative model 110 can be a diffusion model that uses a text-to-image diffusion model to generate a first image, and then applies one or more super-resolution diffusion models to generate a final image. As yet another example, generative model 110 can be an autoregressive generative model that autoregressively generates lexicals representing audio, video, images, or other data. As yet another example, generative model 110 can be a masked lexical generative model that sequentially demasks lexicals representing text, video, audio, images, or other data during generation.

[0051] System 100 receives task-specific input 102 and processes it through a set of generative models 110 to generate a final task-specific output 114. System 100 processes the input across multiple steps using this set of generative models 110. At each step, the corresponding generative model in the set generates a generative output.

[0052] However, in some systems, one or more generative models may fail to complete the task within the computational budget, or may consume excessive computational resources while completing the task.

[0053] A task's computational budget defines the maximum amount of computational resources (e.g., memory and / or processor cycles) that can be used to execute the task. In some implementations, the system can use the entire computational budget to execute the task. For example, a computational budget can define the amount of computational resources (e.g., memory and / or processor cycles) used to execute the task. In some implementations, a computational budget can be assigned to a generative model.

[0054] The computational budget of a generative model can define the maximum amount of computational resources (e.g., memory and / or processor cycles) consumed by the generative model over a period of time (e.g., during intermediate steps). In some implementations, the generative model may use the entire computational budget over a period of time. Therefore, the computational budget can define the amount of computational resources consumed by the generative model over a period of time. For example, one or more generative models may generate too much output, generate task-irrelevant output, or generate too much detail about a topic without making progress toward solving the task. Therefore, it is desirable to set a computational budget to control the generative output generated by multiple generative models at multiple steps. In some cases, the computational budget can define the number of lexical units to be used to complete the task during one or more steps performed by multiple generative models (e.g., a maximum number).

[0055] In some systems, tasks can be assigned to two or more generative models in the group to handle different aspects of the task. For example, one generative model might be a dedicated model for booking hotels, while another might be a dedicated model for booking flights. Therefore, it is desirable to coordinate the generative outputs generated by multiple generative models at multiple steps.

[0056] System 100 uses a control generative model 106 to manage and control task-specific outputs generated from multiple generative models. The control generative model 106 can manage and control the output generated by one of the generative models at each step. In some implementations, the control generative model 106 can be a generative language model trained to manage and control dialogue in a group chat environment. In some implementations, the control generative model 106 can be a large language model (LLM).

[0057] Control generative model 106 and multiple generative models 110 may be implemented locally relative to each other. For example, control generative model 106 and multiple generative models 110 may be implemented on the same user device or on the same set of one or more computers in a server system. In some implementations, control generative model 106 and multiple generative models 110 may be implemented remotely to each other. For example, the system may include a first generative model 110 (1) running on a user device and a second generative model 110 (2) running on a remote server in the cloud.

[0058] The generative control model 106 can generate control data 108, which controls the output generated by the generative model. The system 100 can provide prompt input to the generative control model 106, which can request the generative control model to generate control data 108, which can be parsed by the generative model to generate the desired output.

[0059] For example, the prompt input to the control generative model could be "which model should generate the next output given this context?" The control generative model 106 can process the prompt input and can generate control data indicating which model should generate the next output.

[0060] As another example, the prompt input for controlling the generative model could be "the next generative model is going to be model A, and what should be the number of tokens in the output generated by model A?" The controlling generative model 106 can process the prompt input and can generate control data indicating a limit on the number of tokens in the output generated by model A.

[0061] The system can provide control data 108 as input to the generative model, for example, as a hyperparameter of the generative model, or as part of a prompt for the generative model, or as part of a prompt input that the generative model can resolve. For example, the target generative model to generate the output at the current step can be a first generative model 110(1), and the control generative model 106 can generate control data 108 that controls the output generated by the first generative model 110(1).

[0062] In some implementations, control data 108 may include one or more hyperparameters that define how the target generative model generates the output. Control generative model 106 may generate one or more hyperparameters based on context data 104 for the task at the current step. For example, one or more hyperparameters may set the maximum length of decoded tokens, or the bundle size explored at each step when generating output, constrain the decoding algorithm by providing an additional function of stepwise scores, or skip several rounds of decoding in the target generative model.

[0063] For example, the control generative model 106 can receive a prompt requesting it to generate control data 108, which includes the corresponding control bundle score for each candidate in the bundle of the bundle search algorithm. The generative model 110 can have model bundle scores for each candidate in the bundle. The system can generate modified model bundle scores by multiplying the control bundle scores by weights and adding the weighted control bundle scores to the model bundle scores. Therefore, the generative model 110 can use the modified model bundle scores to control the bundle search based on candidates in the bundle, such as reordering candidates based on their modified bundle scores. In some implementations, for example, the system can reorder candidates in the bundle at specific lexical units when a sentence ends or when a subtask is completed.

[0064] In some implementations, generative model 110 can begin solving a task or subtask through multi-step reasoning. Control generative model 106 can determine, for example, that multi-step reasoning is not useful through hints or scores. Control generative model 106 can generate control data 108 that stops all remaining decoding rounds.

[0065] In some implementations, the control data 108 may include a computational budget for the target generative model. In some implementations, the computational budget may define the amount (or maximum amount) of computational resources consumed (or can be consumed) by the target generative model in intermediate steps, such as memory and processor cycles. In some implementations, the computational budget may define the total amount (or maximum amount) of computational resources that the target generative model can consume across multiple steps. Therefore, based on the total amount of computational resources that the target generative model can consume, the control generative model 106 can determine the amount of computational resources that the target generative model can consume for each of the multiple steps.

[0066] For example, the control generative model 106 can generate control data 108(1) for the first generative model 110(1) in the group based on the context data 104 for the task at step n. The control data 108(1) may include a computational budget for the first generative model 110(1), which defines the amount (or maximum amount) of computational resources that the first generative model 110(1) should consume in the current step n.

[0067] In some implementations, control data 108 may define the number of lexical units in the output generated by the target generative model at the current step. For example, control data 108(1) may define the number of lexical units in the output of the first generative model 110(1). For example, control data 108(1) may define the number of lexical units per second in the output of the first generative model 110(1). Thus, system 100 may provide latency control that limits the number of lexical units per second, thereby allowing sufficient intervention time for the system or user before the amount of computational resources consumed by the generative model exceeds a threshold. For example, control data 108 may include data defining the number of processing rounds performed during the decoding loop, and generative model 110 may exit the decoding loop when that number of processing rounds have been performed. As another example, control data 108 may include data defining the amount of waiting time required to achieve the desired lexical unit rate after several decoding steps.

[0068] In some implementations, control data 108 can limit the number of lexical units in the output generated by the target generative model at the current step. For example, control generative model 106 can determine an upper limit on the number of lexical units in the output generated by the target generative model at the current step.

[0069] In some implementations, the control generative model 106 can generate control data that limits or defines the number of lexical units in the output generated by the target generative model at the current step, such that limiting the number of lexical units does not reduce the final progress score of the task. The final progress score can be a measure of how well the task is completed after the input has been processed across multiple steps to generate the final output for the task. Therefore, computational savings do not slow down progress toward providing the final output 114 of the task. In some cases, computational savings can improve efficiency and accelerate progress toward providing the final output 114 of the task.

[0070] For example, an agent might have generated lengthy outputs in previous steps. Controlling the generative model can limit the number of lexical units in the output generated by the agent at the current step. In response, the agent can generate more concise outputs at the current step without slowing down progress toward completing the task.

[0071] In some implementations, the control data 108 may include semantic control data that enables the target generative model to generate outputs focused on one aspect of the task. The semantic control data can dynamically adjust the level of engagement of each generative model in solving the task across multiple steps.

[0072] For example, at step n+1, the control generative model can generate control data 108(2) for the second generative model 110(2) in the group based on the context data 104 for the task at the current step n+1. The control data 108(2) may include semantic control data that enables the second generative model 110(2) to generate a generative output 112(2) focused on one aspect of the task at step n+1.

[0073] The control generative model 106 can generate control data 108 that controls the output generated by the generative model based on context data 104. For example, the control generative model 106 can generate control data 108 for the target generative model 110 in the group based on the context data 104 for the task at the current step.

[0074] Context data 104 may include any information related to generating a response for the task. In some implementations, context data 104 may include previous outputs generated by the generative model at previous steps prior to the current step, such as the entire dialogue history. For example, the context data at step n+1 may include the context data at step n and the generative output 112 generated by the first generative model at step n.

[0075] In some implementations, context data 104 may include data summarizing previous outputs generated by one or more generative models in the group at one or more previous steps prior to the current step. For example, system 100 may not maintain the entire dialogue history in memory, but instead require the generative models to generate a summary of the dialogue history.

[0076] In some implementations, context data 104 may include the importance level of the task or an aspect of the task. For example, context data may indicate the level of detail required for the final output. Based on the context data indicating the level of detail required for the final output, the controlling generative model 106 may generate control data to control the amount of computation performed by the target generative model.

[0077] In some implementations, context data 104 may include the remaining computational budget of the group of generative models at the current step for completing the task. The remaining computational budget may be based on the total computational budget of the group of generative models. Alternatively, the remaining computational budget may be based on the individual computational budget of each of one or more generative models in the group. For example, if the total computational budget of the group of generative models is less than a threshold, controlling generative model 106 may determine to spend the budget on the generative models most likely to produce some useful results for the task.

[0078] In some implementations, context data 104 may include metadata about one or more generative models in the group. For example, context data 104 may include metadata indicating that a first generative model is specifically for hotel bookings, while a second generative model is specifically for flight bookings.

[0079] In some implementations, contextual data 104 may include user input, such as natural language feedback from the user, and control generative model 106 may generate control data 108 based on user input.

[0080] For example, a user can provide natural language feedback stating, "I think Agent 1 can solve most of the problems related to language learning on its own." The control generative model 106 can select Agent 1 as the target generative model for one or more subsequent steps and generate control data that enables Agent 1 to solve language learning-related problems.

[0081] As another example, a user could provide natural language feedback stating, "I think Agent 4 should provide less advice when it comes to math problems." If the current step relates to solving a math problem, the generative model 106 can generate control data that limits the number of lexical units in the output generated by Agent 4 at the current step.

[0082] In some implementations, contextual data 104 may include input provided by the user through the user interface (UI) of system 100. For example, system 100 may display a slider in the UI to allow the user to limit computational budgets, such as the number of lemmas in the output of the target generative model. In some cases, system 100 may display text boxes in the UI to allow the user to limit the generative output of the target generative model on a particular topic, or provide feedback directly in the dialogue.

[0083] In some implementations, context data 104 may include the various types of context data discussed above, as well as any other suitable combination of context data, such as previous outputs generated by the generative model at one or more previous steps, data summarizing previous outputs, the importance level of the task or an aspect of the task, the remaining computational budget for completing the task at the current step, metadata about one or more generative models in the group, and inputs from the user device.

[0084] In some implementations, the set of generative models can generate outputs across multiple steps (such as sequentially) in a predetermined order. Therefore, the target generative model at the current step can be predetermined.

[0085] In some other implementations, the set of generative models may not generate outputs in a predetermined order, and the controlling generative model 106 may select a target generative model from the set of generative models based on the task-specific context data 104 at the current step. For example, the controlling generative model 106 may select from the set of generative models the one most likely to generate productive outputs to complete the task in input 102.

[0086] In some implementations, the control generative model can select the generative model for solving the task based on the generative model's metadata. For example, context data 104 may include metadata indicating that a first generative model has been previously trained to solve a problem related to subtask A, and a second generative model has been previously trained to solve a problem related to subtask B. Based on context data 104, the control generative model 106 can determine that the multi-agent system 100 should solve subtask A at the current step. Therefore, the control generative model 106 can select the first generative model to solve subtask A at the current step. As another example, the metadata for the generative model can indicate which tools (e.g., a calculator or a search engine) the generative model can access. Based on the subtask to be solved, the control generative model can determine that a generative model with access to a particular tool may be sufficient or necessary. For example, if a subtask involves mathematical calculations, the control generative model can determine that a generative model with access to a calculator is sufficient. If the subtask involves searching for information, the control generative model can determine that a generative model with access to a search engine is needed.

[0087] For example, context data 104 may include previous tasks and responses generated by a set of generative models, and how each generative model contributes to solving the task. Controlling generative models 106 may assign contribution scores, which indicate the generative model's contribution to solving the task. Therefore, controlling generative model 106 may select generative models for solving a specific task or subtask based on their contribution scores to solving previous tasks. In some implementations, controlling generative model 106 may arbitrarily decide to include or exclude some generative models used to solve previous tasks. Controlling generative model 106 may compare the performance of generative models used to solve the same or similar types of tasks to determine how each generative model contributes to solving the task.

[0088] In some implementations, control data 108 can cause the target generative model to generate output at an intermediate step that indicates no change in the context data. System 100 can provide a prompt input to the control generative model 106, which can request the control generative model to generate control data 108 that indicates the target generative model should not generate output at the current step. For example, if context data 104 indicates that the target generative model is not useful for generating output at the current stage, the control generative model 106 can generate control data 108 that causes the target generative model to skip generating output, which could result in the context data 104 remaining unchanged for the next step.

[0089] In some implementations, if the controlling generative model 106 determines that no generative model is generating output that helps solve the task—for example, if the dialogue is deadlocked—the controlling generative model 106 can cause all generative models to stop generating generative output while requesting feedback from the user. In some implementations, the controlling generative model 106 can generate control data 108, and this control data 108 can include a cue that can be parsed by the generative models, indicating that they should not generate output.

[0090] In some implementations, controlling the generative model 106 can determine, through evaluative cues, that the generative model has not generated useful output. The controlling generative model can use cues such as: “on a scale of 1-10, how helpful was this response to solve the task?” or “which parts of this conversation were key reasoning steps to solve the task?”

[0091] In some implementations, the control generative model 106 can determine that no generative model generates output that helps solve the task by summarizing the task-solving state. If multiple rounds of dialogue between generative models yield the same summary of the task-solving state, the control generative model can determine that the generative model is not helpful and has not made progress toward generating useful information.

[0092] Controlling the generative model can prevent all generative models from generating generative output in several ways. In some implementations, controlling the generative model during streaming decoding can stop the streaming of decoding. In some implementations, controlling the generative model can generate a stop signal, and the generative model can be configured to check for the stop signal before generating generative output. If the generative model receives the stop signal, it will not generate output. In some implementations, after several steps, controlling the generative model can generate a prompt, which can be provided as input to the generative models to prevent them from generating generative output. In some implementations, controlling the generative model can generate control data to disconnect the network connecting the generative models to the system.

[0093] In some implementations, the control generative model 106 can predict the possible action of each generative model based on the stated goals of the task, possible interactions between generative models, possible interactions between generative models and the user, or a combination thereof. System 100 can provide cue input to the control generative model 106, which can request the control generative model to generate control data 108 indicating the possible action of each generative model based on the stated goals of the task, possible interactions between generative models, possible interactions between generative models and the user, or a combination thereof. The control generative model 106 can provide control data 108 to a target generative model, which can process the control data 108 to generate generative output 112 based on the stated goals of the task, possible interactions between generative models, possible interactions between generative models and the user, or a combination thereof, according to the possible action of each generative model.

[0094] For example, the generative control model 106 can predict that agent 5 is likely to be dedicated to performing web searches because the user is satisfied with previous search results provided by agent 5 but not with previous search results provided by agent 3. The generative control model 106 can generate control data that causes agent 3 to provide input to agent 5 and causes agent 5 to perform searches based on the input from agent 3. Then, the generative control model 106 can generate control data that causes agent 3 to perform further processing of the task based on the search results generated by agent 5.

[0095] In some implementations, the multi-agent system 100 can simultaneously process input from two or more tasks from multiple users. A control generative model 106 can segment the dialogue into multiple threads, each for a specific task, and can independently track the dialogue flow across multiple tasks. The system 100 can provide cue input to the control generative model 106, which can request the model to generate control data 108 that segments the dialogue into multiple threads, each for a specific task, and independently tracks the dialogue flow across multiple tasks. The control generative model 106 can generate corresponding control data for each of the two or more tasks. The system 100 can provide cue input to the control generative model 106, which can request the model to generate corresponding control data for each of the two or more tasks.

[0096] The control generative model 106 can provide the target generative model 110 in the group with task-specific context data 104 and control data 108 at the current step. For example, the control generative model 106 can provide the context data 104(1) and control data 108(1) as input to the first generative model 110(1). As another example, the control data 108 may include hyperparameters defining how the first generative model generates outputs, such as parameters defining the bundle size. The first generative model can use the hyperparameters to generate generative outputs.

[0097] In some implementations, the set of generative models can be based on a language model, such as an LLM. The interface controlling the interaction between generative model 106 and the set of generative models 110 can be natural language. In some implementations, other forms of interfaces may also be used.

[0098] In some implementations, the control generative model 106 may include context data 104 and control data 108, for example, in the input of the target generative model as prompts. For example, prompts for the first generative model 110 may include task-specific input 102, context data 104(1) including dialogue history, and control data 108(1) that restricts the generative output 112(1) to a maximum of three sentences.

[0099] In some implementations, the control generative model 106 can set one or more parameters of the target generative model based on the control data 108. For example, the parameters of the target generative model may include the number of lexical units in the generative output 112(1) from the first generative model 110(1), and the control generative model 106 can set this parameter based on the control data 108.

[0100] In some implementations, the context data 104(1) provided to the first generative model 110(1) may be the same as the context data 104 processed by the control generative model 106 to generate control data 108(1). In some implementations, the context data 104(1) provided to the first generative model 110(1) may be different from the context data 104 processed by the control generative model 106 to generate control data 108(1). For example, the context data 104(1) may include a portion of the context data 104 that contributes to generating the generative output 112(1) at step n.

[0101] The target generative model 110 can generate a generative output 112 at the current step. For example, based on contextual data 104(1) and control data 108(1) that restricts or defines the number of lexical units in the output of the first generative model, the first generative model 110(1) can generate a generative output 112(1) at step n. The number of lexical units in the generative output 112(1) satisfies the restrictions or definitions provided in the control data 108(1). As another example, based on contextual data 104(2) and semantic control data 108(2), the second generative model 110(2) can generate a generative output 112(2) at step n+1 that focuses on aspects of the task represented in the semantic control data 108(2).

[0102] The control generative model 106 can obtain generative output 112 from the target generative model. The control generative model 106 can update the task-specific context data 104 based on the generative output 112 for use in steps after the current step.

[0103] For example, the control generative model 106 can obtain the generative output 112(1) generated by the first generative model 110(1) at step n, and can update the task-specific context data 104 based on the generative output 112(1) generated at step n. At step n+1, the control generative model 106 can generate control data 108(2) for the second generative model 110(2) based on the updated context data 104 that has been updated based on the generative output 112(1) generated at step n.

[0104] In some implementations, at the end of each step and after obtaining generative output 112 from the target generative model, the controlling generative model 106 can predict a progress score indicating the level of progress achieved toward task completion. The system 100 can provide cue input to the controlling generative model 106, requesting the model to predict the progress score indicating the level of progress achieved toward task completion. In some implementations, the controlling generative model 106 can generate a confidence level for the predicted progress score. The progress score can range from 0% to 100% completion. The confidence level can be a percentage or an actual value.

[0105] For example, at the end of step n and after obtaining generative output 112(1) from the first generative model, the generative model 106 can generate a predicted progress score of 80%, indicating a progress level of 80% in the direction of task completion, and the confidence level of the predicted progress score can be 70%.

[0106] In some implementations, the control generative model 106 may determine a progress score based on user input. In some implementations, the user input may include explicit feedback on whether the task or a subtask of the task has been completed. In some implementations, the user input may include user action data that indirectly indicates whether the task has been completed. For example, after receiving output generated by the generative model in response to a prompt requesting the generative model to solve a subtask, if context data 104 indicates that the user has performed additional research and analysis on the subtask, the control generative model 106 may predict that the particular subtask has not been completed. System 100 may provide prompt input to the control generative model 106 (e.g., prompt input including context data 104 indicating that the user has performed additional research and analysis on the task), and this prompt input may request the control generative model to predict whether a particular subtask has been completed.

[0107] Based on the predicted progress score and (optionally) its confidence level, the control generative model 106 can determine whether the task has been completed. If the control generative model 106 determines that the task has not been completed, the control generative model 106 may include the predicted progress score and (optionally) its confidence level in the context data 104 for the task at the next step.

[0108] After several steps, the control generative model 106 can determine that the task has been completed, for example, based on the predicted progress score and (optionally) its confidence level. For example, after receiving the final output 114 from the generative model in the final step, the control generative model 106 can determine that the task has been completed. The control generative model 106 can provide the final output 114 for the task to the device, for example, by displaying the final output 114 on a display screen of a mobile device.

[0109] Figure 2 This is an example user interface (UI) 200 that displays a dialogue generated by multiple generative models controlled by a control generative model.

[0110] The system receives input 202 for the task "Please book a hotel and flight for my travel to City ABC from April 1-5". Input 202 can be processed by a hotel booking agent and a flight booking agent. The hotel booking agent implements a generative model for hotel booking. The flight booking agent implements a generative model for flight booking. The system can process input 202 across multiple steps (such as step 1, step 2, ..., step N-1 and step N) through the two agents. At each step, either the hotel booking agent or the flight booking agent generates a generative output. After multiple steps are completed, the system can obtain a final output 206 for the task, for example, the hotel has been booked and the flight has been booked.

[0111] Sometimes, without the control generative models described in this specification, the interactions between the group of generative models can get out of control. For example, one or more generative models may dominate the dialogue, veer off in the wrong direction, or focus on irrelevant details, resulting in significant computational costs and being counterproductive.

[0112] For example, for Figure 2 In the "hotel and flight booking task," the flight booking agent dominates the conversation by asking too many questions in steps 2, 3, and 4. Specifically, the flight booking agent veers the conversation in the wrong direction by asking "would you like to read some news for City ABC?" in step 3 and "are you interested in traveling by train?" in step 4.

[0113] The system can use a control generative model to manage and control the task-specific outputs generated from multiple generative models. At intermediate steps, the control generative model can obtain context data for the task at the current step. The context data may include corresponding previous outputs generated by one or more generative models in the group at one or more previous steps prior to the intermediate step. For example, the task-specific context data 204 at step 5 may include previous outputs generated by the hotel booking agent and the flight booking agent at steps 1 through 4.

[0114] Based on contextual data 204 indicating that the flight booking agent generated irrelevant output in previous steps 1 through 4, the controlling generative model can select the hotel booking agent as the target generative model to generate generative output at step 5A. The controlling generative model can allow the flight booking agent to generate flight booking-related output after the hotel booking task is completed. For example, at step 5A, the hotel booking agent could generate the output "which area of ​​City ABC will you be staying in?", while the flight booking agent remains silent at step 5A.

[0115] In some examples, based on context data 204 instructing the flight booking agent to deviate from the direction of the dialogue in previous steps 1 to 4, the control generative model can select the flight booking agent as the target generative model for generating generative output at step 5B, and can generate semantic control data that makes the generative output generated by the flight booking agent focus on booking flights.

[0116] For example, at step 5B, the control data may include semantic control data that enables the flight booking agent to generate task-oriented output on flight booking without diverting the conversation to providing the user with news or other transportation options. Therefore, at step 5B, the flight booking agent could generate the output "One-way or round-wayflight?", which is focused on the flight booking task.

[0117] In some implementations, the generative control model can generate control data based on user input. For example, at step 4, in response to the flight booking agent generating the output "Are you interested in traveling by train?", the user provides "Not interested" in UI 200. The generative control model 106 can generate semantic control data 108, which enables the flight booking agent to generate task-oriented flight booking outputs.

[0118] Figure 3A This is a flowchart of Example Process 300A used to control the participation of two or more generative models. Figure 3B This is a flowchart of an example process 3B executed at each intermediate step in a multi-step process to control the participation of two or more generative models. Processes 300A and 300B will be described as being executed by a suitably programmed computer system such as system 100.

[0119] The system receives task-specific input to be processed by a set of generative models to generate a final output for that task (302). The system processes the input across multiple steps (304) using the set of generative models. The corresponding generative models in the set generate generative outputs at each of the multiple steps.

[0120] In the first step, after receiving task-specific input, the system can directly provide the input to one or more generative models without using a control generative model to regulate the engagement of the generative models. The one or more generative models can generate initial generative outputs in response to the task-specific input. The system can include the initial generated outputs in context data, which can then be used as input to the control generative model to generate control data. In some implementations, the system can begin using the control generative model after the preceding one or more steps.

[0121] For each intermediate step in the multiple steps, the system obtains task-specific context data at that intermediate step (306). In some implementations, the context data may include corresponding previous outputs generated by one or more generative models in the group at one or more previous steps preceding the intermediate step.

[0122] In some implementations, contextual data may include a progress score, indicating the level of progress made toward task completion. In some implementations, contextual data may include the level of importance of the task. In some implementations, contextual data may include the remaining computational budget for completing the task at intermediate steps of the generative model set. In some implementations, contextual data may include metadata about the generative model set.

[0123] The system generates control data (308) for the target generative models in the group based on the task-specific context data at the intermediate step. The system may provide cue input to the control generative model, which may request the control generative model to generate control data for the target generative model. In some implementations, the control data may include a computational budget for the target generative model, and this computational budget may define the amount of computational resources consumed by the target generative model in the intermediate step. In some implementations, the control data may define the number of lexical units in the output generated by the target generative model at the intermediate step.

[0124] In some implementations, control data can limit the number of lexical units in the output generated by the target generative model at intermediate steps, and limiting the number of lexical units in the output generated by the target generative model at intermediate steps does not decrease the final progress score of the task. The final progress score can be a measure of how well the task has been completed after the input has been processed across multiple steps to generate the final output for the task.

[0125] In some implementations, the control data may include semantic control data that enables the target generative model to generate outputs focused on one aspect of the task. In other implementations, the control data may enable the target generative model to generate outputs at intermediate steps that indicate that the context data has not changed.

[0126] In some implementations, for each intermediate step, the system can select a target generative model from the set of generative models based on the task-specific context data at that intermediate step. For example, the system can determine which generative model from the set should generate the output at the current step.

[0127] At intermediate steps, the system provides the task-specific context data and control data to the target generative model (310) in this group. The system can provide cue input to the target generative model, which may include the context data and control data. The cue input may request the target generative model to generate generative output based on the control data and context data. The target generative model processes the context data and control data and generates task-specific generative output.

[0128] The system obtains the output generated in response to context data and control data from the target generative model (312).

[0129] The system updates the task-specific context data based on the output from the target generative model (314). In the next step, the system can generate control data for the next target generative model based on the updated context data.

[0130] The system can determine whether the task is complete after receiving output from the target generative model. If the system determines that the task is not complete, it can use the next target generative model to generate generative output for the task at the next step. If the system determines that the task is complete, it can determine that the output from the target generative model at the current step is the final output, and the system can determine that the current step is the final step. The system can provide the final output to the user or perform further processing based on the final output.

[0131] Figure 4This is a flowchart of an example process 400 for training a control generative model. This process will be described as being performed by an appropriately programmed machine learning training system (e.g., a training system for training generative models).

[0132] The system receives the final output for the task generated by a set of generative models at the final step of a series of steps (402). In some implementations, the system can determine that the task is complete after receiving the final output from the generative models at the final step (404).

[0133] The system generates training data (406) for training the control generative model based on task-specific contextual data at each of the multiple steps. This training data may include multiple training examples. Each training example includes task-specific contextual data and its corresponding ground truth control data. The ground truth control data may be textual data, such as cue input to the generative model, or semantic control data that causes the target generative model to generate output focused on one aspect of the task. The ground truth control data may be values ​​of computational budgets (such as the number of lexical units in the output generated by the target generative model at a given step) or decoding parameters (e.g., bundle size).

[0134] For each or some of the multiple steps involved in completing a task, the system can determine a contribution score of the generative output generated by the generative model at each step to task completion. For example, the system can determine the contribution score of the generative output based on user feedback in a dialogue or based on a comparison of predicted progress scores before and after receiving the generative output. In some implementations, the system can determine the contribution score of the generative output by using a critical generative language model that has been trained to evaluate the output of another language model on some underlying tasks and identify defects in the language model's output. Based on the contribution to task completion, the system can determine the true value control data of the generative model at that step.

[0135] In some implementations, the system can determine the ground truth control data based on the generative model's contribution score to task completion. For example, if the contribution score is below a threshold, indicating that the generative model's contribution to task completion is insignificant, the ground truth control data for that step can include a smaller computational budget, such as fewer tokens in the generative model's output. If the contribution score is above a threshold, indicating that the generative model's contribution to task completion is significant, the ground truth control data for that step can include a larger computational budget, such as more tokens in the generative model's output.

[0136] In some implementations, the system can determine a contribution score for each of several aspects contributing to task completion (e.g., several subtasks of generative output generated by a generative model at a certain step). The system can then determine true value control data based on the contribution scores of the generative model to each aspect of task completion.

[0137] For example, if the generative model's contribution score to completing the first aspect of the task is above a threshold, and its contribution score to completing the second aspect of the task is below a threshold, indicating that the generative model contributes significantly to the first aspect of the task but not significantly to the second aspect, then the ground truth control data for the generative model at this step can include semantic control data. This semantic control data causes the generative model to generate an output focused on the first aspect of the task. For example, the semantic control data could include a prompt input: "please only generate a response to the first aspect of the task and do not generate a response to the second aspect of the task."

[0138] The system uses training data to train a control generative model (408). The system can train the control generative model over multiple iterations. At each iteration, for each training example, the system can generate predicted control data using the control generative model and the parameters at that iteration. The system can calculate the value of the loss function by comparing the predicted control data with the true control data of the training examples at that iteration.

[0139] If the control generative model is a generative language model, then at each iteration, the system can feed the corresponding ground truth words from the ground truth control data into the words from the previous iteration, and the system can generate the probability distribution of the next word in the current iteration based on the ground truth words from the previous iteration. The system can calculate the value of the loss function, which measures the probability of being assigned to the next ground truth word.

[0140] The system can update the parameters of the generative model based on the value of the loss function, for example, to minimize the value of the loss function. The system can update the parameters of the generative model through multiple iterations until training is complete.

[0141] In some implementations, the system can train a control generative model by fine-tuning a pre-trained model. The system can train the control generative model by adjusting the parameters of a previously trained model (rather than learning parameters from scratch). Therefore, fine-tuning requires fewer resources and less computation time compared to training from scratch.

[0142] In some implementations, the training system can generate simulated training data to train the control generative model. For example, the training system can convert data including stepwise data for solving a task (e.g., which may or may not involve multiple agents) into simulated training examples for training the control generative model in a multi-agent environment. In these training examples, each generative model in a set can generate a portion of the solution. In this way, the system can generate a large amount of training and evaluation data for training the control generative model.

[0143] This specification uses the term "configured" in conjunction with system and computer program components. For a system of one or more computers to be configured to perform a specific operation or action, it means that the system has software, firmware, hardware, or a combination thereof installed thereon that causes the system to perform those operations or actions in operation. For one or more computer programs configured to perform a specific operation or action, it means that the one or more programs include instructions that, when executed by a data processing device, cause that device to perform that operation or action.

[0144] Embodiments of the subject matter and functional operation described in this specification may be implemented in digital electronic circuit systems, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their equivalents), or in a combination of one or more of these. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by a data processing device or for controlling the operation of the data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of these. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals—e.g., machine-generated electrical, optical, or electromagnetic signals—to generate artificially generated propagation signals to encode information for transmission to a suitable receiver device for execution by the data processing device.

[0145] The term "data processing equipment" refers to data processing hardware and includes all kinds of equipment, devices, and machines for processing data, such as programmable processors, computers, or multiple processors or computers. The equipment may also include, or further include, off-the-shelf or custom-designed parallel processing subsystems, such as GPUs or another class of dedicated processing subsystems. The equipment may also include, or further include dedicated logic circuit systems, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the equipment may optionally include code that creates an execution environment for computer programs, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, or combinations thereof.

[0146] A computer program, also referred to or described as a program, software, software application, app, module, software module, script, or code, can be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages); and it can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but does not need to, correspond to a file in a file system. A program may be stored as a part of a file containing other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or code sections). A computer program can be deployed to execute on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected via a data communication network.

[0147] As used herein, "engine" or "software engine" refers to a software-implemented input / output system that provides an output distinct from the input. An engine may be a functional code block, such as a library, platform, software development kit ("SDK"), or object. Each engine may be implemented on any suitable type of computing device (e.g., server, mobile phone, tablet computer, laptop computer, music player, e-book reader, laptop or desktop computer, PDA, smartphone, or other fixed or portable device) including one or more processors and computer-readable media. Furthermore, two or more of the engines may be implemented on the same computing device or on different computing devices.

[0148] The processes and logic flows described in this specification can be executed by one or more programmable computers, which execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processes and logic flows can also be executed by a dedicated logic circuit system (e.g., an FPGA or ASIC) or by a combination of a dedicated logic circuit system and one or more programmable computers.

[0149] A computer suitable for executing computer programs can be based on a general-purpose or special-purpose microprocessor, or both, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are the central processing unit for executing instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by or incorporated into a special-purpose logic circuit system. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from or transfer data to or from them, or both. However, a computer does not necessarily have to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0150] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0151] To provide interaction with the user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) and a keyboard and pointing device (e.g., a mouse, trackball, or a sensitive display or other surface) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Additionally, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's device in response to a request received from a web browser. Furthermore, the computer can interact with the user by sending text messages or other forms of messages to a personal device—e.g., a smartphone running a messaging application—and receiving responsive messages from the user in response.

[0152] While this specification contains numerous details of specific implementations, these details should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features that may be characteristic of particular embodiments of a particular invention. Certain features described in the context of individual embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed in this way, in some cases one or more features from a claimed combination may be removed from that combination, and a claimed combination may involve sub-combinations or variations thereof.

[0153] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order shown or in sequential order, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0154] Specific embodiments of this subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired result. As an example, the processes depicted in the drawings do not necessarily require a specific order or sequence shown to achieve the desired result. In some cases, multitasking and parallel processing can be advantageous.

Claims

1. A method executed by one or more computers, the method comprising: Receive task-specific inputs to be processed by a set of generative models to generate a final output for the task; as well as The input is processed by the group of generative models across multiple steps, wherein the corresponding generative models in the group generate generative outputs at each of the multiple steps, and processing the input includes processing the input for each intermediate step of the multiple steps. Contextual data for the task at the intermediate step is obtained by controlling the generative model; The control generative model generates control data for the target generative model in the group based on the context data for the task at the intermediate step; The control generative model provides the context data and control data for the task at the intermediate step to the target generative model in the group; The control generative model obtains the output generated in response to the context data and the control data from the target generative model; as well as The control generative model updates the context data for the task based on the output from the target generative model.

2. The method of claim 1, wherein the control data includes a computational budget for the target generative model, and the computational budget defines the amount of computational resources consumed by the target generative model in the intermediate step.

3. The method of claim 1 or 2, wherein the control data defines the number of lexical units in the output generated at the intermediate step from the target generative model.

4. The method of any one of claims 1 to 3, wherein the control data limits the number of lexical units in the output generated at the intermediate step from the target generative model, and wherein limiting the number of lexical units in the output generated at the intermediate step from the target generative model does not reduce the final progress score of the task, the final progress score being a measure of the degree to which the task has been completed after the input has been processed across the plurality of steps to generate the final output for the task.

5. The method of any one of claims 1 to 4, wherein the control data includes semantic control data that causes the target generative model to generate an output focused on one aspect of the task.

6. The method of any one of claims 1 to 5, wherein the control data causes the target generative model to generate, at the intermediate step, the output indicating that the context data has not changed.

7. The method according to any one of claims 1 to 6, comprising: For each intermediate step The control generative model selects the target generative model from the group generative models based on the contextual data for the task at the intermediate step.

8. The method of any one of claims 1 to 7, wherein the context data includes corresponding previous outputs generated by one or more generative models in the group at one or more previous steps prior to the intermediate step.

9. The method of any one of claims 1 to 8, wherein the context data includes a progress score indicating the level of progress achieved in the direction of task completion.

10. The method of any one of claims 1 to 9, wherein the context data includes the importance level of the task.

11. The method of any one of claims 1 to 10, wherein the context data includes the remaining computational budget of the group generative model at the intermediate step for completing the task.

12. The method of any one of claims 1 to 11, wherein the context data includes metadata about the group generative model.

13. The method of any one of claims 1 to 12, further comprising: Receive the final output for the task generated by the group generative model at the final step of the plurality of steps; Training data for training the control generative model is generated based on the context data for the task at each of the plurality of steps; as well as The control generative model is trained using the training data.

14. The method of claim 13, comprising: After receiving the final output from the generative model at the final step, the controlling generative model determines that the task has been completed.

15. A system comprising one or more computers and one or more storage devices, the one or more storage devices storing instructions that, when executed by the one or more computers, are operable to cause the one or more computers to perform operations, the operations including: Receive task-specific inputs to be processed by a set of generative models to generate a final output for the task; as well as The input is processed by the group of generative models across multiple steps, wherein the corresponding generative models in the group generate generative outputs at each of the multiple steps, and processing the input includes processing the input for each intermediate step of the multiple steps. Contextual data for the task at the intermediate step is obtained by controlling the generative model; The control generative model generates control data for the target generative model in the group based on the context data for the task at the intermediate step; The control generative model provides the context data and control data for the task at the intermediate step to the target generative model in the group; The control generative model obtains the output generated in response to the context data and the control data from the target generative model; as well as The control generative model updates the context data for the task based on the output from the target generative model.

16. The system of claim 15, wherein the control data includes a computational budget for the target generative model, and the computational budget defines the amount of computational resources consumed by the target generative model in the intermediate step.

17. The system of claim 15 or 16, wherein the control data defines the number of lexical units in the output generated at the intermediate step from the target generative model.

18. The system of any one of claims 15 to 17, wherein the control data limits the number of lexical units in the output generated from the target generative model at the intermediate step, and wherein limiting the number of lexical units in the output generated from the target generative model at the intermediate step does not reduce the final progress score of the task, the final progress score being a measure of the degree to which the task has been completed after the input has been processed across the plurality of steps to generate the final output for the task.

19. The system of any one of claims 15 to 18, wherein the control data includes semantic control data that causes the target generative model to generate an output focused on one aspect of the task.

20. One or more non-transitory storage media, said one or more non-transitory storage media being encoded with instructions that, when executed by a computing device, cause the computing device to perform an operation, said operation including: Receive task-specific inputs to be processed by a set of generative models to generate a final output for the task; as well as The input is processed by the group of generative models across multiple steps, wherein the corresponding generative models in the group generate generative outputs at each of the multiple steps, and processing the input includes processing the input for each intermediate step of the multiple steps. Contextual data for the task at the intermediate step is obtained by controlling the generative model; The control generative model generates control data for the target generative model in the group based on the context data for the task at the intermediate step; The control generative model provides the context data and control data for the task at the intermediate step to the target generative model in the group; The control generative model obtains the output generated in response to the context data and the control data from the target generative model; as well as The control generative model updates the context data for the task based on the output from the target generative model.

Images