Generating natural language solutions for tasks by iteratively evolving populations of solutions using a generative neural network

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating natural language solutions for tasks. In one aspect, a method comprises: obtaining data defining a task; determining one or more natural language solutions to the task using a generative neural network; and selecting one or more natural language solutions for the task from the one or more populations of natural language solutions for the task.

Description

[0001] Attorney Docket No. 45288-0552WO1

[0002] GENERATING NATURAL LANGUAGE SOLUTIONS FOR TASKS BY ITERATIVELY EVOLVING POPULATIONS OF SOLUTIONS USING A GENERATIVE NEURAL

[0003] NETWORK

[0004] CROSS-REFERENCE TO RELATED APPLICATIONS

[0005] [1] This application claims priority to U.S. Provisional Application No. 63 / 702,111, filed October 1, 2024, and U.S. Provisional Application No. 63 / 742,593, filed January 7, 2025. The disclosures of the prior applications are considered part of and are incorporated by reference in their entirety in the disclosure of this application.

[0006] BACKGROUND

[0007] [2] This specification relates to processing data using machine learning models.

[0008] [3] Machine learning models receive an input and generate an output, e.g., a predicted output, based on the received input. Some machine learning models are parametric models and generate the output based on the received input and on values of the parameters of the model.

[0009] [4] Some machine learning models are deep models that employ multiple layers of models to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers that each apply a non-linear transformation to a received input to generate an output.

[0010] SUMMARY

[0011] [5] This specification generally describes a system implemented as computer programs on one or more computers in one or more locations that can determine one or more natural language solutions to a task using a generative neural network.

[0012] [6] A natural language solution to a task includes natural language text that represents a solution to the task. The solution can include, for example, a plan for a plan generation task, or text for a text generation task.

[0013] [7] In more detail, a natural language solution to a task may include a body of text that represents a resolution or plan for that task. While expressed in human-readable language, a solution is not limited to prose and can include structured or semi-structured data to be both interpretable by humans and parsable by machines. The specific form and content of the solution are dependent on the nature of the task. Attorney Docket No. 45288-0552WO1

[0014] [8] For example, for a planning or scheduling task, a natural language solution could be a detailed schedule defining the execution of jobs across multiple computational resources, a sequence of steps for an agent like a robot to perform a project in an environment, or a complex travel itinerary that satisfies various constraints like budget limitations and flight connectivity.

[0015] [9] For a text generation task, the solution may be the final textual artifact itself, such as a summary of a longer document, a translation of text into another language, a block of generated source code, or a creative text designed to encode a hidden message.

[0016]

[0010] For a design task, a solution may take the form of a textual specification describing an engineering design or a protocol outlining the steps of a scientific experiment.

[0017]

[0011] A natural language solution may be required to adhere to specific requirements beyond its primary goal, and it may be evaluated based on how well it satisfies one or more explicit constraints defined by the task. Furthermore, a solution may be required to conform to a specific machine-readable format, such as a JavaScript Object Notation (JSON) string, to facilitate automated processing and evaluation. A natural language solution may be an output that can be evaluated and assigned a fitness score, which characterizes how effectively it solves the given task in its entirety.

[0018]

[0012] A computing unit may be, e.g., a computer, a core within a computer having multiple cores, or other hardware or software, e.g., a dedicated thread, within a computer capable of independently perform operations. The computing units may include processor cores, processors, microprocessors, special-purpose logic circuitry, e g., an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit), or any other appropriate computing units. In some examples, the computing units are all the same type of computing unit. In other examples, the computing units may be different types of computing units. For example, one computing unit may be a CPU while other computing units may be GPUs.

[0019]

[0013] The subject matter described in this specification can be implemented in particular embodiments so as to realize one or more of the following advantages.

[0020]

[0014] The system described in this specification can generate natural language solutions to a task by iteratively evolving one or more populations of natural language solutions using a generative neural network. More specifically, for each population of natural language solutions, the system determines a “fitness score” for each natural language solution in the population, selects a set of “parent” natural language solutions based on the fitness scores, and then prompts Attorney Docket No. 45288-0552WO1 the generative neural network to generate new natural language solutions based on the parent natural language solutions. The generative neural network can merge and modify various aspects of the parent natural language solutions to generate new natural language solutions that may have higher fitness scores. The system can add the new natural language solutions to the population, and continue iteratively evolving the population in this manner to identify increasingly effective solutions for the task.

[0021]

[0015] The system performs an evolutionary search and optimization in the space of natural language and enables the identification of solutions to complex problems without requiring the transformation of those problems into formal language or notation. Evolving solutions in the space of natural language allows the system to identify solutions for tasks that are not formalizable (many practical, real-world tasks fall into this category). Further, formalization requires significant effort, expert knowledge, and understanding of optimization for individual problems, while the system described in this specification is broadly applicable without requiring formalization.

[0022]

[0016] Put another way, the system can address and solve complex problems typically handled by specialized, formal solvers (e.g., mathematical solvers, logic solvers, or optimization engines) that usually require problems to be expressed in a formal, structured way, such as mathematical equations, logic statements, or programming code. However, unlike traditional methods, the system does not require the user to manually convert or "translate" the problem into a formal language or notation (such as symbolic logic or algebraic expressions). Instead, the system can process problems described in natural language or less formal terms, without the need for precise, formal inputs that traditional solvers usually demand.

[0023]

[0017] The system thus provides a technical solution - in particular, combining and leveraging the complementary strengths of generative neural networks and evolutionary optimization - to a technical problem - in particular, identifying solutions to complex problems without requiring the transformation of those problems into formal language or notation. Further, the system can identify effective solutions for tasks within fixed maximum compute budgets, in contrast to traditional approaches that may be unable to identify solutions within the same compute budgets (or that may be unable to find solutions altogether, irrespective of compute budgets). The system thus provides an improvement in the functioning of computers by reducing consumption of computational resources (e.g., memory and computing power) used to generate solutions for Attorney Docket No. 45288-0552WO1 tasks, while also enabling solutions to be identified for tasks that are not solvable using traditional approaches.

[0024]

[0018] In some implementations, the system can computationally evolve multiple populations of natural language solutions substantially in parallel. Collectively the populations of natural language solutions can evolve to reflect a diverse variety of natural language solutions, thus allowing more thorough and efficient exploration of the space of possible solutions. By evolving the populations of natural language solutions substantially in parallel the system can reduce the latency required to identify effective natural language solutions.

[0025]

[0019] In some implementations, the system can generate natural language solutions that are more effective, e.g., more accurate, more efficient, satisfy a larger number of constraints, etc., solutions to the task than alternative techniques. For example, in some implementations where the system iteratively evolves multiple populations, the system can explore a wide area of the potential solution space to generate a superior solution. As another example, in implementations where the system generates a new natural language solution from multiple parent natural language solutions, the system can generate new solutions that combine high-performing features of the parent natural language solutions.

[0026]

[0020] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

[0027] BRIEF DESCRIPTION OF THE DRAWINGS

[0028]

[0021] FIG. 1 shows an example solution generation system.

[0029]

[0022] FIG. 2 is a flow diagram of an example process for generating a solution for a task.

[0030]

[0023] FIG. 3 is a flow diagram of an example process for selecting parent natural language solutions from a population of natural language solutions.

[0031]

[0024] FIG. 4 is a flow diagram of an example process for replacing a population of natural language solutions with a new population of natural language solutions.

[0032]

[0025] FIG. 5 shows the performance of an example solution generation system

[0033]

[0026] Like reference numbers and designations in the various drawings indicate like elements.

[0034] DETAILED DESCRIPTION Attorney Docket No. 45288-0552WO1

[0035]

[0027] FIG. 1 is a diagram of an example solution generation system 100. The solution generation system 100 is an example of a system implemented as computer programs on one or more computers in one or more locations, in which the systems, components, and techniques described below can be implemented.

[0036]

[0028] The solution generation system 100 is a system that generates one or more natural language solutions, e.g., a natural language solution 140, for a task 102 using a generative neural network 130. The generative neural network 130 is described in further detail below.

[0037]

[0029] The system 100 obtains data defining a task 102. The task 102 can be any appropriate task that can be performed according to a natural language solution 140 for the task. Examples of tasks are described in further detail below with reference to FIG. 2.

[0038]

[0030] The system iteratively evolves one or more populations of natural language solutions 108 to the task using the generative neural network 130. For example, the system can iteratively evolve each population of natural language solutions over multiple evolutionary iterations. Each population includes one or more natural language solutions. A population can also be referred to as a “set.”

[0039]

[0031] At each evolutionary iteration, the system can obtain data defining a fitness score distribution 112, e.g., using a scoring engine 110. The fitness score distribution defines a respective fitness score for each natural language solution in the population of natural language solutions. Each fitness score can be represented as a numerical value that characterizes how effectively the natural language solution solves the task.

[0040]

[0032] The system can be configured to generate a fitness score for each natural language solution, e.g., using a scoring function. Example scoring functions are described below with reference to FIG. 2.

[0041]

[0033] At each evolutionary iteration, the system can select one or more “parent” natural language solutions from the population of natural language solutions, e.g., using a selection engine 120. The selection engine 120 can be configured to select one or more natural language solutions based on the fitness score distribution 112 as the one or more “parent” natural language solutions.

[0042]

[0034] At each evolutionary iteration, the system can process a prompt 122 that includes each of the parent natural language solutions using the generative neural network 130 to generate a new natural language solution 132. Attorney Docket No. 45288-0552WO1

[0043]

[0035] The system can add the new natural language solution 132 to the population of the populations of natural language solutions 108.

[0044]

[0036] The system can select one or more natural language solutions, e.g., that includes the natural language solution 140, from the populations of natural language solutions 108. In some examples, the system can output data representing the natural language solution 140, e.g., for display to a user. In some examples, the system can perform the task according to the natural language solution 140.

[0045]

[0037] In some cases, the system 100 may not generate a natural language solution for the task using the generative neural network 130. For example, the populations of natural language solutions 108 may not include a natural language solution that has a fitness score that satisfies a threshold fitness score within a predetermined number of evolutionary iterations, or within the consumption of a predetermined amount of computational resources.

[0046]

[0038] In some of these cases, the system 100 can use an additional generative neural network to determine one or more natural language solutions to the task. For example, the system 100 can perform a similar process using the additional generative neural network to determine the one or more natural language solutions to the task as using the generative neural network 130 to determine the one or more natural language solutions to the task.

[0047]

[0039] In some implementations, the system 100 can include the additional generative neural network that is different from the generative neural network 130. The additional generative neural network can be a larger, e.g., have a larger number of parameters, neural network than the generative neural network 130. In some examples, the additional generative neural network can have a similar architecture as the generative neural network 130.

[0048]

[0040] The system can thus generate natural language solutions to a larger number or variety of tasks using a more powerful generative neural network, while saving the computational resources that would otherwise be required to generate solutions for every task using the more powerful generative neural network. By only using the additional generative neural network for more difficult tasks that the generative neural network 130 could not generate a solution for, the system maximizes the performance of generating a solution to the task while minimizing the computational cost.

[0049]

[0041] The generative neural network 130 can be trained to perform a language modeling task. The language modeling task can be, e g., a next token prediction task, where the generative neural Attomev Docket No. 45288-0552WO1 network is trained to predict the identity of a token in a sequence of tokens based on some or all of the preceding tokens in the sequence of tokens. (Throughout this specification, a “token” can represent, e.g., a character, or a word piece, or a word, and so forth).

[0050]

[0042] The generative neural network 130 can process a prompt that includes a sequence of tokens representing natural language data, and optionally, one or more additional types of data, e g., image data (that is, pixel-level data defining at least one intensity value for each of a multi-dimensional array of pixels; the image data may for example be the output of a camera arranged to capture the image data by imaging part of the real world), or audio data (e.g. sound samples at multiple corresponding times during a time period, such as sound samples captured from the real world by a microphone), or video data (that is image data for each of a sequence of frames of a video spanning a time period, the video data optionally also including audio data for the time period, and optionally being video data captured from the real world by a video camera), or genetic data, or protein sequence data, or protein structure data, or data representing chemical structures (e.g., in the form of SMILES strings), and so forth. The genetic data, protein sequence data protein structure data, or data representing a chemical structures may be data obtained experimentally (i.e. by measurement or imaging) from real-world samples of genetic material, protein material or chemical material. The generative neural network 130 may be operative to perform a task of generating a natural language output describing content of the image data, audio data, video data, sensor data, genetic data, protein sequence data or data representing a chemical structure. The task in this case may be text generation task. For example, the natural language solutions may be textual answers to a question defined by the sequence of tokens representing natural language data, where the question is about the one or more additional types of data. For example, the sequence of tokens representing natural language may define a question about content of the image data (e.g. “what object does the image show?), audio data (e.g. “whose voice is present in the audio data?) or video data (e.g. “what events occur in the video data?”), the genetic data, protein sequence data, protein structure data, or data representing a chemical structure (“when this genetic / protein / chemical material is in aqueous solution, how would its properties vary with the pH of the solution”?).

[0051]

[0043] The system 100 can generate natural language solutions for any appropriate type of task. For example, the task can be a summarization task, or a text translation task, or a paraphrasing task, or a code explanation task, or a plan generation task, or a medical diagnosis or prognosis task, and so forth. Attorney Docket No. 45288-0552WO1

[0052]

[0044] The generative neural network 130 can have any appropriate architecture that enables the generative neural network to perform its described functions. Examples of architectures for the generative neural network now follow.

[0053]

[0045] For example, the generative neural network 130 can be an auto-regressive generative neural network that generates each token in the output sequence conditioned on the preceding tokens in the output sequence and at least some of the tokens in the input sequence.

[0054]

[0046] For example, the generative neural network 130 can be configured to process an input sequence of tokens from a vocabulary of tokens to generate an output sequence of tokens from the vocabulary.

[0055]

[0047] More generally, the generative neural network 130 can be any appropriate neural network that receives an input sequence made up of tokens selected from a vocabulary and auto- regressively generates an output sequence made up of tokens from the vocabulary. For example, the generative neural network can be a Transformer-based generative neural network or a recurrent neural network-based generative neural network.

[0056]

[0048] In some situations, the generative neural network 130 can be referred to as an autoregressive neural network that auto-regressively generates an output sequence of tokens. More specifically, the auto-regressively generated output is created by generating each particular token in the output sequence conditioned on a current input sequence that includes any tokens that precede the particular text token in the output sequence, i.e., the tokens that have already been generated for any previous positions in the output sequence that precede the particular position of the particular token, and a context input that provides context for the output sequence.

[0057]

[0049] For example, the current input sequence when generating a token at any given position in the output sequence can include the input sequence and the tokens at any preceding positions that precede the given position in the output sequence. As a particular example, the current input sequence can include the input sequence followed by the tokens at any preceding positions that precede the given position in the output sequence. Optionally, the input and the current output sequence can be separated by one or more predetermined tokens within the current input sequence.

[0058]

[0050] More specifically, to generate a particular token at a particular position within an output sequence, the generative neural network 130 can process the current input sequence to generate a score distribution (e.g., a probability distribution) that assigns a respective score, e.g., a respective probability, to each token in the vocabulary of tokens. The generative neural network 130 can then Attorney Docket No. 45288-0552WO1 select, as the particular token, a token from the vocabulary using the score distribution. For example, the generative neural network 130 can greedily select the highest-scoring token or can sample, e.g., using nucleus sampling or another sampling technique, a token from the distribution.

[0059]

[0051] As a particular example, the generative neural network 130 can be an auto-regressive Transformer-based neural network that includes (i) a plurality of attention blocks, at least some of which apply a self-attention operation and (ii) an output subnetwork that processes an output of the last attention block to generate the score distribution.

[0060]

[0052] The generative neural network 130 can have any of a variety of Transformer-based neural network architectures. Examples of such architectures include those described in J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al. Training compute-optimal large language models, arXiv preprint arXiv:2203.15556, 2022; J.W. Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, H. F. Song, J. Aslanides, S. Henderson, R. Ring, S. Young, E. Rutherford, T. Hennigan, J. Menick, A. Cassirer,

[0061] R. Powell, G. van den Driessche, L. A. Hendricks, M. Rauh, P. Huang, A. Glaese, J. Welbl, S. Dathathri, S. Huang, J. Uesato, J. Mellor, I. Higgins, A. Creswell, N. McAleese, A.Wu, E. Eisen,

[0062] S. M. Jayakumar, E. Buchatskaya, D. Budden, E. Sutherland, K. Simonyan, M. Paganini, L. Sifre, L. Martens, X. L. Li, A. Kuncoro, A. Nematzadeh, E. Gribovskaya, D. Donato, A. Lazaridou, A. Mensch, J. Lespiau, M. Tsimpoukelli, N. Grigorev, D. Fritz, T. Sottiaux, M. Pajarskas, T. Pohlen, Z. Gong, D. Toyama, C. de Masson d’Autume, Y. Li, T. Terzi, V. Mikulik, I. Babuschkin, A. Clark, D. de Las Casas, A. Guy, C. Jones, J. Bradbury, M. Johnson, B. A. Hechtman, L. Weidinger, I. Gabriel, W. S. Isaac, E. Lockhart, S. Osindero, L. Rimell, C. Dyer, O. Vinyals, K. Ayoub, J. Stanway, L. Bennett, D. Hassabis, K. Kavukcuoglu, and G. Irving. Scaling language models: Methods, analysis & insights from training gopher. CoRR, abs / 2112.11446, 2021; Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv: 1910.10683, 2019; Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. Towards a humanlike open-domain chatbot. CoRR, abs / 2001.09977, 2020; and Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Attorney Docket No. 45288-0552WO1

[0063] Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020.

[0064]

[0053] In some implementations, the generative neural network 130 can have been trained to perform a language modeling task, e.g., a task that requires predicting, given a current sequence of tokens, the next token that follows the current sequence in the training data. As a particular example, the generative neural network 130 can be pre-trained on a maximum-likelihood objective on a large dataset of text, e.g., text that is publicly available from the Internet or another text corpus.

[0065]

[0054] FIG. 2 is a flow diagram of an example process 200 for generating a solution for a task. For convenience, the process 200 will be described as being performed by a system of one or more computers located in one or more locations. For example, a solution generation system, e g., the solution generation system 100 of FIG. 1, appropriately programmed, can perform the process 200.

[0066]

[0055] The system obtains data defining a task (step 202). The system can obtain the data defining the task from any appropriate source, , e.g., from a user or from another system, by way of an appropriate interface, e.g., an application programming interface (API) or a user interface (e.g., a graphical user interface). As an example, the other system can be an upstream system that generates data defining a task.

[0067]

[0056] In some examples, the task requires scheduling jobs across multiple computational resources. Each job can have a respective resource requirement, and each computational resource can have a respective total amount of resources. In these examples, each natural language solution to the task can define a schedule for executing the jobs across the plurality of computational resources.

[0068]

[0057] In some of these examples, the respective resource requirement for each job can be represented across multiple resource dimensions. Each computational resource is allocated a respective total resource amount along each resource dimension.

[0069]

[0058] As an example, the multiple resource dimensions can include a memory dimension and a compute dimension. Alternatively or in addition, the multiple resource dimensions can include one or more accelerator resource dimensions that correspond to resources provided by one or more hardware accelerators. Attorney Docket No. 45288-0552WO1

[0070]

[0059] In some examples, the computational resources are virtual machines that are each allocated a respective total amount of resources.

[0071]

[0060] In some of these examples, the system can execute the jobs across the plurality of computational resources in accordance with a schedule defined by a selected natural language solution for the task.

[0072]

[0061] In some examples, the task can include generating a sequence of steps, expressed in natural language, for an agent to perform a project in an environment. For example, the agent can include a robot (an electromechanical device) or an autonomous vehicle. In some of these examples, the environment can be a real-world environment. In some of these examples, the system can control the agent to perform actions in accordance with a selected natural language solution for the task. In some of these examples, the sequence of steps may include steps which are dependent conditions being met. The conditions may be defined based on observations of the environment captured by a sensor. That is, the sequence of steps may include steps which the agent should only carry out if a condition, defined based on the data output of the sensor, are met (e.g. a step in which the agent moves forward in the environment if the sensor data indicates that no object is in the agent’s path forward).

[0073]

[0062] In some examples, the task can be a natural language planning task, e.g., a travel planning task, a trip planning task, a meeting planning task, etc.

[0074]

[0063] As other examples, the task can be a text generation task, e.g., a hidden message encoding task, a code generation task, an experiment design task, an engineering design task, a text summarization task, or a text analysis task. The text generation task may be generating text descriptive of a real -world environment, or a human being, animal or object in the real -world environment based on a prompt which comprises data describing the real-world environment, such as sensor data output by at least one sensor. In an example, the task may a task of generating text describing a diagnosis or prognosis relating to a human or animal subject, based on a prompt which includes medical data describing the subject (e.g. heart rate data relating to the subject, results of chemical tests carried out on a biological sample obtained from the subject (e.g. a blood sample or a biopsy sample), or a medical image obtained by imaging the subject using a medical imaging system such as an X-ray or ultrasound imaging device).

[0075]

[0064] In some examples, the task defines one or more constraints that are to be satisfied by natural language solutions to the task. As an example, for a travel planning task, the one or more Attorney Docket No. 45288-0552WO1 constraints can include accommodation and dining constraints, budget limitations, or other preferences. As another example, for a trip planning task, the one or more constraints can include a number of days in a location, specific days for events, flight connectivity, etc. As another example, for a meeting planning task, the one or more constraints can include maximizing a number of people to meet with. As another example, for a hidden message encoding task, the task can include encoding a hidden message in generated text while adhering to formatting and spacing constraints.

[0076]

[0065] In some examples, the data defining the task can include a description of the task and the one or more constraints.

[0077]

[0066] The system determines one or more natural language solutions to the task using a generative neural network (step 204).

[0078]

[0067] To determine the one or more natural language solutions to the task, the system iteratively evolves one or more populations of natural language solutions to the task using the generative neural network (step 206).

[0079]

[0068] In some implementations, the system can initialize each population of natural language solutions prior to being iteratively evolved. For example, the system can initialize each population using multiple initial natural language solutions, e.g., more than 1 initial natural language solution, more than 10 initial natural language solutions, more than 100 initial natural language solutions, or more than 1,000 initial natural language solutions, that are each generated by processing a prompt that defines the task and includes an instruction to generate a natural language solution to the task using the generative neural network. For example, the prompt that defines the task can include the data defining the task.

[0080]

[0069] In some implementations, the system can update the initialized population of natural language solutions. For example, at each of multiple initialization iterations, the system can generate an updated solution for each natural language solution in the initialized population. The system can add the updated solutions for the multiple initialization iterations to the initialized population.

[0081]

[0070] For example, to generate the updated solution for each natural language solution at each initialization iteration, the system can process a prompt that includes the natural language solution of any previous initialization iterations using the generative neural network. In some examples, the prompt can include one or more of a fitness score for the natural language Attorney Docket No. 45288-0552WO1 solution of the previous initialization iteration, a natural language evaluation of an effectiveness of the natural language solution in solving the task, or an instruction to generate a new natural language solution with a higher fitness score than the natural language solution included in the prompt. In some examples, the natural language evaluation can include a critique of the natural language solution generated by the generative neural network. For example, the system can generate the critique by processing a prompt that includes: (i) the natural language solution of the previous initialization iteration, and (ii) an instruction to generate a critique of the natural language solution of the previous initialization iteration, using the generative neural network to generate a natural language output that defines the critique of the natural language solution of the previous initialization iteration.

[0082]

[0071] In some implementations, the system can iteratively evolve multiple populations of natural language solutions substantially in parallel. The system can iteratively evolve, e.g., more than 2, more than 5, more than 10, more than 100, of the populations of natural language solutions substantially in parallel. As an example, the system can iteratively evolve multiple populations in parallel by performing steps 208-214 at each of multiple evolutionary iterations for each of the multiple populations in parallel, e.g., simultaneously.

[0083]

[0072] This parallel processing is achieved by distributing the workload of evolving different populations across a plurality of computing units. The computing units may be physically separate processor devices, such as devices having independent respective clock signals, or devices which are separate respective areas of one or more integrated circuits.

[0084]

[0073] For instance, the evolution of a first population can be assigned to a first computing unit (e.g., a specific processor core or a dedicated thread), while the evolution of a second population is simultaneously handled by a second computing unit.

[0085]

[0074] Each computing unit can then independently and concurrently manage the complete evolutionary loop for its assigned population(s). This includes performing the operations of obtaining a fitness score distribution, selecting parent solutions , processing prompts with the generative neural network to generate new solutions, and adding those new solutions back into the population. By having these operations for different populations occur at the same time on distinct hardware or software resources, the system can more thoroughly explore the space of possible solutions in less time, thereby reducing the latency required to identify effective natural language solutions. The computing units may also be configured to communicate periodically to Attorney Docket No. 45288-0552WO1 enable the sharing of best -performing solutions between populations, further accelerating the overall evolution process.

[0086]

[0075] For each population of natural language solutions, the system iteratively evolves the population of natural language solutions by performing steps 208-214 at each of multiple evolutionary iterations.

[0087]

[0076] In some implementations, the number of evolutionary iterations is predetermined. In some other implementations, the system can perform the steps 208-214 at each of multiple evolutionary iterations until a termination criterion is met, e.g., the population of natural language solutions includes a natural language solution that has a fitness score that satisfies a threshold fitness score, a threshold amount of computational resources, e.g., computing time, has been consumed, etc.

[0088]

[0077] The system obtains data defining a fitness score distribution (step 208). The fitness score distribution defines a respective fitness score for each natural language solution in the population of natural language solutions. A fitness score for a natural language solution characterizes how effectively the natural language solution solves the task.

[0089]

[0078] In some implementations, for each natural language solution in the population of natural language solutions, the fitness score for the natural language solution can be generated by processing the natural language solution using a scoring function.

[0090]

[0079] In some examples, the scoring function can include the generative neural network (for example, the generative neural network may generate the output of the scoring function). In these examples, the system can generate the fitness score for each natural language solution in the population of natural language solutions by processing a prompt that comprises: (i) data characterizing the task, (ii) the natural language solution to the task, and (iii) an instruction to generate a fitness score for the natural language solution, using the generative neural network to generate the fitness score for the natural language solution.

[0091]

[0080] As another example, the scoring function can include a machine learning model that is configured, e.g., through training, to generate a fitness score for the task given a natural language solution. As another example, the scoring function can include a predefined mapping between the task and the fitness score. In examples where the task defines one or more constraints, the scoring function can include a predefined mapping between the one or more constraints for the Attorney Docket No. 45288-0552WO1 task and the fitness score. For example, the scoring function can apply a penalty to the fitness score for each constraint that is not satisfied.

[0092]

[0081] In some examples where the task defines one or more constraints that are to be satisfied by natural language solutions to the task, the fitness score for each natural language solution in the population of natural language solutions can characterize whether the natural language solution satisfies the one or more constraints.

[0093]

[0082] In some examples, for each natural language solution in the population of natural language solutions, the fitness score for the natural language solution further characterizes whether the natural language solution conforms with a required format. For example, the required format can be a JavaScript Object Notation (JSON) string format.

[0094]

[0083] The system selects one or more parent natural language solutions from the population of natural language solutions (step 210). For example, the system can select the one or more parent natural language solutions based on the fitness score distribution. An example process for selecting the one or more parent natural language solutions is described below with reference to FIG. 3.

[0095]

[0084] The system generates a new natural language solution to the task (step 212). For example, the system can generate the new natural language solution to the task by processing a prompt that includes each of the parent natural language solutions using the generative neural network.

[0096]

[0085] In some implementations, the prompt can further include an instruction to generate a new natural language solution with a higher fitness score than the parent natural language solutions included in the prompt.

[0097]

[0086] In some implementations, the prompt can further include a respective fitness score for each parent natural language solution included in the prompt.

[0098]

[0087] In some implementations, the prompt can further include, for each parent natural language solution, a natural language evaluation of an effectiveness of the parent natural language solution in solving the task. In examples where the task defines one or more constraints that are to be satisfied by the natural language solution to the task, for each parent natural language solution, the natural language evaluation of the effectiveness of the parent natural language solution in solving the task can include, for any constraint that is not satisfied Attorney Docket No. 45288-0552WO1 by the parent natural language solution, a natural language statement that the parent natural language solution does not satisfy the constraint.

[0099]

[0088] As an example, for a meeting planning task, the natural language evaluation of the effectiveness of the parent natural language solution in solving the task can include how many constraints are violated, how many valid meeting events are included in the schedule, and whether the plan follows the required format.

[0100]

[0089] In some implementations, for each parent natural language solution included in the prompt, the system can process a prompt that includes: (i) the parent natural language solution, and (ii) an instruction to generate a critique of the parent natural language solution, using the generative neural network to generate a natural language output that defines the critique of the parent natural language solution. In these implementations, for each parent natural language solution, the natural language evaluation of the effectiveness of the parent natural language solution in solving the task can include the critique of the parent natural language solution that is generated by the generative neural network.

[0101]

[0090] The system adds the new natural language solution to the population of natural language solutions (step 214).

[0102]

[0091] In some implementations, at each evolutionary iteration, the system can generate multiple new natural language solutions over multiple prompting iterations. In these examples, the system can add each new natural language solution generated at each of the multiple prompting iterations to the population of natural language solutions.

[0103]

[0092] For example, at each of multiple prompting iterations, the system can process a prompt that includes: (i) each of the parent natural language solutions, and (ii) new natural language solutions generated by the generative neural network at any previous prompting iterations, using the generative neural network to generate a new natural language solution for the task at the prompting iteration. In some examples, at each of the multiple prompting iterations, the prompt processed by the generative neural network at the prompting iteration further includes a respective fitness score for each of the new natural language solutions generated by the generative neural network at any previous prompting iterations.

[0104]

[0093] In some implementations, for each population of natural language solutions, at one or more of the multiple evolutionary iterations, the system can select multiple best-performing natural language solutions from one or more other populations of natural language solutions Attorney Docket No. 45288-0552WO1 based on respective fitness scores of the multiple best-performing natural language solutions. For example, each of the multiple best-performing natural language solutions can have the highest fitness scores of the one or more other populations of natural language solutions. The system can add the multiple best-performing natural language solutions selected from the one or more other populations of natural language solutions to the population of natural language solutions.

[0105]

[0094] For example, after one or more of the multiple evolutionary iterations for a population i, the system can clone the best-performing natural language solutions from the population i to a next population i+1 out of 1...N populations. For population N, the system can clone the bestperforming natural language solutions from the population N to the first population. The system can thus accelerate the overall evolution process.

[0106]

[0095] The system selects one or more natural language solutions for the task (step 216). For example, the system can select the one or more natural language solutions from the one or more populations of natural language solutions for the task.

[0107]

[0096] In some examples, the system can select the N best-performing natural language solutions from the one or more populations of natural language solutions, where N can be, e.g., 1, 5, 10, etc.

[0108]

[0097] As an example, the system can select the one or more natural language solutions from the one or more populations of natural language solutions based on the respective fitness scores of the selected one or more natural language solutions. For example, each of the selected natural language solutions can have the highest fitness scores of the one or more populations of natural language solutions.

[0109]

[0098] In some examples, the system replaces a population of natural language solutions with a new population of natural language solutions, as described in further detail below with reference to FIG. 4.

[0110]

[0099] In some implementations, the system may determine that the one or more natural language solutions to the task may not satisfy the task. For example, the system can determine that the one or more natural language solutions to the task do not satisfy the task. For example, the system can determine that the one or more natural language solutions to the task do not satisfy the task if each of the one or more natural language solutions to the task does not satisfy a threshold fitness score within a threshold number of evolutionary iterations. As a particular Attorney Docket No. 45288-0552WO1 example, the threshold number of evolutionary iterations can be predetermined. As another example, the threshold number of evolutionary iterations can be based on a predetermined amount of computational resources consumed, e.g., during the determination of the one or more natural language solutions to the task.

[0111]

[0100] In response to determining that the one or more natural language solutions to the task does not satisfy the task, the system determines one or more additional natural language solutions to the task using an additional generative neural network. The additional generative neural network can include a larger number of parameters than the generative neural network.

[0112]

[0101] The system can perform a similar process to the process 200 to determine the one or more additional natural language solutions to the task. Further details are described above with reference to iteratively evolving populations of natural language solutions.

[0113]

[0102] For example, the system can iteratively evolve one or more additional populations of additional natural language solutions to the task using the additional generative neural network.

[0114]

[0103] As an example, for each additional population of natural language solutions, the system can iteratively evolve the additional population of additional natural language solutions using the additional generative neural network. At each of multiple additional evolutionary iterations, the system can obtain data defining an additional fitness score distribution that defines a respective additional fitness score for each additional natural language solution in the additional population of additional natural language solutions. An additional fitness score for a natural language solution characterizes how effectively the additional natural language solution solves the task. The system can select one or more additional parent natural language solutions from the additional population of additional natural language solutions based on the additional fitness score distribution. The system can process an additional prompt that includes each of the additional parent natural language solutions using the additional generative neural network to generate an additional new natural language solution to the task. The system can add the additional new natural language solution to the task to the additional population of additional natural language solutions.

[0115]

[0104] The system can select the one or more additional natural language solutions for the task from the one or more additional populations of additional natural language solutions for the task.

[0116]

[0105] FIG. 3 is a flow diagram of an example process for selecting parent natural language solutions from a population of natural language solutions. For convenience, the process 300 will Attorney Docket No. 45288-0552WO1 be described as being performed by a system of one or more computers located in one or more locations. For example, a solution generation system, e.g., the solution generation system 100 of FIG. 1, appropriately programmed, can perform the process 300.

[0117]

[0106] The system can perform the process 300 as part of step 210 described above with reference to FIG. 2.

[0118]

[0107] The system processes the fitness score distribution to generate a probability distribution over the population of natural language solutions (step 302). The probability distribution can assign a probability of selecting each natural language solution in the population of natural language solutions.

[0119]

[0108] In some implementations, the system can process the fitness score distribution using a soft-max function parameterized by a temperature parameter having a value strictly greater than one to generate the probability distribution. For example, for higher temperature parameters, e.g., having a value greater than one, the probability distribution can include probabilities that are more similar to each other, encouraging exploration and diversity.

[0120]

[0109] The system stochastically samples each parent natural language solution from the probability distribution over the population of natural language solutions (step 304). For example, the system can randomly select each parent natural language solution from the probability distribution.

[0121] [HO] For example, the system can perform Boltzmann tournament selection to sample each parent natural language solution. Thus, higher-performing solutions are more likely to be selected, while other solutions can be occasionally selected for diversity.

[0122] [Hl] FIG. 4 is a flow diagram of an example process for replacing a population of natural language solutions with a new population of natural language solutions. For convenience, the process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a solution generation system, e.g., the solution generation system 100 of FIG. 1, appropriately programmed, can perform the process 400.

[0123]

[0112] The system can perform the process 400 for one or more populations of natural language solutions. In some examples, the system can perform the process 400 at one or more evolutionary iterations, e.g., at regular intervals.

[0124]

[0113] The system determines that the population of natural language solutions satisfies a criterion to be replaced (step 402). For example, the system can determine that the population of Attomev Docket No. 45288-0552WO1 natural language solutions satisfies the criterion to be replaced based on a measure of central tendency, e.g., mean, median, mode, etc., of fitness scores of natural language solutions included in the population. In some examples, the criterion to be replaced can include whether the measure of central tendency satisfies a threshold measure. In some examples, the criterion to be replaced can include whether the measure of central tendency is among a number of the lowest measures of central tendencies of the populations of natural language solutions, e.g., the lowest, or the two lowest, etc. In some implementations, the number of lowest average fitness scores is predetermined.

[0125]

[0114] As an example, the system can determine that the population of natural language solutions satisfies the criterion to be replaced if the average fitness score of the population of natural language solution is lower than a threshold average fitness score, or if the average fitness score is among a number of lowest average fitness scores of the populations of natural language solutions.

[0126]

[0115] In response, the system replaces the population of natural language solution with a new population of natural language solutions (step 404).

[0127]

[0116] The system can perform steps 406-408 to replace the population of natural language solutions with a new population of natural language solutions.

[0128]

[0117] The system selects multiple best-performing natural language solutions from the populations of natural language solutions (step 406). For example, the system can select the best-performing natural language solutions based on respective fitness scores of the multiple best-performing natural language solutions.

[0129]

[0118] In some examples, the system can select the N best-performing natural language solutions over all of the one or more populations of natural language solutions, where N can be, e.g., 2, 5, 10, etc.

[0130]

[0119] As an example, the system can select multiple best-performing natural language solutions from the one or more populations of natural language solutions based on the respective fitness scores of the selected one or more natural language solutions. For example, each of the selected natural language solutions can have the highest fitness scores of the one or more populations of natural language solutions.

[0131]

[0120] The system initializes the new population of natural language solutions using the multiple best-performing natural language solutions (step 408). Attorney Docket No. 45288-0552WO1

[0132]

[0121] For example, the system can add the multiple best-performing natural language solutions to the new population of natural language solutions.

[0133]

[0122] As another example, the system can process a prompt that includes: (i) the multiple bestperforming natural language solutions, and (ii) an instruction to generate new natural language solutions to the task that are substantially different from the multiple best-performing natural language solutions, using the generative neural network to generate new natural language solutions to the task. The system can add the new natural language solutions to the new population of natural language solutions.

[0134]

[0123] As another example, the system can process a prompt that includes: (i) the multiple bestperforming natural language solutions, and (ii) an instruction to select natural language solutions from the multiple best-performing natural language solutions to the task that are substantially different from each other, using the generative neural network to generate new natural language solutions to the task. The system can add the new natural language solutions to the new population of natural language solutions.

[0135]

[0124] In some implementations, the system can iteratively evolve the new population of natural language solutions using the generative neural network. An example process for iteratively evolving a population of natural language solutions is described above with reference to step 206 of FIG. 2.

[0136]

[0125] FIG. 5 shows the performance of an example solution generation system. The example solution generation system is described above with reference to FIGS. 1-4.

[0137]

[0126] FIG. 5 shows the performance of the example solution generation system in terms of success rate, number of generative neural network calls, input tokens, output tokens, and API cost, on different tasks, compared to alternative techniques. The alternative techniques include generating a solution using a single forward pass of a generative neural network (“1-Pass”), independently generating candidate solutions until a successful solution is found (“Best-of-N”), and an iterative search baseline that includes multiple independent threads of refinement

[0138] (“ S equenti al -Revi si on+”) .

[0139]

[0127] FIG. 5 shows that the example solution generation system (“Mind Evolution”) and (“+pro”) has a higher success rate than alternative techniques, while also using fewer input tokens than the iterative search baseline. Attorney Docket No. 45288-0552WO1

[0140]

[0128] FIG. 5 also shows that in implementations where the example solution generation system includes an additional larger generative neural network for generating solutions for unsolved tasks, the example solution generation system limits the amount of computational resources that would otherwise be required to use the larger generative neural network for all tasks.

[0141]

[0129] This specification uses the term “configured” in connection with systems and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0142]

[0130] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

[0143]

[0131] The term “data processing apparatus” refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for Attorney Docket No. 45288-0552WO1 computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0144]

[0132] A computer program, which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

[0145]

[0133] In this specification the term “engine” is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Generally, an engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines can be installed and running on the same computer or computers.

[0146]

[0134] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.

[0147]

[0135] Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated Attorney Docket No. 45288-0552WO1 in, special purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0148]

[0136] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD- ROM and DVD-ROM disks.

[0149]

[0137] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone that is running a messaging application, and receiving responsive messages from the user in return.

[0150]

[0138] Data processing apparatus for implementing machine learning models can also include, for example, special-purpose hardware accelerator units for processing common and computeintensive parts of machine learning training or production, i.e., inference, workloads.

[0151]

[0139] Machine learning models can be implemented and deployed using a machine learning framework, e.g., a TensorFlow framework, or a Jax framework. Attorney Docket No. 45288-0552WO1

[0152]

[0140] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g., the Internet.

[0153]

[0141] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data, e.g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client. Data generated at the user device, e.g., a result of the user interaction, can be received at the server from the device.

[0154]

[0142] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0155]

[0143] Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, Attorney Docket No. 45288-0552WO1 to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0156]

[0144] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

[0157]

[0145] This specification also provides the subject-matter of the following clauses:

[0158]

[0146] Clause 1. A method performed by one or more computers, the method comprising: obtaining data defining a task; determining one or more natural language solutions to the task using a generative neural network, comprising: iteratively evolving one or more populations of natural language solutions to the task using the generative neural network; wherein for each population of natural language solutions, iteratively evolving the population of natural language solutions using the generative neural network comprises, at each of a plurality of evolutionary iterations: obtaining data defining a fitness score distribution that defines a respective fitness score for each natural language solution in the population of natural language solutions, wherein a fitness score for a natural language solution characterizes how effectively the natural language solution solves the task; selecting one or more parent natural language solutions from the population of natural language solutions based on the fitness score distribution; processing a prompt that includes each of the parent natural language solutions using the generative neural network to generate a new natural language solution to the task; and Attorney Docket No. 45288-0552WO1 adding the new natural language solution to the task to the population of natural language solutions; and selecting one or more natural language solutions for the task from the one or more populations of natural language solutions for the task.

[0159]

[0147] Clause 2. The method of clause 1, wherein for each natural language solution in the population of natural language solutions, the fitness score for the natural language solution is generated by processing the natural language solution using a scoring function.

[0160]

[0148] Clause 3. The method of clause 2, wherein the scoring function comprises the generative neural network; and wherein for each natural language solution in the population of natural language solutions, generating the fitness score for the natural language solution comprises: processing a prompt that comprises: (i) data characterizing the task, (ii) the natural language solution to the task, and (iii) an instruction to generate a fitness score for the natural language solution, using the generative neural network to generate the fitness score for the natural language solution.

[0161]

[0149] Clause 4. The method of any preceding clause, wherein the task defines one or more constraints that are to be satisfied by natural language solutions to the task; and wherein for each natural language solution in the population of natural language solutions, the fitness score for the natural language solution characterizes whether the natural language solution satisfies the one or more constraints.

[0162]

[0150] Clause 5. The method of any preceding clause, wherein for each natural language solution in the population of natural language solutions, the fitness score for the natural language solution further characterizes whether the natural language solution conforms with a required format.

[0163]

[0151] Clause 6. The method of clause 5, wherein the required format is a JavaScript Object Notation (JSON) string format.

[0164]

[0152] Clause 7. The method of any preceding clause, wherein the prompt processed by the generative neural network further comprises an instruction to generate a new natural language solution with a higher fitness score than the parent natural language solutions included in the prompt. Attorney Docket No. 45288-0552WO1

[0165]

[0153] Clause 8. The method of any preceding clause, wherein the prompt processed by the generative neural network further comprises a respective fitness score for each parent natural language solution included in the prompt.

[0166]

[0154] Clause 9. The method of any preceding clause, wherein the prompt processed by the generative neural network further comprises, for each parent natural language solution, a natural language evaluation of an effectiveness of the parent natural language solution in solving the task.

[0167]

[0155] Clause 10. The method of clause 9, wherein the task defines one or more constraints that are to be satisfied by the natural language solution to the task; and for each parent natural language solution, the natural language evaluation of the effectiveness of the parent natural language solution in solving the task comprises, for any constraint that is not satisfied by the parent natural language solution, a natural language statement that the parent natural language solution does not satisfy the constraint.

[0168]

[0156] Clause 11. The method of any one of clauses 9-10, further comprising, for each parent natural language solution included in the prompt: processing a prompt that comprises: (i) the parent natural language solution, and (ii) an instruction to generate a critique of the parent natural language solution, using the generative neural network to generate a natural language output that defines the critique of the parent natural language solution; wherein for each parent natural language solution, the natural language evaluation of the effectiveness of the parent natural language solution in solving the task comprises the critique of the parent natural language solution that is generated by the generative neural network.

[0169]

[0157] Clause 12. The method of any preceding clause, wherein processing the prompt that includes each of the parent natural language solutions using the generative neural network to generate the new natural language solution to the task comprises, at each of a plurality of prompting iterations: processing a prompt that includes: (i) each of the parent natural language solutions, and (ii) new natural language solutions generated by the generative neural network at any previous prompting iterations, using the generative neural network to generate a new natural language solution for the task at the prompting iteration. Attorney Docket No. 45288-0552WO1

[0170]

[0158] Clause 13. The method of clause 12, wherein at each of the plurality of prompting iterations, the prompt processed by the generative neural network at the prompting iteration further includes a respective fitness score for each of the new natural language solutions generated by the generative neural network at any previous prompting iterations.

[0171]

[0159] Clause 14. The method of any one of clauses 12-13, wherein adding the new natural language solution to the task to the population of natural language solutions comprises: adding each new natural language solution generated at each of the plurality of prompting iterations to the population of natural language solutions.

[0172]

[0160] Clause 15. The method of any preceding clause, wherein each population of natural language solutions is initialized, prior to being iteratively evolved, using a plurality of initial natural language solutions that are each generated by processing a prompt that defines the task and includes an instruction to generate a natural language solution to the task using the generative neural network.

[0173]

[0161] Clause 16. The method of any preceding clause, wherein selecting one or more parent natural language solutions from the population of natural language solutions based on the fitness score distribution comprises: processing the fitness score distribution to generate a probability distribution over the population of natural language solutions; and stochastically sampling each parent natural language solution from the probability distribution over the population of natural language solutions.

[0174]

[0162] Clause 17. The method of clause 16, wherein processing the fitness score distribution to generate the probability distribution over the population of natural language solutions comprises: processing the fitness score distribution using a soft-max function parameterized by a temperature parameter having a value strictly greater than one.

[0175]

[0163] Clause 18. The method of any preceding clause, further comprising, for each population of natural language solutions, at one or more of the plurality of evolutionary iterations: selecting a plurality of best-performing natural language solutions from one or more other populations of natural language solutions based on respective fitness scores of the plurality of best-performing natural language solutions; and Attorney Docket No. 45288-0552WO1 adding the plurality of best-performing natural language solutions selected from the one or more other populations of natural language solutions to the population of natural language solutions.

[0176]

[0164] Clause 19. The method of any preceding clause, further comprising, for one or more populations of natural language solutions: determining that the population of natural language solutions satisfies a criterion to be replaced based on a measure of central tendency of fitness scores of natural language solutions included in the population; and in response, replacing the population of natural language solutions with a new population of natural language solutions, comprising: selecting a plurality of best-performing natural language solutions from the populations of natural language solutions based on respective fitness scores of the plurality of best-performing natural language solutions; and initializing the new population of natural language solutions using the plurality of best-performing natural language solutions.

[0177]

[0165] Clause 20. The method of clause 19, wherein initializing the new population of natural language solutions using the plurality of best-performing natural language solutions comprises: adding the plurality of best-performing natural language solutions to the new population of natural language solutions.

[0178]

[0166] Clause 21. The method of clause 19 or clause 20, wherein initializing the new population of natural language solutions using the plurality of best-performing natural language solutions comprises: processing a prompt that comprises: (i) the plurality of best-performing natural language solutions, and (ii) an instruction to generate new natural language solutions to the task that are substantially different from the plurality of best-performing natural language solutions, using the generative neural network to generate new natural language solutions to the task; and adding the new natural language solutions to the new population of natural language solutions.

[0179]

[0167] Clause 22. The method of any one of clauses 19-21, further comprising iteratively evolving the new population of natural language solutions using the generative neural network. Attorney Docket No. 45288-0552WO1

[0180]

[0168] Clause 23. The method of any preceding clause, wherein the generative neural network is an autoregressive neural network.

[0181]

[0169] Clause 24. The method of any preceding clause, wherein the generative neural network has been trained to perform a language modeling task.

[0182]

[0170] Clause 25. The method of any preceding clause, wherein the generative neural network has a Transformer architecture.

[0183]

[0171] Clause 26. The method of any preceding clause, wherein iteratively evolving one or more populations of natural language solutions to the task using the generative neural network comprises: iteratively evolving a plurality of populations of natural language solutions substantially in parallel.

[0184]

[0172] Clause 27. The method of any preceding clause, wherein the task requires scheduling jobs across a plurality of computational resources, each job having a respective resource requirement, and each computational resource having a respective total amount of resources; and wherein each natural language solution to the task defines a schedule for executing the jobs across the plurality of computational resources.

[0185]

[0173] Clause 28. The method of clause 27, wherein the respective resource requirement for each job is represented across a plurality of resource dimensions, and wherein each computational resource is allocated a respective total resource amount along each resource dimension.

[0186]

[0174] Clause 29. The method of clause 28, wherein the plurality of resource dimensions comprise a memory dimension and a compute dimension.

[0187]

[0175] Clause 30. The method of clause 28 or 29, wherein the plurality of resource dimensions comprise one or more accelerator resource dimensions that correspond to resources provided by one or more hardware accelerators.

[0188]

[0176] Clause 31. The method of any one of clauses 27-30, wherein the computational resources are virtual machines that are each allocated a respective total amount of resources.

[0189]

[0177] Clause 32. The method of any one of clauses 27-31, further comprising executing the jobs across the plurality of computing resources in accordance with a schedule defined by a selected natural language solution for the task. Attorney Docket No. 45288-0552WO1

[0190]

[0178] Clause 33. The method of any preceding clause, wherein the task requires generating a sequence of steps, expressed in natural language, for an agent to perform a project in an environment.

[0191]

[0179] Clause 34. The method of clause 33, wherein the agent comprises a robot or an autonomous vehicle.

[0192]

[0180] Clause 35. The method of clause 33 or clause 34, wherein the environment is a real- world environment.

[0193]

[0181] Clause 36. The method of any one of clauses 33-35, further comprising controlling the agent to perform actions in accordance with a selected natural language solution for the task.

[0194]

[0182] Clause 37. The method of any one of clauses 1-36 in which the data defining the task comprises one or more of image data, audio data, video data, sensor data output by a sensor and describing a real-world environment, genetic data, protein sequence data or data representing a chemical structure, the genetic data, protein sequence data or data representing a chemical structure being data obtained experimentally from real-world samples of genetic material, protein material or chemical material.

[0195]

[0183] Clause 38. The method of any one of clauses 1-37 in which there are plural populations of natural language solutions, and said iteratively evolving is performed in parallel for different ones of said populations of natural language solutions, by different corresponding ones of a plurality of computing units.

[0196]

[0184] Clause 39. The method of any one of clauses 1-38, further comprising: determining that the one or more natural language solutions to the task does not satisfy the task; and in response to determining that the one or more natural language solutions to the task does not satisfy the task, determining one or more additional natural language solutions to the task using an additional generative neural network.

[0197]

[0185] Clause 40. The method of clause 39, wherein the additional generative neural network comprises a larger number of parameters than the generative neural network.

[0198]

[0186] Clause 41. The method of any one of clauses 39-40, wherein determining that the one or more natural language solutions to the task does not satisfy the task comprises determining that each of the one or more natural language solutions to the task does not satisfy a threshold fitness score within a threshold number of evolutionary iterations. Attorney Docket No. 45288-0552WO1

[0199]

[0187] Clause 42. The method of any one of clauses 39-41, wherein the threshold number of evolutionary iterations is predetermined.

[0200]

[0188] Clause 43. The method of any one of clauses 39-41, wherein the threshold number of evolutionary iterations is based on a predetermined amount of computational resources consumed.

[0201]

[0189] Clause 44. The method of any one of clauses 39-43, wherein determining one or more additional natural language solutions to the task using an additional generative neural network comprises iteratively evolving one or more additional populations of additional natural language solutions to the task using the additional generative neural network.

[0202]

[0190] Clause 45. The method of clause 44, wherein for each additional population of natural language solutions, iteratively evolving the additional population of additional natural language solutions using the additional generative neural network comprises, at each of a plurality of additional evolutionary iterations: obtaining data defining an additional fitness score distribution that defines a respective additional fitness score for each additional natural language solution in the additional population of additional natural language solutions, wherein an additional fitness score for a natural language solution characterizes how effectively the additional natural language solution solves the task; selecting one or more additional parent natural language solutions from the additional population of additional natural language solutions based on the additional fitness score distribution; processing an additional prompt that includes each of the additional parent natural language solutions using the additional generative neural network to generate an additional new natural language solution to the task; and adding the additional new natural language solution to the task to the additional population of additional natural language solutions.

[0203]

[0191] Clause 46. The method of any one of clauses 39-45, further comprising selecting the one or more additional natural language solutions for the task from the one or more additional populations of additional natural language solutions for the task.

[0204]

[0192] Clause 47. A system comprising: one or more computers; and Attorney Docket No. 45288-0552WO1 one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the respective method of any one of clauses 1-46.

[0205]

[0193] Clause 48. One or more non -transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the respective method of any one of clauses 1-46.

[0206]

[0194] Clause 49. A method performed by one or more computers, the method comprising: obtaining data defining a task; determining one or more natural language solutions to the task using a generative neural network, comprising iteratively evolving one or more populations of natural language solutions to the task using the generative neural network; determining that the one or more natural language solutions to the task does not satisfy the task; and in response to determining that the one or more natural language solutions to the task does not satisfy the task, determining one or more additional natural language solutions to the task using an additional generative neural network.

[0207]

[0195] Clause 50. The method of clause 49, wherein the additional generative neural network comprises a larger number of parameters than the generative neural network.

[0208]

[0196] Clause 51. The method of any one of clauses 49-50, wherein determining that the one or more natural language solutions to the task does not satisfy the task comprises determining that each of the one or more natural language solutions to the task does not satisfy a threshold fitness score within a threshold number of evolutionary iterations.

[0209]

[0197] Clause 52. The method of any one of clauses 49-51, wherein the threshold number of evolutionary iterations is predetermined.

[0210]

[0198] Clause 53. The method of any one of clauses 49-51, wherein the threshold number of evolutionary iterations is based on a predetermined amount of computational resources consumed.

[0211]

[0199] Clause 54. The method of any one of clauses 49-53, wherein determining one or more additional natural language solutions to the task using an additional generative neural network Attorney Docket No. 45288-0552WO1 comprises iteratively evolving one or more additional populations of additional natural language solutions to the task using the additional generative neural network.

[0212]

[0200] Clause 55. The method of clause 54, wherein for each additional population of natural language solutions, iteratively evolving the additional population of additional natural language solutions using the additional generative neural network comprises, at each of a plurality of additional evolutionary iterations: obtaining data defining an additional fitness score distribution that defines a respective additional fitness score for each additional natural language solution in the additional population of additional natural language solutions, wherein an additional fitness score for a natural language solution characterizes how effectively the additional natural language solution solves the task; selecting one or more additional parent natural language solutions from the additional population of additional natural language solutions based on the additional fitness score distribution; processing an additional prompt that includes each of the additional parent natural language solutions using the additional generative neural network to generate an additional new natural language solution to the task; and adding the additional new natural language solution to the task to the additional population of additional natural language solutions.

[0213]

[0201] Clause 56. The method of any one of clauses 49-55, further comprising selecting the one or more additional natural language solutions for the task from the one or more additional populations of additional natural language solutions for the task.

[0214]

[0202] Clause 57. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the respective method of any one of clauses 49-56.

[0215]

[0203] Clause 58. One or more non -transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the respective method of any one of clauses 49-56. Attorney Docket No. 45288-0552WO1

[0216]

[0204] What is claimed is:

Claims

Attorney Docket No. 45288-0552WO1CLAIMS1. A method performed by one or more computers, the method comprising: obtaining data defining a task; determining one or more natural language solutions to the task using a generative neural network, comprising: iteratively evolving one or more populations of natural language solutions to the task using the generative neural network; wherein for each population of natural language solutions, iteratively evolving the population of natural language solutions using the generative neural network comprises, at each of a plurality of evolutionary iterations: obtaining data defining a fitness score distribution that defines a respective fitness score for each natural language solution in the population of natural language solutions, wherein a fitness score for a natural language solution characterizes how effectively the natural language solution solves the task; selecting one or more parent natural language solutions from the population of natural language solutions based on the fitness score distribution; processing a prompt that includes each of the parent natural language solutions using the generative neural network to generate a new natural language solution to the task; and adding the new natural language solution to the task to the population of natural language solutions; and selecting one or more natural language solutions for the task from the one or more populations of natural language solutions for the task.

2. The method of claim 1, wherein for each natural language solution in the population of natural language solutions, the fitness score for the natural language solution is generated by processing the natural language solution using a scoring function.

3. The method of claim 2, wherein the scoring function comprises the generative neural network; andAttorney Docket No. 45288-0552WO1 wherein for each natural language solution in the population of natural language solutions, generating the fitness score for the natural language solution comprises: processing a prompt that comprises: (i) data characterizing the task, (ii) the natural language solution to the task, and (iii) an instruction to generate a fitness score for the natural language solution, using the generative neural network to generate the fitness score for the natural language solution.

4. The method of any preceding claim, wherein the task defines one or more constraints that are to be satisfied by natural language solutions to the task; and wherein for each natural language solution in the population of natural language solutions, the fitness score for the natural language solution characterizes whether the natural language solution satisfies the one or more constraints.

5. The method of any preceding claim, wherein for each natural language solution in the population of natural language solutions, the fitness score for the natural language solution further characterizes whether the natural language solution conforms with a required format.

6. The method of claim 5, wherein the required format is a JavaScript Object Notation (JSON) string format.

7. The method of any preceding claim, wherein the prompt processed by the generative neural network further comprises an instruction to generate a new natural language solution with a higher fitness score than the parent natural language solutions included in the prompt.

8. The method of any preceding claim, wherein the prompt processed by the generative neural network further comprises a respective fitness score for each parent natural language solution included in the prompt.

9. The method of any preceding claim, wherein the prompt processed by the generative neural network further comprises, for each parent natural language solution, a natural language evaluation of an effectiveness of the parent natural language solution in solving the task.Attorney Docket No. 45288-0552WO110. The method of claim 9, wherein the task defines one or more constraints that are to be satisfied by the natural language solution to the task; and for each parent natural language solution, the natural language evaluation of the effectiveness of the parent natural language solution in solving the task comprises, for any constraint that is not satisfied by the parent natural language solution, a natural language statement that the parent natural language solution does not satisfy the constraint.

11. The method of any one of claims 9-10, further comprising, for each parent natural language solution included in the prompt: processing a prompt that comprises: (i) the parent natural language solution, and (ii) an instruction to generate a critique of the parent natural language solution, using the generative neural network to generate a natural language output that defines the critique of the parent natural language solution; wherein for each parent natural language solution, the natural language evaluation of the effectiveness of the parent natural language solution in solving the task comprises the critique of the parent natural language solution that is generated by the generative neural network.

12. The method of any preceding claim, wherein processing the prompt that includes each of the parent natural language solutions using the generative neural network to generate the new natural language solution to the task comprises, at each of a plurality of prompting iterations: processing a prompt that includes: (i) each of the parent natural language solutions, and (ii) new natural language solutions generated by the generative neural network at any previous prompting iterations, using the generative neural network to generate a new natural language solution for the task at the prompting iteration.

13. The method of claim 12, wherein at each of the plurality of prompting iterations, the prompt processed by the generative neural network at the prompting iteration further includes a respective fitness score for each of the new natural language solutions generated by the generative neural network at any previous prompting iterations.Attorney Docket No. 45288-0552WO114. The method of any one of claims 12-13, wherein adding the new natural language solution to the task to the population of natural language solutions comprises: adding each new natural language solution generated at each of the plurality of prompting iterations to the population of natural language solutions.

15. The method of any preceding claim, wherein each population of natural language solutions is initialized, prior to being iteratively evolved, using a plurality of initial natural language solutions that are each generated by processing a prompt that defines the task and includes an instruction to generate a natural language solution to the task using the generative neural network.

16. The method of any preceding claim, wherein selecting one or more parent natural language solutions from the population of natural language solutions based on the fitness score distribution comprises: processing the fitness score distribution to generate a probability distribution over the population of natural language solutions; and stochastically sampling each parent natural language solution from the probability distribution over the population of natural language solutions.

17. The method of claim 16, wherein processing the fitness score distribution to generate the probability distribution over the population of natural language solutions comprises: processing the fitness score distribution using a soft-max function parameterized by a temperature parameter having a value strictly greater than one.

18. The method of any preceding claim, further comprising, for each population of natural language solutions, at one or more of the plurality of evolutionary iterations: selecting a plurality of best-performing natural language solutions from one or more other populations of natural language solutions based on respective fitness scores of the plurality of best-performing natural language solutions; andAttorney Docket No. 45288-0552WO1 adding the plurality of best-performing natural language solutions selected from the one or more other populations of natural language solutions to the population of natural language solutions.

19. The method of any preceding claim, further comprising, for one or more populations of natural language solutions: determining that the population of natural language solutions satisfies a criterion to be replaced based on a measure of central tendency of fitness scores of natural language solutions included in the population; and in response, replacing the population of natural language solutions with a new population of natural language solutions, comprising: selecting a plurality of best-performing natural language solutions from the populations of natural language solutions based on respective fitness scores of the plurality of best-performing natural language solutions; and initializing the new population of natural language solutions using the plurality of best-performing natural language solutions.

20. The method of claim 19, wherein initializing the new population of natural language solutions using the plurality of best-performing natural language solutions comprises: adding the plurality of best-performing natural language solutions to the new population of natural language solutions.

21. The method of claim 19 or claim 20, wherein initializing the new population of natural language solutions using the plurality of best-performing natural language solutions comprises: processing a prompt that comprises: (i) the plurality of best-performing natural language solutions, and (ii) an instruction to generate new natural language solutions to the task that are substantially different from the plurality of best-performing natural language solutions, using the generative neural network to generate new natural language solutions to the task; and adding the new natural language solutions to the new population of natural language solutions.Attorney Docket No. 45288-0552WO122. The method of any one of claims 19-21, further comprising iteratively evolving the new population of natural language solutions using the generative neural network.

23. The method of any preceding claim, wherein the generative neural network is an autoregressive neural network.

24. The method of any preceding claim, wherein the generative neural network has been trained to perform a language modeling task.

25. The method of any preceding claim, wherein the generative neural network has a Transformer architecture.

26. The method of any preceding claim, wherein iteratively evolving one or more populations of natural language solutions to the task using the generative neural network comprises: iteratively evolving a plurality of populations of natural language solutions substantially in parallel.

27. The method of any preceding claim, wherein the task requires scheduling jobs across a plurality of computational resources, each job having a respective resource requirement, and each computational resource having a respective total amount of resources; and wherein each natural language solution to the task defines a schedule for executing the jobs across the plurality of computational resources.

28. The method of claim 27, wherein the respective resource requirement for each job is represented across a plurality of resource dimensions, and wherein each computational resource is allocated a respective total resource amount along each resource dimension.

29. The method of claim 28, wherein the plurality of resource dimensions comprise a memory dimension and a compute dimension.Attorney Docket No. 45288-0552WO130. The method of claim 28 or 29, wherein the plurality of resource dimensions comprise one or more accelerator resource dimensions that correspond to resources provided by one or more hardware accelerators.

31. The method of any one of claims 27-30, wherein the computational resources are virtual machines that are each allocated a respective total amount of resources.

32. The method of any one of claims 27-31, further comprising executing the jobs across the plurality of computing resources in accordance with a schedule defined by a selected natural language solution for the task.

33. The method of any preceding claim, wherein the task requires generating a sequence of steps, expressed in natural language, for an agent to perform a project in an environment.

34. The method of claim 33, wherein the agent comprises a robot or an autonomous vehicle.

35. The method of claim 33 or claim 34, wherein the environment is a real-world environment.

36. The method of any one of claims 33-35, further comprising controlling the agent to perform actions in accordance with a selected natural language solution for the task.

37. The method of any one of claims 1-36 in which the data defining the task comprises one or more of image data, audio data, video data, sensor data output by a sensor and describing a real-world environment, genetic data, protein sequence data or data representing a chemical structure, the genetic data, protein sequence data or data representing a chemical structure being data obtained experimentally from real-world samples of genetic material, protein material or chemical material.

38. The method of any one of claims 1-37 in which there are plural populations of natural language solutions, and said iteratively evolving is performed in parallel for different ones ofAttorney Docket No. 45288-0552WO1 said populations of natural language solutions, by different corresponding ones of a plurality of computing units.

39. The method of any one of claims 1-38, further comprising: determining that the one or more natural language solutions to the task does not satisfy the task; and in response to determining that the one or more natural language solutions to the task does not satisfy the task, determining one or more additional natural language solutions to the task using an additional generative neural network.

40. The method of claim 39, wherein the additional generative neural network comprises a larger number of parameters than the generative neural network.

41. The method of any one of claims 39-40, wherein determining that the one or more natural language solutions to the task does not satisfy the task comprises determining that each of the one or more natural language solutions to the task does not satisfy a threshold fitness score within a threshold number of evolutionary iterations.

42. The method of any one of claims 39-41, wherein the threshold number of evolutionary iterations is predetermined.

43. The method of any one of claims 39-41, wherein the threshold number of evolutionary iterations is based on a predetermined amount of computational resources consumed.

44. The method of any one of claims 39-43, wherein determining one or more additional natural language solutions to the task using an additional generative neural network comprises iteratively evolving one or more additional populations of additional natural language solutions to the task using the additional generative neural network.

45. The method of claim 44, wherein for each additional population of natural language solutions, iteratively evolving the additional population of additional natural language solutionsAttorney Docket No. 45288-0552WO1 using the additional generative neural network comprises, at each of a plurality of additional evolutionary iterations: obtaining data defining an additional fitness score distribution that defines a respective additional fitness score for each additional natural language solution in the additional population of additional natural language solutions, wherein an additional fitness score for a natural language solution characterizes how effectively the additional natural language solution solves the task; selecting one or more additional parent natural language solutions from the additional population of additional natural language solutions based on the additional fitness score distribution; processing an additional prompt that includes each of the additional parent natural language solutions using the additional generative neural network to generate an additional new natural language solution to the task; and adding the additional new natural language solution to the task to the additional population of additional natural language solutions.

46. The method of any one of claims 39-45, further comprising selecting the one or more additional natural language solutions for the task from the one or more additional populations of additional natural language solutions for the task.

47. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the respective method of any one of claims 1-46.

48. One or more non -transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the respective method of any one of claims 1-46.Attorney Docket No. 45288-0552WO149. A method performed by one or more computers, the method comprising: obtaining data defining a task; determining one or more natural language solutions to the task using a generative neural network, comprising iteratively evolving one or more populations of natural language solutions to the task using the generative neural network; determining that the one or more natural language solutions to the task does not satisfy the task; and in response to determining that the one or more natural language solutions to the task does not satisfy the task, determining one or more additional natural language solutions to the task using an additional generative neural network.

50. The method of claim 49, wherein the additional generative neural network comprises a larger number of parameters than the generative neural network.

51. The method of any one of claims 49-50, wherein determining that the one or more natural language solutions to the task does not satisfy the task comprises determining that each of the one or more natural language solutions to the task does not satisfy a threshold fitness score within a threshold number of evolutionary iterations.

52. The method of any one of claims 49-51, wherein the threshold number of evolutionary iterations is predetermined.

53. The method of any one of claims 49-51, wherein the threshold number of evolutionary iterations is based on a predetermined amount of computational resources consumed.

54. The method of any one of claims 49-53, wherein determining one or more additional natural language solutions to the task using an additional generative neural network comprises iteratively evolving one or more additional populations of additional natural language solutions to the task using the additional generative neural network.

55. The method of claim 54, wherein for each additional population of natural language solutions, iteratively evolving the additional population of additional natural language solutionsAttorney Docket No. 45288-0552WO1 using the additional generative neural network comprises, at each of a plurality of additional evolutionary iterations: obtaining data defining an additional fitness score distribution that defines a respective additional fitness score for each additional natural language solution in the additional population of additional natural language solutions, wherein an additional fitness score for a natural language solution characterizes how effectively the additional natural language solution solves the task; selecting one or more additional parent natural language solutions from the additional population of additional natural language solutions based on the additional fitness score distribution; processing an additional prompt that includes each of the additional parent natural language solutions using the additional generative neural network to generate an additional new natural language solution to the task; and adding the additional new natural language solution to the task to the additional population of additional natural language solutions.

56. The method of any one of claims 49-55, further comprising selecting the one or more additional natural language solutions for the task from the one or more additional populations of additional natural language solutions for the task.

57. A system comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the respective method of any one of claims 49-56.

58. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations of the respective method of any one of claims 49-56.

Images