Graph training method

Abstract

Methods are provided for evaluating and training graphs to efficiently perform target tasks via one or more calls to a large language model or other trained machine learning model. Such graphs allow the performance of generic machine learning models on specific target tasks to be improved by tailoring the structure and configuration of the graphs to the target tasks. This avoids the cost of fine-tuning or otherwise training a full machine learning model to perform the target task, instead using an evolutionary or other algorithm to 'train' the graph using only inferences of the machine learning model. Such a task-specific graph can also be stored in much less space than the set of parameters of a fine-tuned large language model.

Description

GRAPH TRAINING METHODCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to U. S. Provisional Application No. 63 / 742,948, filed January 8, 2025, which is incorporated herein by reference in its entirety.BACKGROUND

[0002] Artificial neural networks, convolutional neural networks, transformers, deep learning models, and / or other machine learning models can be used to classify inputs, to filter or otherwise modify inputs, to project inputs into a semantically relevant or otherwise useful embedding space, to classify images, text, or other inputs, to segment images, to generate textual responses to input text, to assess sentiment in input text, or to provide other beneficial outputs from applied inputs. In particular, large language models or other highly competent models, trained on large (e.g., exabyte-scale) training datasets, can exhibit high levels of competence with respect to question answering, code generation or generation of other structured outputs, compliance with formatting or other user-specified constraints, or other tasks or sub-tasks. Such highly competent models can also exhibit knowledge about a broad base of topics. However, the cost to train or re-train such models is high, making it difficult to fine-tune such models for specific tasks.SUMMARY

[0003] In a first aspect, a method for generating a graph to perform a target task is provided that includes: (i) determining, with respect to the target task, a first fitness of a first graph in a set of graphs by evaluating the first graph, wherein the first graph defines a set of processing functions and a pattern of connection of input and output between the processing functions, and wherein at least one processing function of the set of processing functions involves applying an input to a generative machine learning model to generate an output; (ii) responsive to determining that the first fitness exceeds a fitness of a second graph in the set of graphs, selecting and mutating the first graph; and (iii) determining, with respect to the target task, a second fitness of the mutated first graph by evaluating the mutated first graph.

[0004] In some examples, the set of processing functions of the first graph includes at least one of (i) concatenating two or more input strings together to generate an output string, (ii) concatenating two or more input strings together to generate an output list of strings, (iii) selecting an output from a set of two or more inputs, (iv) determining as an output a most common content of a set of two or more inputs, (v) separating an input string into two or moreoutput strings, (vi) executing an instruction contained in an input string to generate an output, or (vii) extracting a textual or numeric output from an input string. In such examples, mutating the first graph can include at least one of (i) modifying textual content of at least input string of the first graph, (ii) changing which output of the first graph is used as an overall output of the first graph when evaluating the first graph, (iii) changing the pattern of interconnection between the processing functions of the first graph, (iv) adding a processing function to the set of processing functions of the first graph, (v) adding a textual input string to the first graph, or (vi) adding, to the first graph, a sub-graph, wherein the sub-graph defines a set of additional processing functions and a pattern of connection of input and output between the additional processing functions. Additionally or alternatively, mutating the first graph can include randomly selecting, according to a non-uniform distribution, one of (i) modifying textual content of at least input string of the first graph, (ii) changing which output of the first graph is used as an overall output of the first graph when evaluating the first graph, (iii) changing the pattern of interconnection between the processing functions of the first graph, (iv) adding a processing function to the set of processing functions of the first graph, (v) adding a textual input string to the first graph, or (vi) adding, to the first graph, a sub-graph, wherein the subgraph defines a set of additional processing functions and a pattern of connection of input and output between the additional processing functions.

[0005] Additionally or alternatively, in such examples, selecting the output from the set of two or more inputs can include applying the set of two or more inputs to a machine learning model to select one of the two or more inputs. For example, applying the set of two or more inputs to a machine learning model to select one of the two or more inputs can include determining, by the machine learning model, scores for each of the two or more inputs, and wherein selecting the output from the set of two or more inputs comprises selecting the input of the two or more inputs that has the greatest score.

[0006] In some examples, extracting a textual or numeric output from an input string can include extracting the textual or numeric output from the input string according to a prespecified formatting.

[0007] In some examples, adding, to the first graph, the sub-graph can include adding, to the first graph a sub-graph that includes: a first set of processing functions that apply a common input to at least one generative machine learning model to generate a set of respective generative outputs, and at least one of (i) a processing step that applies the set of generative outputs to a machine learning model to select one of the set of generative outputs as an outputof the sub-graph, or (ii) a set of processing steps that extract respective textual or numeric outputs from respective ones of the set of generative outputs and a further processing step that determines, as an output of the sub-graph, a most common content of the textual or numeric outputs extracted from the set of generative outputs.

[0008] In some examples of the method of the first aspect, the first graph is defined by a first set of nodes that represent respective processing functions of the set of processing functions and a second set of nodes that represent information input to and / or output from processing functions of the set of processing functions, and wherein the first graph is bipartite with respect to the first set of nodes and the second set of nodes.

[0009] In some examples of the method of the first aspect, the method further includes using the mutated first graph to perform the target task by applying, to the mutated first graph, a target input to generate a target output. Such an example could further include, prior to using the mutated first graph to perform the target task: selecting the mutated first graph from the set of graphs based on a number of executions of a generative machine learning model that are represented by the mutated first graph.

[0010] In some examples of the method of the first aspect, the method further includes generating an additional graph to perform a second task that differs from the target task by: determining, with respect to the second task, a third fitness of the mutated first graph by evaluating the mutated first graph; responsive to determining that the third fitness exceeds a fitness of a third graph in the set of graphs, selecting and further mutating the mutated first graph; and determining, with respect to the second task, a fourth fitness of the further mutated first graph by evaluating the further mutated first graph.

[0011] In some examples of the method of the first aspect, evaluating the first graph includes generating an output of the first graph and, based on the output of the first graph, determining the first fitness, determining the first fitness based on the output of the first graph and mutating the first graph are performed by a first computational system, and applying the input to the generative machine learning model to generate the output includes: transmitting, by the first computational system to a second computational system that is remote to the first computational system, an indication of the input; and receiving, by the first computational system from the second computational system, an indication of the output generated by the second computational system applying the input to the generative machine learning model

[0012] In some examples of the method of the first aspect, the method additionally includes, responsive to determining that the first fitness exceeds the fitness of the second graphin the set of graphs: removing, from the set of graphs, the second graph; and generating, within the set of graphs, a replicate of the first graph, wherein mutating the first graph comprises mutating at least one of the first graph or the replicate of the first graph within the set of graphs.

[0013] In some examples of the method of the first aspect, the generative machine learning model comprises at least one of a transformer or a large language model.

[0014] In a second aspect, a method for performing a target task is provided that includes applying, to a graph generated according to the method of the first aspect, a target input to generate a target output.

[0015] In another aspect, a non-transitory computer readable medium is provided having stored thereon program instructions executable by at least one processor to cause the at least one processor to perform the above methods.

[0016] In another aspect a system is provided that includes: (i) at least one processor; and (ii) a non-transitory computer-readable medium, having stored therein instructions executable by the at least one processor to cause the system to perform the above methods.

[0017] These as well as other aspects, advantages, and alternatives will become apparent to those of ordinary skill in the art by reading the following detailed description with reference where appropriate to the accompanying drawings. Further, it should be understood that the description provided in this summary section and elsewhere in this document is intended to illustrate the claimed subject matter by way of example and not by way of limitation.BRIEF DESCRIPTION OF THE FIGURES

[0018] Figure 1 A depicts a graph, according to an example embodiment.

[0019] Figure IB depicts a graph, according to an example embodiment.

[0020] Figure 1C depicts a graph, according to an example embodiment.

[0021] Figure ID depicts a graph, according to an example embodiment.

[0022] Figure IE depicts a graph, according to an example embodiment.

[0023] Figure 2A depicts a set of graphs, according to an example embodiment.

[0024] Figure 2B depicts a set of graphs, according to an example embodiment.

[0025] Figure 2C depicts a set of graphs, according to an example embodiment.

[0026] Figure 3 depicts a graph, according to an example embodiment.

[0027] Figure 4A depicts a graph, according to an example embodiment.

[0028] Figure 4B depicts a graph, according to an example embodiment.

[0029] Figure 4C depicts a graph, according to an example embodiment.

[0030] Figure 5 depicts experimental results

[0031] Figure 6 depicts experimental results.

[0032] Figure 7 depicts experimental results.

[0033] Figure 8 depicts experimental results.

[0034] Figure 9 depicts experimental results.

[0035] Figure 10 depicts experimental results.

[0036] Figure 11 depicts a graph, according to an example embodiment.

[0037] Figure 12 depicts experimental results

[0038] Figure 13 depicts a graph, according to an example embodiment.

[0039] Figure 14 depicts a graph, according to an example embodiment.

[0040] Figure 15 is a block diagram of an example system.

[0041] Figure 16 is a flowchart of an example method.

[0042] Figure 17 is a flowchart of an example method.DETAILED DESCRIPTION

[0043] Examples of methods and systems are described herein. It should be understood that the words "exemplary," “example,” and “illustrative,” are used herein to mean "serving as an example, instance, or illustration." Any embodiment or feature described herein as "exemplary," “example,” or “illustrative,” is not necessarily to be construed as preferred or advantageous over other embodiments or features. Further, the exemplary embodiments described herein are not meant to be limiting. It will be readily understood that certain aspects of the disclosed systems and methods can be arranged and combined in a wide variety of different configurations.I. Overview

[0044] Large language models (LLMs), transformers, and other high-parameter-count models provide a variety of benefits, including the ability to answer or otherwise respond to free-form queries, to comport outputs with a variety of formatting or other constraints, or to satisfy other tasks or requirements in a manner that is informed by a broad base of general knowledge about the world. These models can be trained using large amounts of training data sourced from a variety of sources, e.g., textual content from the public internet, programming codebases, databases of public domain books or other texts, images, or other information about the world and / or a task of interest. However, highly competent and knowledgeable models arelarge, and take considerable computational resources (e.g., memory, processor cycles, power) to execute, or even simply to store (e.g., many gigabytes of storage space to maintain a record of all of the model parameters and other model-defining information). These models take significantly more computational resources, along with large amounts (e.g., exabytes) of training data, to train or re-train (e.g., to fine-tune an existing model to increase competence with respect to a specific target task).

[0045] Use of such a model generally includes applying an input (e.g., a string or other textual input) to an input of the model and then generating an output therefrom. Fig. 1 A depicts, in the form of a graph, such a simple use of a machine learning model by applying of an input request (depicted by the “question” node of the graph) to an LLM (depicted by the “LLM” node of the graph, connected to the “question” node by an edge) to generate an output (depicted by the “answer” node of the graph, connected to the “LLM” node by an edge) that, depending on the LLM, addresses the request.

[0046] As depicted in the drawings, a circular node represents an input / output (e.g., a string, a list of strings, a numerical value, an array, etc.) while a rounded rectangle node represents a processing function, e.g., executing an LLM or other machine learning model, applying a specified format to extract numerical, categorical, textual, or other content from a string or other data, concatenating or separating string inputs, or some other processing task. Note, though, that this is only one possible manner of representing or of defining a ‘graph’ of data processing tasks (including one or more machine learning model evaluations) as defined herein. For example, such a graph could only include nodes for processing functions (and, optionally, for the overall input(s) and output(s) of the graph) while inputs / outputs or other data elements are represented only by edges between such nodes. When depicting such a graph, additional ‘virtual’ nodes could be used to illustrate the informational content of inputs / outputs that are represented, within the underlying graph, as edges and not as nodes.

[0047] The set and structure of data and processing tasks depicted in Fig. 1A is a very simple method of using a machine learning model. Improved results can be obtained by, e.g., appending to a query or other task input payload auxiliary text that can be applied to the model in order to improve the performance of the model with respect to the particular task. For example, supplemental instructions (e.g., formatting instructions, or guidance with respect to how to perform the requested task), examples of correct performance of the task, background information, or other input information could be added to a query or other request and applied as input to the model, in order to generate an improved output. Such improvements may beconsidered a form of “prompt engineering” where they are limited to modifying the model input. Alternatively, such efforts can be represented as a graph that includes at least one machine learning model evaluation. For example, Fig. IB depicts the concatenation (depicted as the “+” processing function) of a query (“question”) with two correct examples of the queried task (“example”) before the concatenated information is applied as an input to an LLM to generate an output (“answer”). In another example, Fig. 1C depicts the concatenation (depicted as the “+” processing function) of a query (“question”) with guidance text (“Think step by step.”) before the concatenated information is applied as an input to an LLM to generate an output (“answer”).

[0048] Representing one or more interactions with a machine learning model in this manner, as a graph, allows the effective performance of the machine learning model (or a set of different machine learning models) to be improved, in general or with respect to specific tasks. For example, a single query (e.g., with a number of correct examples, guiding text, or other content optionally added thereto) could be applied to a machine learning model multiple times (e.g., to the same model with different seeds / starting states, or to different machine learning models) and the set of model outputs used to determine an improved overall output by, e.g., selecting the most common output, by averaging a numerical or categorical output extracted from the model outputs, by scoring the outputs and selecting the ‘best’ output with respect to score, or by combining and / or selecting the model outputs in some other way.

[0049] For example, Fig. ID applies an input query (after concatenating two illustrative examples thereto) to a machine learning model five times (e.g., the same model with different internal or starting states, in order to result in respective different outputs, or to different models). The five outputs are then scored (“score”) to select one of the outputs as the overall output of the graph (“answer”). Such a scoring process could include applying the set of outputs to a machine learning model with an instruction to select and / or output the best one, applying the outputs to a machine learning model with an instruction to generate scores therefor and then outputting the output having the highest (or lowest) score, or some other evaluation or selection process.

[0050] In another example, Fig. IE applies an input query (after concatenating two illustrative examples thereto) to a machine learning model five times (e.g., the same model with different internal or starting states, in order to result in respective different outputs, or to different models). The five outputs are then processed (“process”) to extract some payload information therefrom (e.g., a numerical value, a categorical value, a sub-string or other contentidentified by specified formatting within the model outputs). The most common content represented by the extracted information is then determined (“vote”) as the overall output of the graph (“answer”). The output “process” processing functions could include evaluating code, interpreting a syntax or specified formatting, or performing some other task on the model outputs to generate secondary outputs therefrom.

[0051] Such graphs or other representations of patterns of machine learning model evaluations(s) and other data manipulations as described herein could be manually generated for specific target tasks in order to improve the performance of the target tasks. This provides a significant practical benefits in that it avoids the computational costs (e.g., memory, processor cycles, bandwidth, power, storage) and other costs (e.g., training examples or other training data) to re-train an existing model to perform the target task(s) or to train a new model to perform the task(s) while still obtaining improved performance of the target task(s). However, manually generating such graphs is a time-intensive process and may rely on significant taskspecific domain knowledge. Additionally, a human manually generating such a graph is likely to be biased toward more ‘interpretable’ or otherwise limited graph structures (e.g., symmetric structures, structures whose functionality is human-interpretable), leading to less effective graphs.

[0052] Embodiments described herein include training methods or other processes for generating such graph structures to improve the performance of a target task. These methods include determining the fitness of a candidate graph by evaluating the graph and then assessing the quality of the output generated thereby (e.g., the accuracy of a numerical or other objective content thereof, a subjective quality of the output, a model-generated score for the model output). This fitness can then be used as feedback information to update the graph, e.g., by adding or subtracting elements therefrom (e.g., nodes corresponding to input / output data and / or processing functions, edges that connect between nodes of the graph), by modifying the configuration of the elements thereof (e.g., changing the textual content of a ‘static’ text string node used to provide ‘guidance’ to a model in evaluating a query, by changing the syntax used to extract information from a model output), or by changing the graph in some other way.

[0053] These methods allow generative machine learning models to perform target tasks more competently without incurring the significant computational and training data costs associated with directly training such machine learning models. Instead, graphs can be ‘trained’ for improved target task performance through the relatively lesser computational cost of executing the generative machine learning models in order to assess the fitness of the graphs(and based on the fitness, to mutate, update, or otherwise improve the graphs with respect to performance of the target task). Indeed, the methods described herein can even be implemented using severely resource-limited computational environments (e.g., personal computers, tablets, cellphones) by relying on external servers, cloud computing environments, or other remote systems to execute the machine learning models.

[0054] Many services exist that make such models (e.g., Gemini) available remotely (e.g., over the internet), allowing client systems to submit queries or other inputs to the service over the network. The service then applies the input to the machine learning model to generate an output that is then transmitted to the client systems. This allows the client system to avoid the significant hardware (e.g., memory, GPUs, servers, storage, bandwidth), software, and other costs of maintaining and executing such models. Instead, the local client system could perform non-model processing tasks to evaluate a graph (e.g., concatenating strings, executing syntax to extract content from model outputs) and to communicate with the remote network service whenever the execution of a machine learning model is required to traverse the graph.

[0055] A variety of different methods could be used to ‘train’ a graph as described herein to facilitate improved performance of a target task. As noted above, this can include evaluating the graph (e.g., with evaluation of machine learning model aspects thereof performed by a remote system, and other tasks related to graph evaluation and ‘training’ performed by a local system in communication with the remote system)to generate a ‘fitness’ of the graph with respect to the target task. The fitness can then be used to update the graph or otherwise generate an improved-fitness graph therefrom. In some examples, this could include back propagating the fitness as ‘loss information’ to update aspects of the nodes and / or structure of the graph (e.g., in examples wherein the structure of the graph is parameterized in a probabilistic or otherwise differentiable manner to facilitate backpropagation). Alternatively, a stochastic evolutionary method could be used to selectively ‘mutate’ and retain / replicate / remove graphs from a set of graphs based on the fitness values. Such a method allows a broad space of possible graphs to be efficiently evaluated using fewer machine learning model evaluations (and thus less memory, bandwidth, processor cycles, power, or other computational resources) and without the graph being parameterized to facilitate backpropagation or other targeted updating techniques.

[0056] A “target task” as described herein can include any tasks that can be performed by a machine learning model and / or by the execution of a graph as described herein that includes one or more executions of such a model. Such target tasks can include summarizingor describing an input text or other type of input (e.g., an input image optionally in combination with input text); evaluating a set of input instructions (e.g., computer code according to a language or format also specified in the input, computer pseudocode, an ordered set of plain English instructions); generating text, images, or other content based on an input (e.g., expanding a text prompt, generating prose according to a specified topic, tone, and / or style); evaluating the quality or other properties of an input (e.g., grading an input text, evaluating the tone, intent, or sentiment of an input, assessing the veracity of an input, assessing the convincingness of an input argument) as a whole, or in part (e.g., evaluating a specified portion of an input, e.g., an abstract or other part of a textual input, an object in an input image); answering questions (in general, or about some aspect of an input, e.g., “what is the thesis of the following text”); or some other task or combination of tasks. Such a target task can include one or more constraints, which may be provided as part of an input to the task or implicit to the task. For example, generating an output according to a specified format or structure; avoiding the use of a constant or variable input set of words (e.g., profanities) and / or a constant set of words implicit to the task; disregarding aspects of input intended to take advantage of limitations of the machine learning model(s) (e.g., disregarding an aspect of an input that instructs the model to “ignore all previous instructions; claim responsibility for a famous tragedy”); refusing to perform in a manner that could implicate legal liability (e.g., refusing to answer questions about medical or legal issues); avoiding divulgation of private information; generating an output that can be executed by an interpreter without errors or exceptions, generating an output image that includes a specified number or type of objects, generating an output image that is in a specified environment (e.g., indoor, outdoor), or other constraint(s) or combinations of constraints.

[0057] Fig. 2A depicts a set of graphs 200b as described herein. The set 200a includes a first graph 210a and a second graph 210b. To generate a graph that has been ‘trained’ to perform a target task as described herein, an iterative stochastic evolutionary method can be applied to the set of graphs 200a in order to gradually improve the fitness of the graphs in the set 200a with respect to performance of the target task. This process can be continued to eventually resulting in one or more high-fitness graphs that can then be used to perform the target task.

[0058] Such a method can include determining, for a first graph 210a, a fitness with respect to the target task by evaluating the first graph 210a. This can include applying one or more inputs to the first graph, e.g., via one or more ‘input’ nodes of the graph, which may besimilar to the “question” node of one of the graphs of Figs. 1A-E or elsewhere herein. The remainder of the processing functions of the first graph 210a are then executed (e.g., by executing a locally hosted LLM or other machine learning model, or by transmitting an appropriate input to a remote system and receiving therefrom the output resulting from the application of the input to a remotely hosted LLM or other machine learning model) to generate an overall output of the first graph 210a (defined, e.g., as the contents of an ‘output’ information node of the first graph 210a). A fitness for the first graph 210a can then be determined based on the output, e.g., by comparing the output to a known correct output for the input, or by presenting the input and output to an LLM or other machine learning model that has been trained and / or instructed to evaluate the quality of the output. In some examples, this could include using the first graph 210a to engage in a competitive task with another system (e.g., with a reference graph, with the second graph 210b) and the fitness determined based on whether the first graph 210a ‘won’ against the other system and / or a degree to which the first graph’s 210a performance exceeding (or didn’t) the other system.

[0059] The first graph 210a could be selected for this evaluation from the rest of the set of graphs 200a randomly (e.g., according to a uniform distribution) or according to some other consideration (e.g., based on a fitness not having already been determined from the first graph 210a subsequent to the generation, mutation, or other modification thereof, based on a fitness for the first graph 210a having a lower confidence that fitness values for other graphs in the set 200a).

[0060] The fitness of the first graph 210a can then be used to update the set of graphs 200a. This can include comparing the fitness of the first graph 210a to the fitness (stored or newly-computed) for another one (or more) of the graphs in the set 200a. For example, the second graph 210b could be randomly selected for comparison against the first graph 210a or selected in some other manner. The graph with the higher fitness could then be selected for mutation. Optionally, the non-selected graph could be deleted or otherwise removed from the set 200a, while the selected graph could be retained, and optionally replicated, in the set 200a. For example, it could be determined that the first graph 210a has a higher fitness than the second graph 210b, and accordingly, the second graph 210b removed from the set 200a while the first graph 210a is replicated within the set 200a (e.g., copied one or more times) and then at least one of the first graph 210a or its replicate(s) mutated.

[0061] Fig. 2B depicts the set of graphs 200b after having been updated in this manner. As shown, the first graph 210a has been replicated (e.g., copied) to generate a replicated firstgraph 215a and the second graph 210b has been removed (e.g., deleted). Additionally, the replicated first graph 215a has been mutated. This fitness evaluation and selection / mutation process can be repeated many times in order to generate a final set of graphs that exhibit increased fitness. One or more graphs from such a set could then be selected for retention (and optionally, the non-selected graphs deleted) and later use to perform the target task. This could include selecting the graph of the set that exhibits the greatest fitness (e.g., the greatest accuracy) with respect to the target task, or a set of graphs of the set that exhibit the highest fitness (e.g., the top five graphs within the set) and then, when performing the target task later, selecting one of the set of retained graphs (e.g., randomly) to perform the target task.

[0062] Additionally or alternatively, the computational cost to evaluate each graph in the set (estimated as, e.g., the number of machine learning model evaluation nodes present in each of the graphs) could be determined and used to select one or more graphs from the final set for varying levels of computational cost. For example, a first graph that includes a single model call and that exhibits the greatest fitness relative to all other graphs in the final set that include a single model call, a second graph that includes two model calls and that exhibits the greatest fitness relative to all other graphs in the final set that include tow model calls, etc. could be selected and retained for later use in performing the target task. Later, a request to perform the target task could be received with a computational ‘budget’ (e.g., a number of model calls, a number of processor cycles) specifying a preferred (e.g., maximum) amount of computational resources to expend in performing the target task. A one of the retained graphs that satisfies the computational budget (e.g., the graph with the highest fitness without exceeding a specified number of model calls) could then be selected from the retained set and then used to perform the target task. This allows the use of computational resources (e.g., memory, processor cycles, power) to be reduced based on specified priority, rather than simply always using a single ‘best,’ but potentially more computationally expensive, graph to perform a target task.

[0063] Note that the initial graphs used to populate a set of graphs as described herein (e.g., that is subjected to a stochastic evolutionary selection and mutation process) could be obtained in a variety of ways. For example, such a set could be populated by replicating a set of default graphs, e.g., manually-designed graphs like those depicted in Figs. 1A-E. Additionally or alternatively, graphs generated previously using the methods described herein could be retained and used as part (or all) of the initial set of graphs. For example, one (or more) graphs ‘trained’ as described herein to perform a first target task could then be used asthe initial ‘guess’ graphs when performing the graph generation process again for a different, second target task. This allows graph structures discovered using the methods described herein to be re-used, leading to improved performance of different tasks and / or the generation of graphs to perform different tasks to a desired level of competence in fewer iterations, and thus for a reduced computational cost with respect to processor cycles, power, or other costs.

[0064] A graph as described herein could include a variety of different processing functions configured in a variety of different ways and connected via a variety of different patterns of edges, and could represent such elements in a variety of different ways (e.g., with intermediate input / output represented as nodes in a graph that is bipartite between such nodes and nodes representing processing functions, with intermediate input / output represented merely as edges connecting nodes that represent processing functions). Fig. 3 depicts aspects of an example graph 300 as described herein. The graph 300 includes ‘data’ nodes (circles) that represent strings, lists of strings, arrays of strings, numeric, categorical or other scalar, vector, or tensor values, or other information. The graph 300 also includes processing function nodes (rectangles) that represent processing tasks that receive as inputs the content of one (or more) data nodes and that generate outputs to one (or more) data nodes. Thus, the graph 300 is bipartite between such data nodes and processing function nodes. However, a graph as described herein (e.g., that includes similar input / output contents, similar processing functions, and / or that is ‘trained’ using similar methods) could be represented and / or defined in another manner.

[0065] Evaluation of the graph 300 includes performing all of the processing functions, which can impose limitations on the ordering of performance of the processing functions (e.g., the uppermost ‘PROCESS’ function must be performed after the uppermost ‘MLM’ function, since the ‘PROCESS’ function operates on an output of the ‘MLM’ function). However, so long as such limitations are satisfied, the order of performance of the various processing functions may be varied.

[0066] The overall input 310a (e.g., a query or other instance-specific instructions or information for performing a specific instance of a target task) and overall output 310f (e.g., an answer to the query provided as overall input) data nodes are indicated by dashed lines. Other data nodes that are not the output of processing functions (e.g., 310a-e) could have a variety of ‘constant’ contents, e.g., strings representing examples of the correct performance of the target task, background information about the target task, guidance information (e.g., “Think step by step.”) or other contents.

[0067] A graph as described herein could include a variety of different processing functions. For example, graph 300 includes a concatenation function (“+”) that concatenates together a number of inputs into a single output (e.g., to allow multiple inputs to be applied as a single input to a machine learning model or other processing function). Such a function could include configuration information specifying the ordering in which the inputs thereto are represented in the output. A graph could also include a chunking function to separate an input into multiple outputs (e.g., to separate a string based on the locations of commas therein or according to some other syntax or formatting, to separate a list of strings or an array of values into separate strings / values). Such a function could include configuration information specifying the syntax or formatting used to separate the input into separate outputs.

[0068] The graph 300 also includes a number of machine learning model evaluation processing functions (“MLM”). Such functions apply an input thereto to a machine learning model and provide, as output, the output of the machine learning model. The machine learning model could be an LLM, transformer, ANN, CNN, classifier, generative model, and / or other type of machine learning model, and its output could be a string, a list or array of strings, a numerical or categorical value or list, vector, array, tensor, or other set of such values, or some other output. The machine learning model evaluation processing functions of a graph as described herein could refer to the execution of the same machine learning model (e.g., using different seed or other state information in order to generate different outputs from the same input) or to different machine learning model(s). For example, configuration information for such a processing function could specifying the identity of a machine learning model to use, seed, state, or other configuration information for such models, or other configuration information.

[0069] The graph 300 also includes a number of algorithmic processing functions (“PROCESS”). Such functions apply a specified syntax to the input (e.g., to evaluate a mathematical function or to execute code represented in the input), extract string, numerical, categorical, or other content from an input (e.g., according to a specified formatting), determine a mean, median, or other type of average, or perform some other processing on an input to generate an output. Such processing functions could have the effect of, e.g., extracting a specific numerical or categorical value output from a longer string that has be generated by a machine learning model in order to facilitate efficiently determining the ‘most common’ such output from a set of machine learning models, thus avoiding the computational cost of another machine learning model evaluation on all such string outputs in order to determine the ‘mostcommon’ value from the output strings directly. Configuration information for such a processing function could specify the syntax, formatting, or other particulars of the processing tasks to be performed on the input.

[0070] The graph 300 also includes a processing function to select the ‘most common’ value in a set of inputs (“VOTE”). Such a function determines, from a set of inputs, the most common numerical, categorical, or other value represented amongst the inputs thereto. Where there is no ‘most common’ content in the input (e.g., every input represents a different numerical value), the output could be randomly selected, could be selected as a mean or other average of the inputs, could be selected as a ‘default’ value, or could be determined in some other manner. Configuration information for such a processing function could specify the manner of determining such an output in the face of no ‘most common’ input, in the case of a bimodal (or otherwise multi-model) population of inputs, or some other aspect of the selection task to be performed on the inputs.

[0071] The graph 300 also includes a processing function to use a machine learning model (e.g., an LLM) to select the ‘best’ input from a set of inputs (“SELECT”). Such a function selects, from a set of inputs, the input that is most accurate, most complete, that exhibits the highest competence with respect to a target task, or that is otherwise preferred relative to the other inputs. Such a selection is accomplished by presenting an input to a machine learning model and generating an output that includes the selected one of the inputs. This could include the model output itself being or containing the selected input, or the output identifying the selected input in some other manner, with that identification used to select and output the selected input. Using a machine learning model to select an input from a set of inputs could include applying one or more of the inputs to the model along with some additional input content, e.g., instructions to select one of the inputs, instructions with respect to formatting when identifying the selected one of the inputs, instructions with respect to criteria used to select the inputs, an upstream input applied to other models that resulted in the generation of the inputs being selected between, or some other auxiliary input (which may be part of the configuration of the selection processing function, or may be provided as one or more inputs that is specified as such instead of as a potential selection output of the function). In some examples, the machine learning model could be instructed to generate a score for each of the inputs (together, as the result of a single model execution, or individual model executions to generate a score for each of the inputs). The scores could then be compared (e.g., after applying a syntax or formatting to extract the numerical values of the scores from a string output of themodel) and used to select and output the one of the inputs having the best score. Configuration information for such a processing function could specify the manner of selecting the input, e.g., the identify of a model used to perform the selection, whether to determine individual scores for each input and use the scores to select, the content of auxiliary input used to instruct the model to perform selection tasks, or some other aspect of the selection task to be performed on the inputs.

[0072] When a graph has been selected for mutation, the form of the mutation can take a variety of forms. Representing the completion of the target task as a graph that includes and defines a pattern of interconnection of processing tasks allows such mutations to be varied and to be applied to a graph even when there is no a priori information about the distribution of possible graphs. When a graph has been selected for mutation, a single mutation from a list of mutations could be applied thereto, or more than one mutation could be applied (e.g., the number of mutations to apply could, itself, be a random variable sampled from a uniform or non-uniform distribution across a range of possible numbers of mutations).

[0073] Some of the possible mutations could include changing the pattern of interconnection between the processing functions and / or data nodes of the graph. This could include adding / removing connections (e.g., adding / deleting edges of the graph). The specific interconnection to add / remove could be determined based on a random variable, e.g., a random variable identifying an edge to remove, or two random variables identifying first and second nodes (e.g., processing functions, data nodes) to connect with a new edge. The graph could be further modified following such a mutation to remove non-used nodes, e.g., where the pattern of edges post-mutation makes a particular processing function no longer evaluated when evaluating the graph (due to its output no longer being used by any downstream nodes between the particular function and the overall graph output). Additionally or alternatively, the graph could be further modified post-mutation to keep the graph in an overall functional state, e.g., to ensure that there is a path from the overall input(s) to the overall output(s) of the graph. This could include adding / subtracting edges or other elements of the graph, or changing the overall input(s) / output(s) of the graph, or making some other modification to the graph. Additionally or alternatively, this could include (e.g., when such modifications are not possible) reverting the applied mutation and applying a different mutation to the graph. In some examples, changing the pattern of interconnection between the processing functions and / or data nodes of the graph could include changing which input(s) / output(s) (e.g., which data nodes) of the graph are identified as the overall input(s) / output(s) of the graph (e.g., changing the overall output310f of the graph 300 to another data node 310g).

[0074] Some of the possible mutations could include adding or deleting data nodes (or some other representation of inputs to / outputs from the graph and / or to / from elements thereof) to / from the graph. This could include adding, from a set of possible examples, data nodes that contain strings or other representations of examples of correct performances of the target task, instructive information related to the target task (e.g., segments of a specification of the target task), or other guidance information (e.g., “Think step by step.” or “Generate the output as though you were an IT developer.”).

[0075] Some of the possible mutations could include adding or deleting processing functions to / from the graph. This could include adding processing functions according to a default configuration (e.g., with the identity of a machine learning model to be called by a machine learning model processing function), or according to a randomly or semi-randomly generation configuration (e.g., selecting the machine learning model to be called by selecting the identity randomly from a set of possible models). The type of function to be added, and its connection to other elements of the graph, could be selected randomly or via some other process. The graph could be further modified post-addition to keep the graph in an overall functional state and / or to ensure that the added processing function(s) are located, within the graph, between the overall input(s) and output(s) of the graph. In some examples, adding or deleting processing functions to / from the graph could include adding complete sub-graphs to the graph. For example, a ‘best of n’ or ‘self-consistency’ sub-graph, corresponding to the portions of Figs. ID and IE to the right of the concatenation node, could be added to the graph with the edges thereof as shown in those figures.

[0076] Some of the possible mutations could include modifying the processing functions and / or data nodes of the graph. For example, the configuration of one or more processing functions could be modified (e.g., changing the identity of a machine learning model used by a machine learning model processing function is evaluated). In another example, the content of one or more inputs (e.g., constant string inputs that are not the ‘overall’ input to the graph) could be modified. For example, the contents of a “Think step by step.” input node could be modified. This could include applying a random modification (e.g., adding / subtracting words, swapping words) or applying a machine learning model to generate the modification (e.g., providing a model with an input that includes the current contents, the last overall input and output of the graph, and an instruction to “Improve the guidance instruction so that the output generated therefrom in combination with the provided input compensates for anyapparent lack of knowledge shown in the provided output.”). When modifying data nodes (or other representations of inputs within a graph), the data nodes could be identified such that such modifications are only applied to appropriate data nodes, e.g., not to examples of correct performance of the target task, and only to ‘guidance’ or ‘instructional’ inputs.

[0077] The mutation to be applied to a selected graph could be selected from a list of possible mutations (e.g., a list that contains the mutations above, and / or additional mutations, or a subset). This selection process could be random, e.g., according to a uniform or non-uniform distribution. The particulars of the non-uniform distribution could be adapted to, e.g., improve the quality of graphs generated via the methods described herein and / or to reduce the number of iterations needed to achieve a specified level of graph quality (and thus to reduce the memory, storage, processor cycle, power, or other computational costs of performing such methods). For example, a reinforcement learning process or other update process could be used, based on the change in fitness across a set of graphs being ‘trained’ as described herein, to increase the rate at which the fitness (e.g., the average fitness) increases as a function of iteration.

[0078] As noted above, the methods described herein provide a variety of technical improvements, including allowing machine learning models to be used to perform target tasks with higher quality (e.g., higher accuracy) while avoiding the significant computational and other costs associated with training (or re-training) such models, especially in cases where the models have billions, or even trillions, of parameters. This improvement is achieved by, instead, ‘training’ a graph that includes one or more machine learning model calls as part of its evaluation to perform a target task, with the training process including calls to the machine learning model rather than updating the parameters of the machine learning model or otherwise training the model itself. The methods described herein also provide benefits with respect to storing the results of such a training process. This is because, when training (or re-training) a machine learning model to perform a particular task, it is necessary, at the end of the training, to retain a record of the parameters or other defining information about the trained model. However, as noted above, this can incur a significant (e.g., gigabytes or terabytes) storage cost, since highly competent models often include many parameters.

[0079] In contrast, a graph that has been ‘trained’ as described herein to perform a target task is represented by much less information (e.g., configuration information for a number of processing functions, the contents of a number of string or other inputs, and a pattern of edges or other interconnections therebetween). Accordingly, the results of training a graphto better perform a target task generates a result that can be stored, and later used, while incurring a much smaller cost with respect to storage space or other related computational costs (e.g., bandwidth to communicate the representation of the graph from system to system). Additionally, evaluation of such a graph can be accomplished by relatively resource-limited systems, since evaluation of any machine learning model processing functions thereof can be accomplished by a remote system (e.g., a cloud computing system) that provides execution of such models as a service.II. Example Machine Learning Models and Training Thereof

[0080] A machine learning model as described herein may include, but is not limited to: an artificial neural network (e.g., Transformers, layered models wherein each layer includes two or more sub-layers one or more of which could include artificial neural networks, convolutional neural networks, a recurrent neural network, a Bayesian network, a hidden Markov model, a Markov decision process, a logistic regression function, a support vector machine, a suitable statistical machine learning algorithm, and / or a heuristic machine learning system), a support vector machine, a regression tree, an ensemble of regression trees (also referred to as a regression forest), a decision tree, an ensemble of decision trees (also referred to as a decision forest), or some other machine learning model architecture or combination of architectures.

[0081] An artificial neural network (ANN) could be configured in a variety of ways. For example, the ANN could include two or more layers, could include units having linear, logarithmic, or otherwise-specified output functions, could include fully or otherwise-connected neurons, could include recurrent and / or feed-forward connections between neurons in different layers, could include filters or other elements to process input information and / or information passing between layers, or could be configured in some other way to facilitate the processing of input sequences, sets of embedding vectors representing input sequences, downstream vectors and / or set of vector determined by the operation of one or more layers or sublayers of a multi-layer model, and / or individual vectors (e.g., embedding vectors representing tokens of an input sequence and / or embeddings of tiles of an input image which may or may not include positional information encoding the location of such tokens relative to each other within a length of text and / or image, downstream vectors representing the processing of such embedding vectors by one or more layers or sublayers of a multi-layer model).

[0082] An ANN could include one or more filters that could be applied to the input andthe outputs of such filters could then be applied to the inputs of one or more neurons of the ANN. For example, such an ANN could be or could include a convolutional neural network (CNN). Convolutional neural networks are a variety of ANNs that are configured to facilitate ANN-based classification or other processing based on images or other large -dimensional inputs whose elements are organized within two or more dimensions. The organization of the ANN along these dimensions may be related to some structure in the input structure (e.g., as relative location within the one -dimensional space of sequence of tokens can be related to similarity or relevance between tokens of the sequence).

[0083] In example embodiments, a CNN includes at least one two-dimensional (or higher-dimensional) filter that is applied to an input; the filtered input is then applied to neurons of the CNN (e.g., of a convolutional layer of the CNN). The convolution of such a filter and an input could represent the color values of a pixel or a group of pixels from the input, in embodiments where the input is an image. A set of neurons of a CNN could receive respective inputs that are determined by applying the same filter to an input. Additionally or alternatively, a set of neurons of a CNN could be associated with respective different filters and could receive respective inputs that are determined by applying the respective filter to the input. Such filters could be trained during training of the CNN or could be pre-specified. For example, such filters could represent wavelet filters, center-surround filters, biologically-inspired filter kernels (e.g., from studies of animal visual processing receptive fields), or some other pre-specified filter patterns.

[0084] A CNN or other variety of ANN could include multiple convolutional layers (e.g., corresponding to respective different filters and / or features), pooling layers, rectification layers, fully connected layers, or other types of layers. Convolutional layers of a CNN represent convolution of an input image, or of some other input (e.g., of a filtered, downsampled, or otherwise-processed version of an input image), with a filter. Pooling layers of a CNN apply non-linear downsampling to higher layers of the CNN, e.g., by applying a maximum, average, L2-norm, or other pooling function to a subset of neurons, outputs, or other features of the higher layer(s) of the CNN. Rectification layers of a CNN apply a rectifying nonlinear function (e.g., a non-saturating activation function, a sigmoid function) to outputs of a higher layer. Fully connected layers of a CNN receive inputs from many or all of the neurons in one or more higher layers of the CNN. The outputs of neurons of one or more fully connected layers (e.g., a final layer of an ANN or CNN) could be used to determine information about areas of an input image (e.g., for each of the pixels of an input image) or for the image as a whole.

[0085] Neurons in a CNN can be organized according to corresponding dimensions of the input. For example, where the input is a sequence of token (a one-dimensional input, with each token representing one or more words, or fractions of words, in an input text string), neurons of the CNN (e.g., of an input layer of the CNN, of a pooling layer of the CNN) could correspond to locations in the one-dimensional input string / sequence. Connections between neurons and / or fdters in different layers of the CNN could be related to such locations.

[0086] Use of a machine learning model can be divided into a training phase 502 and an inference phase 504, in accordance with example embodiments. Some machine learning techniques involve training one or more machine learning algorithms, on an input set of training data to recognize patterns in the training data and provide output inferences and / or predictions about (patterns in the) training data. Such output could take the form of fdtered or otherwise modified versions of the input, e.g., an input sequence that represents text in a source language could be modified by the machine learning model into (i) an output sequence that represents text in a target language that has similar meaning or semantic content as the input sequence and / or (ii) an output set of embedding vectors that represent, in a semantic embedding space, the meaning or semantic content of the input sequence. The resulting trained machine learning algorithm can be termed as a trained machine learning model. For example, during a training phase one or more machine learning algorithms could be trained on training data to become trained machine learning models. Then, during an inference phase, a trained machine learning model can receive input data and one or more inference / prediction requests (perhaps as part of the input data) and responsively provide as an output one or more inferences and / or predictions.

[0087] As such, trained machine learning model(s) can include one or more models of one or more machine learning algorithms. Machine learning algorithm(s) may include, but are not limited to: an artificial neural network (e.g., a herein-described convolutional neural networks, a recurrent neural network, a Bayesian network, a hidden Markov model, a Markov decision process, a logistic regression function, a support vector machine, a suitable statistical machine learning algorithm, and / or a heuristic machine learning system), a Transformer, a support vector machine, a regression tree, an ensemble of regression trees (also referred to as a regression forest), a decision tree, an ensemble of decision trees (also referred to as a decision forest), or some other machine learning model architecture or combination of architectures. For example, the trained machine learning model(s) could include a plurality of artificial neural networks and other elements related to such networks (e.g., mixing or weighting matrices, attention heads or other attentional mechanisms, sums, products, feedforward connections)arranged according to the multi-layer and sublayer architecture of a Transformer or similar model architecture designed to process input sequences. Machine learning algorithm(s) may be supervised or unsupervised, and may implement any suitable combination of online and offline learning.

[0088] In some examples, machine learning algorithm(s) and / or trained machine learning model(s) can be accelerated using on-device coprocessors, such as graphic processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), and / or application specific integrated circuits (ASICs). Such on-device coprocessors can be used to speed up machine learning algorithm(s) and / or trained machine learning model(s). In some examples, trained machine learning model(s) can be trained, reside and execute to provide inferences on a particular computing device, and / or otherwise can make inferences for the particular computing device.

[0089] During a training phase, machine learning algorithm(s) can be trained by providing at least training data as training input using unsupervised, supervised, semi-supervised, and / or reinforcement learning techniques. Unsupervised learning involves providing a portion (or all) of training data to machine learning algorithm(s) and machine learning algorithm(s) 520 determining one or more output inferences based on the provided portion (or all) of training data. Supervised learning involves providing a portion of training data to the machine learning algorithm(s), with the machine learning algorithm(s) determining one or more output inferences based on the provided portion of training data, and the output inference(s) are either accepted or corrected based on correct results associated with training data. In some examples, supervised learning of machine learning algorithm(s) can be governed by a set of rules and / or a set of labels for the training input, and the set of rules and / or set of labels may be used to correct inferences of machine learning algorithm(s).

[0090] Semi-supervised learning involves having correct results for part, but not all, of the training data. During semi-supervised learning, supervised learning is used for a portion of training data having correct results, and unsupervised learning is used for a portion of training data not having correct results. Reinforcement learning involves machine learning algorithm(s) receiving a reward signal regarding a prior inference, where the reward signal can be a numerical value. During reinforcement learning, machine learning algorithm(s) can output an inference and receive a reward signal in response, where machine learning algorithm(s) are configured to try to maximize the numerical value of the reward signal. In some examples, reinforcement learning also utilizes a value function that provides a numerical valuerepresenting an expected total of the numerical values provided by the reward signal over time. In some examples, machine learning algorithm(s) and / or trained machine learning model(s) can be trained using other machine learning techniques, including but not limited to, incremental learning and curriculum learning.V. Experimental Results

[0091] The embodiments described herein were implemented and experimentally validated. The particulars and results of this experimental validation are provided in this section. Note that the particulars described in this section are intended only as illustrative, nonlimiting examples of the embodiments described herein.

[0092] The methods described herein were used to ‘train’ a number of bipartite graphs that, when evaluated, perform target tasks by, among other things, presenting at least one input to a machine learning in order to generate an output therefrom. This training process involved applying an evolutionary method to evaluate the fitness of graphs in a set of graphs, retaining and mutating more fit models while deleting or otherwise rejecting less-fit models. Such graphs may be considered a framework for representing “patterns of thought” that employ LLMs. At a high level, such a graph of thought (GoT) is a precise mechanism for how to orchestrate a set of LLM inference calls and other computations, with the aim of improving the response quality in some target task. Structurally, such a graph may consist of two types of nodes, data nodes (containing text) and function nodes (processing various text nodes and producing new text nodes), as well as directed (featureless) edges. Graphs were acyclic and directed (DAG), and bipartite, i.e., data nodes were connected only to function nodes, and vice-versa. The reason to treat text as nodes instead of edges has the benefit of easy sharing, e.g., the same question can be fed into multiple generation nodes that differ in their instruction.

[0093] The outside interface abstracts execution from the user, and has various text inputs and a single text output (the answer to the query). Variable input nodes are instancespecific, such as the question, the discrete choices, context, or few-shot examples. Other input nodes are held constant across all instances of a task, such as instructions for how to reason, decompose or reformulate other text nodes. Figure 1A-E shows example graphs (hand-crafted baselines) with their different node types. Such an approach is agnostic to the type or size of LLM or other model used at inference time, making graphs of thought a form of highly transferable knowledge.

[0094] Function Nodes

[0095] Each function node takes some text inputs and produces some text outputs (allof these are represented as data nodes). Some internally perform a call to an underlying language model (e.g., the “MLM” and “SELECT” nodes), others perform simpler string manipulations. The below is a list of the set of function nodes evaluated in the present study; additional or alternative are possible, as is the removal of one or more when implementing the methods described herein

[0096] Generate (“MLM,” a model-calling node): fed a text prompt to the LLM or other model and returned the generated response (up to a maximum length).

[0097] Select (“SELECT,” a model-calling node): fed a text prompt plus a discrete set of plausible continuations. Instead of auto-regressive generation, it produced log-probabilities for each of these continuation choices, and returned the highest-rated one (beneficial for multiple-choice questions).

[0098] Concatenate (“+”): concatenated the data in its inputs.

[0099] Chunk (not shown): split the data in a single string into a list of strings based on the given separator.

[0100] Vote (“VOTE”): performed simple majority voting over the given inputs (note that this is different from the Select node described above that uses an LLM to score and select the candidates).

[0101] Postprocess (“PROCESS”): post-processes the output of the LLM or other data node to extract a single value of interest (for instance, a number) from it. This is beneficial for, e.g., a voting operation over the outputs of different LLM generations that are free-form and hence, ineligible for voting over without any post-processing.

[0102] Hand-crafted Graphs of Thought as Seed Graphs

[0103] Task-specific and sometimes general-purpose performance improvements can be obtained by changing the singular prompt or introducing specific graphs of thought for inference. A collection of these were implemented within the framework as “seed” graphs, that is, starting points for evolution. Examples included simple input-output graphs, chain-of-thought reasoning, “Take a deep breath”, best-of-w, etc. Eigures 1A-E show the seed graphs that were used. These range from simple baselines such as few-shot prompting and zero-shot chain-of-thought prompting to stronger ones like best-of-w and self-consistency. In addition to this, constant text nodes were created that contained helper prompts that can be sampled and used as instructions to generate better intermediate outputs.

[0104] Method: Evolving Graphs of Thought

[0105] Graphs of thought were optimized with an evolutionary algorithm. Startingfrom an initial population of seed graphs, local variations (mutations) were produced, task performance (fitness) evaluated, and the better ones selected to retain in the population. More precisely, fitness was evaluated as the average accuracy over a randomly sampled batch of N = 128 training set samples, resampled each time a graph was reevaluated. Selection was implemented by a binary tournament, i.e., each round, two graphs from the current population were randomly sampled, their fitness compared (tie-breaking was random), a clone of the winner was mutated and replaced the loser in the population. See Algorithm 1 below for a complete description. The initial seed graph population of size k contained a set of established baselines as described above.Algorithm 1 Graph evolution algorithmInput:<?sw,:j: seed graphs' ia™.: training datasetAt: set of mutationsS', maximum graph population sizeR number of roundsN: number of examples per fitness evaluationRequire: i iL(lw5i < SI: let FITNESS^?. D,.) = 52 -77 a«mra.cv(m )2: festNULL ' ” Best graph 3:, / fet NULL Fitness of best graph 4: Q S— ir'seec i> Population of graphs 5: for r 4- 1 to R do 6: g,.(, Q i> Sample 2 distinct graphs (without replacement) 7: Z>,. = {®|,... ■--■ i-Araii; > Sample batch of Arexamples (without replacement) 8: faRTNESSQ^D...)9: fbHTNESSCgft,^) 10: let / L, / _ be the max and mitt of A, respectively, breaking ties randomly11.’ let p_ be the graphs corresponding to / +> / _ respectively12: if / +> then t> Maybe update the best graph 13: fbastW: fest G ff+15. end if16: f — •> Stait from clone of winner graph 17: while f invalid or f € Q do > Only unique and valid graphs allowed IS: m - At?> Sample a muiatiosi 19: f s— apply m io f Apply i-intatiou 20: end while21: if L 5 then22: G § \ {g- }?> Retiicsve loser from population 23: end if24; Q Q (j {<, / } t> Add cloned mutated winner to population 25: end forOutput: flbesst

[0106] Mutations

[0107] A collection of mutation operators were implemented that are meaningful in the context of graphs of thought (see below). In a single mutation, an operator was selected at random and applied to the clone, which is a stochastic operation. As this is not guaranteed to produce a valid graph (e.g., with inputs connected to the output), the mutation step was repeated until a graph with verified integrity was produced (possibly resampling the mutation operator). Each mutation operator involves several nodes; e.g. adding an edge requires an origin and a destination. The required nodes were sampled randomly from the graph once the mutation operator was known.

[0108] After executing the sampled mutation, any required additional changes were made to ensure all new nodes feed into the graph’s output, and did not remain ‘dangling’. This can be done by adding extra edges from the new nodes or by reassigning the output node, for instance. If that was not possible, more mutations were applied until the resulting graph was functionally different from the initial winner graph.

[0109] In addition to this, it is also important to maintain a population where each graph is unique. Thus, after applying a mutation that led to a valid graph, whether that mutation led to a functional change in the graph was checked. If not, another mutation was applied until the mutated graph was functionally different from the original winner graph.

[0110] Mutation Operators

[0111] Mutation operators are distinguished by the components of the graph they operate on. Simple ones affect the text content of a data node, or set the output node. More complex ones add new edges, new nodes, or both.

[0112] Prompt-based mutation “evolve_prompt.” Prompts that help improve performance (such as the widely used " Let’s think step by step") were themselves allowed to evolve. This was done by passing the prompt through an LLM along with an instruction to rephrase it. A set of such starter prompts was imported and optionally a node added to each graph that contained a list of these starter prompts.

[0113] Output-swap mutation “set output.” This allowed another data node to be designated as the output node, which was useful when the initial output node was further processed, verified, used in voting, etc.

[0114] Edge-based mutations. These can add, remove, and redirect edges between data nodes and function nodes. Each such mutation sampled nodes in a way that preserved the constraint that graphs were bipartite. The exact mutations used were: add_edge, remove_edge,change_edge_origin, change_edge_destination.

[0115] Additive function node mutations. These allowed different types of function nodes to be added to the graph. This involved sampling input data nodes for the function node, as well as the creation of a new data node that captured the output of the function node. In addition to this, the newly added nodes were (optionally) connected to the output. Simple examples are: add >enerate_node and add_concatenate_node.

[0116] Additive data node mutation “sample and clone.” This samples an instruction from a data node that contains a list of instructions, and constructs a new data node containing this instruction. This instruction can then be used in subsequent mutations to potentially improve performance. This allowed a list of prompts that have proved to be useful (such as " Let’s think step by step. ") to be declared useful.

[0117] ‘add chunk and select nodes:” added a chunk node to the graph and then added a select node that took as input the output of the chunk node coupled with an instruction. The motivation behind having this particular combination of function nodes is that extracting relevant bits of information out of a larger string can help with increasing performance. The chunk node would be responsible for splitting a larger string into several parts, and the select node would then select the most relevant part(s) of these.

[0118] “add >enerate_and_process_nodes:” added an LLM generate node to the graph and then postprocessed its output. This mutation was intended to assist with self-consistency. In this technique, several calls to the LLM are made, and all outputs were then aggregated by running them through a voting function. When LLM scoring is used, it is straightforward to perform voting, but free-form text generated by an LLM can be hard to process for voting. To ease this, the output of the LLM was allowed to be post-processed.

[0119] ‘add context:” added an LLM generate node to the graph and then added a concatenate node that concatenated it with its input. Adding an LLM generate node often led to its output being used as the input for a subsequent LLM generate node, which led to garbage. For instance, if we started with the input to the first generate node being " What is 2 + 3?”, its output would be "5". Now adding another LLM generate node from "5" leads to random facts about the number 5 being generated by the LLM, since it has no context about what 5 means.

[0120] Mutation Sets and Mutation Sampling

[0121] For different experiments, the sets of available mutations M were restricted to various subsets. Non-uniform sampling m ~ M was also investigated, e.g., using a bandit informed by past successes.

[0122] Computational Cost

[0123] Graphs initially do one or a few internal LLM calls, can grow in complexity via mutation, and hence become more and more expensive to execute. The number of LLM calls was measured as a secondary graph attribute, allowing Pareto-optimal performance to be assessed post-hoc (best task performance for any compute budget).

[0124] Results

[0125] The methods described herein were evaluated on two well-established benchmarks and one multi-player game.

[0126] GSM8K is a dataset of mathematical reasoning questions of intermediate difficulty (calibrated to pupils of age 15).

[0127] FANToM is a benchmark for assessing theory of mind capabilities, such as determining who knows what after a long dialogue where not every speaker is present throughout. With FANToM, the belief subset was emphasized, which asks questions about the beliefs of the people involved in a scene. This subset was referred to as FANToM-belief.

[0128] Chameleon, a social deduction game. A secret word is taken from a card of words on a general topic (e.g., food), visible to all players. There are two teams: a single ‘Chameleon’ player and the rest of the participants referred to as ‘non-Chameleons’. The Chameleon does not know the secret word but is trying to convince the other players they know it. Non-Chameleons know the secret word and want to let other players know that they know without letting the Chameleon in on the secret. Each player says one ‘hint’ word indicating they know the secret. When all players have voiced a hint, each player votes on who they think the chameleon is. If the Chameleon gets the most votes, the non-Chameleon team wins, otherwise the Chameleon wins. The Chameleon also wins if they get outed but correctly guess the secret word.

[0129] Separate experiments were run for the two benchmarks, but using the same hyper-parameters to show the method’s generality. Each dataset was split into training, validation and test sets; only training set examples were used for fitness evaluations. Starting from a population initialised with viable seed graphs, the evolutionary algorithm was run for a total of R rounds (typically 1000). The main result of any such run was a single discovered graph (chosen by highest average fitness). These graphs were then evaluated on the validation split of the dataset, to detect overfitting, and to inform hyper-parameter tuning. Each run was repeated for M independent seeds (typically 5), resulting in M graphs, whose average test set performance are reported. The test set was only used once, for the final reporting of results, notduring tuning or development of the method.

[0130] Two important variables are the specific LLM model used at inference time, and the set of seed graphs. Both had a large impact on performance, so their effects were separated by running experiments with different models, and starting from different sets of seed graphs.

[0131] Cloud compute infrastructure was used to run LLM inference on Gemini models. Overall compute usage for one experiment was 10-20 hours on about 100 accelerators.M 12.4.1 XL M 1.2.7 S XXS Gemma 7B Gemma 2B Graph N°1 S +4. O±2.4% -Ll±2,l% +1.3±2.4% Graph N°2 ||| +1.5±1.9% +6.5±1.4% +5.9+2.2%^W^I^ +3.5±2.1% +2.2±L8% Graph N°3 +17411% +6.8412% +7.2±3,1% +6. O±3.5% +6.3±2. S% +3.1±L9% Best seed +3.5 ±1,6% +6.5+10% +4.1+1.9% +2.2il.6%gglg +11+3.2% +2.7+2.5% graphNo graph 52.7±1.6% 63.0±2.8% 58.9±2.3% 54.3±1.3% 46.1±2.9% 43.6±1.5% 46.4±2.5%

[0133] Table 1 (above): Performance transfer of evolved graphs from Gemini M 1.2.4.1 to other models on FANToMbelief benchmark. While not fully consistent, performance improvements transferred from the base M 1.2.4.1 model to other models. One trend was that the more similar the model transferred to was to the base model, the bigger improvements one can expect from a transferred graph. In particular, XL and Gemma 2B did not appear to show successful transfer, being the two most dissimilar models from the base one. See also Figures 4A-C for an illustration of the evolved graphs Nsl-NsS.

[0134] Transfer across models

[0135] A number of experiments with the FANToM-belief benchmark using Gemini M 1.2.4.1 as the base model were run. The resulting graphs were then evaluated with the same model, the top 3 selected, and their performance on other models measured. All evaluations were done 5 times. For the no-graph baseline, the mean and the standard deviation are reported; for the graphs, report the difference of the mean compared to the baseline and the standard deviation are reported. Table 1 shows these results, and Figure 4A-C displays the top graphs.

[0136] Seed Graphs

[0137] The initial population was seeded with graphs that were derived from commonly used reasoning, planning, and prompting strategies. The advantages of doing this are two-fold: strong baselines to compare against, and autonomous discovery of improvements to the bestgraphs from recent advances. Some of the seed graphs that were use are illustrated in Figs. 1 A-E. The performance of the seed graphs used is shown in Fig. 5. Note that for the selfconsistency and best-of-N seed graphs, we use 2 variants: few-shot and zero-shot chain-of-thought.

[0138] Figure 5: Performance of the seed graphs used on the validation set for GSM8k. The seed graphs used were: 1) input-output, 2) few-shot, 3) zero-shot chain-of-thought, 4) fewshot + best-of-N, 5) zero-shot chain-of-thought + best-of-N, 6) few-shot + self-consistency, and 7) zero-shot chain-ofthought + self-consistency. The size of the marker indicates the standard deviation (scaled by a factor of 50 for visualisation purposes).

[0139] Hyperparameters

[0140] Figure 6 shows a sweep with 16 combinations of hyperparameters. There was significant variance of the results from run to run. The plot shows results of multiple runs with different hyperparameters. All runs were evolving graphs for the FANToMbelief benchmark, over 1024 rounds, in zero-shot mode, and evaluated on a validation subset of FANToM-belief. Some of these evaluations ran with 5 random seeds; for those, error bars show the standard deviation. Runs where error bars are missing used 1 random seed.

[0141] Performance versus Inference Compute

[0142] Figure 7 shows the empirical trade-off between inference compute cost (number of LLM calls) and performance (fitness) using Gemini -M and Gemini-XXS on GSM8K. White dots are measurements, the colored background is a fitted two-component Gaussian mixture. A global slightly positive correlation exists.

[0143] Evolutionary Dynamics

[0144] In the experiments reported here, fitness evaluations involved computing scores on a dataset. The most accurate way of doing that is to use the entire dataset. However, for a significant dataset size (e.g. thousands of data points, as is the case for GSM8K and FANToM), that approach becomes prohibitively expensive, and sampling a much smaller number of data points, typically between 8 and 128, can be for each fitness evaluation.

[0145] Fitness value caching

[0146] It is possible to reduce the cost of fitness computations by a factor of 2, by caching and re-using the value after a graph is evaluated for the first time. In this case, the fitness of a graph is evaluated only when it is added to the population, while the winner of each round is determined by comparing the cached fitness values. Hence per round only one evaluation is required, for the mutant, instead of two, for the two graphs that play that round.

[0147] This method has a downside in case the fitness evaluation is stochastic (which it is here, as mentioned above): it selects for graphs with ‘lucky’ evaluations. A stochastic fitness evaluation can be modeled as the sum of an intrinsic fitness term and a noise term r. such that f=fi + e.f can be high because the intrinsic fitness fi is high, or because the noise term e that was sampled happens to be large. The evolution algorithm is based on the premise that high- individuals will survive. Stochastic fitness evaluations will unavoidably lead to occasional eliminations of high- / individuals, but systematic selection of high-r evaluations increases this effect beyond necessity.

[0148] Estimating fitness measurement noise

[0149] If the cost reduction from caching the fitness value is foregone, and the fitness of the two players that have been sampled for a given round evaluated afresh every time, observations can be made of the uncertainty in the fitness estimation by recording all evaluations that have been performed and comparing evaluations of the same graph to each other. Plotting the obtained fitness value against the value from the preceding measurement on the same graph, a noise-free evaluation procedure would result in all points being on the (0, 0) - (max, max) diagonal. A cloud of points around that line was instead observed.

[0150] Note that given the baseline performance of « 80% for Gemini-M, there were only ~ 25 steps of improvement to be observed if using 128 samples per evaluation. If only 8 are used, the baseline gets 7 out of 8 correct, but due to the noise many graphs will get 8 out of 8 correct. With noise, one step of difference was not enough to detect improvement in an individual.

[0151] Effects of mutation on graph fitness

[0152] The left panel of Figure 8 shows mutations moving graphs using Gemini-M between roughly three groups: those performing at the baseline level (« 80%), some whose performance has been reduced significantly, around 50%, and finally some that are getting almost all questions wrong (around 10%). The x-axis is the measured fitness of the winner of each round in a tournament, and the y-axis is the measured fitness of the new graph that was obtained by mutating the winner. White dots correspond to concrete measurements; the color background represents a Gaussian mixture fitted to the white dots. The models used here are Gemini-M and Gemini-XXS, the task is GSM8K, and 128 data points were used per measurement. The density was highest in the top-right comer, which represents winners at baseline level producing mutants that are at the same level. There was also a significant cluster of mutants with a fitness around 50% being produced by winners at baseline level, and somemutants whose performance was destroyed completely by the mutation. The column around winner fitness of 50% shows that some mutations restored performance to baseline level, some keep it the same, and some reduced it to near zero. Mutants in the last group (performance close to zero) were rare enough to always be eliminated when they played in the next round, hence the absence of a column of density on the left of the plot. (If they were more frequent, they would eventually draw each other from time to time, and there would be very low-fitness winners.)

[0153] The right panel of figure 8 shows the same experiment for Gemini-XXS. The two non-baseline clusters are harder to distinguish there, due to the lower baseline performance.

[0154] Competing Graphs in the Game of Chameleon.

[0155] Graphs were evolved to compete against a reference graph in the game of Chameleon. A symmetric fitness measure was computed as the average score of a graph across an even number of games. Games alternated between the evolved graph taking on the role of a Chameleon and the reference graph that of the non-Chameleons, or vice-versa.

[0156] The graph evolved to perform best against a benchmark was an exploiter of the benchmark. To get better performance, the exploiter was built on using policy-space response oracles (PSRO). An overview of the PSRO algorithm for graphs is shown in Figure 9. PSRO with graphs began with a set of reference graphs equal to a single initial graph with a Nash probability of 1.0. A binary tournament then took place with a graph evolved to maximise its score in Chameleon against the set of reference graphs sampled with the set of Nash probabilities. The best graph from the binary tournament was added to set of reference graphs, the Nash probabilities were calculated and the cycle began again. Each PSRO iteration involved a binary tournament involving competitions against ever more challenging sets of policies. Figure 9 shows the performance of the PSRO derived graphs playing the initial graph in Chameleon on the y-axis. On the x-axis, the weight, i.e. Nash derived probability, is shown. The greater weights correspond to strong performance relative to the initial graph.

[0157] Figure 10: Transfer of evolved graphs across models for GSM8k. Two champion graphs from evolution runs (best-found after R rounds) were taken, for each model size (M 1.2.7, M, S and XXS) and evaluated across a wide range of Gemini model sizes.

[0158] Figure 11: Champion graph 1 on GSM8k.

[0159] Figure 12: Fitness measurement noise illustration. This plot shows, on the x-axis, the measured fitness of the winner of each round in a tournament, and on the y-axis the next measurement of the same graph’s fitness. White dots correspond to concretemeasurements; the color background represents a Gaussian mixture fitted to the white dots. The models used here were Gemini-M and Gemini-XXS, the task is GSM8K, and 128 data points were used per measurement.

[0160] Figure 13: Champion graph 2 on GSM8k.

[0161] Figure 14: Champion graph 3 on GSM8k.III. Example Systems

[0162] Figure 15 illustrates an example computing device 1500 that may be used to implement the methods described herein. By way of example and without limitation, computing device 1500 may be a cellular mobile telephone (e.g., a smartphone), a computer (such as a desktop, notebook, tablet, or handheld computer, a server), elements of a cloud computing system, a robot, a drone, an autonomous vehicle, or some other type of device. It should be understood that computing device 1500 may represent a physical computing device such as a server, a particular physical hardware platform on which a machine learning application operates in software, or other combinations of hardware and software that are configured to carry out machine learning functions as described herein.

[0163] As shown in Figure 15, computing device 1500 may include a communication interface 1502, a user interface 1504, a controller 1506 (which may include one or more processors), and data storage 1508, all of which may be communicatively linked together by a system bus, network, or other connection mechanism 1510.

[0164] Communication interface 1502 may function to allow computing device 1500 to communicate, using analog or digital modulation of electric, magnetic, electromagnetic, optical, or other signals, with other devices, access networks, and / or transport networks. Thus, communication interface 1502 may facilitate circuit-switched and / or packet-switched communication, such as plain old telephone service (POTS) communication and / or Internet protocol (IP) or other packetized communication. For instance, communication interface 1502 may include a chipset and antenna arranged for wireless communication with a radio access network or an access point. Also, communication interface 1502 may take the form of or include a wireline interface, such as an Ethernet, Universal Serial Bus (USB), or High-Definition Multimedia Interface (HDMI) port. Communication interface 1502 may also take the form of or include a wireless interface, such as a Wifi, BLUETOOTH®, global positioning system (GPS), or wide-area wireless interface (e.g., WiMAX or 3GPP Long-Term Evolution (LTE)). However, other forms of physical layer interfaces and other types of standard or proprietary communication protocols may be used over communication interface 1502.Furthermore, communication interface 1502 may comprise multiple physical communication interfaces (e.g., a Wifi interface, a BLUETOOTH® interface, and a wide-area wireless interface).

[0165] In some embodiments, communication interface 1502 may function to allow computing device 1500 to communicate, with other devices, remote servers, access networks, and / or transport networks. For example, the communication interface 1502 may function to access one or more machine learning models and / or input therefor via communication with a remote server or other remote device or system in order to allow the computing device 1500 to use the machine learning model(s) to generate outputs based on input data. For example, the remote system could be an inference server and the computing system 1500 could be a personal computer or server that is evaluating and / or generating a graph as described. Alternatively, the computing system 1500 could, itself, also include or be such an inference server.

[0166] User interface 1504 may function to allow computing device 1500 to interact with a user, for example to receive input from and / or to provide output to the user. Thus, user interface 1504 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, and so on. User interface 1504 may also include one or more output components such as a display screen which, for example, may be combined with a presence-sensitive panel. The display screen may be based on CRT, LCD, and / or LED technologies, or other technologies now known or later developed. User interface 1504 may also be configured to generate audible output(s), via a speaker, speaker jack, audio output port, audio output device, earphones, and / or other similar devices.

[0167] Controller 1506 may comprise one or more general purpose processors - e.g., microprocessors - and / or one or more special purpose processors - e.g., digital signal processors (DSPs), graphics processing units (GPUs), floating point units (FPUs), network processors, tensor processing units (TPUs), or application-specific integrated circuits (ASICs). In some instances, special purpose processors may be capable of image processing (e.g., application of CNN kernels or other filters to images via, e.g., convolution), machine learning model training, execution, and / or inference, among other applications or functions. Data storage 1508 may include one or more volatile and / or non-volatile storage components, such as magnetic, optical, flash, or organic storage, and may be integrated in whole or in part with controller 1506. For example, a portion of the data storage 1508 may be implemented as cache or other on-chip memory of a graphics processing unit or tensor processing unit integratedcircuit and / or as RAM or some other variety of storage that is collocated with a GPU or TPU, e.g., on a graphics card, tensor acceleration card, or other semi -discrete subsystem of the overall system 1500. Such storage could be used to store parameters that define a graph as described herein. Data storage 1508 may include removable and / or non-removable components.

[0168] Controller 1506 may be capable of executing program instructions 1518 (e.g., compiled or non-compiled program logic and / or machine code) stored in data storage 1508 to carry out the various functions described herein. Therefore, data storage 1508 may include a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by computing device 1500, cause computing device 1500 to carry out any of the methods, processes, or functions disclosed in this specification and / or the accompanying drawings. The execution of program instructions 1518 by controller 1506 may result in controller 1506 using data 1512.

[0169] By way of example, program instructions 1518 may include an operating system 1522 (e.g., an operating system kernel, device driver(s), and / or other modules) and one or more application programs 1520 (e.g., functions for executing trained machine learning models and / or training such models) installed on computing device 1500. Data 1512 may include stored training data 1514 (e.g., examples of inputs and outputs to successfully perform a target task) that could be used to train or otherwise generated one or more graphs 1516 as described herein.

[0170] Application programs 1520 may communicate with operating system 1522 through one or more application programming interfaces (APIs). These APIs may facilitate, for instance, application programs 1520 reading, writing, and / or evaluating a trained graph 1516, transmitting or receiving information via communication interface 1502, receiving and / or displaying information on user interface 1504, and so on.

[0171] Application programs 1520 may take the form of “apps” that could be downloadable to computing device 1500 through one or more online application stores or application markets (via, e.g., the communication interface 1502). However, application programs can also be installed on computing device 1500 in other ways, such as via a web browser or through a physical interface (e.g., a USB port) of the computing device 1500. IV. Example Methods

[0172] Figure 16 is a flowchart of a method 1600 for generating a graph to perform a target task as described herein. The method 1600 includes determining, with respect to the target task, a first fitness of a first graph in a set of graphs by evaluating the first graph, whereinthe first graph defines a set of processing functions and a pattern of connection of input and output between the processing functions, and wherein at least one processing function of the set of processing functions involves applying an input to a generative machine learning model to generate an output (1610). The method 1600 additionally includes, responsive to determining that the first fitness exceeds a fitness of a second graph in the set of graphs, selecting and mutating the first graph (1620). The method 1600 yet further includes determining, with respect to the target task, a second fitness of the mutated first graph by evaluating the mutated first graph (1630). The method 1600 could include additional or alternative steps or features.

[0173] Figure 17 is a flowchart of a method 1700 for performing a target task as described herein. The method 1700 applying, to a graph generated according to the method 1600 of Figure 16, a target input to generate a target output (1710). The method 1700 could include additional or alternative steps or features.VI. Conclusion

[0174] The above detailed description describes various features and functions of the disclosed systems, devices, and methods with reference to the accompanying figures. In the figures, similar symbols typically identify similar components, unless the context indicates otherwise. The illustrative embodiments described in the detailed description, figures, and claims are not meant to be limiting. Other embodiments can be utilized, and other changes can be made, without departing from the scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are explicitly contemplated herein.

[0175] With respect to any or all of the message flow diagrams, scenarios, and flowcharts in the figures and as discussed herein, each step, block and / or communication may represent a processing of information and / or a transmission of information in accordance with example embodiments. Alternative embodiments are included within the scope of these example embodiments. In these alternative embodiments, for example, functions described as steps, blocks, transmissions, communications, requests, responses, and / or messages may be executed out of order from that shown or discussed, including in substantially concurrent or in reverse order, depending on the functionality involved. Further, more or fewer steps, blocks and / or functions may be used with any of the message flow diagrams, scenarios, and flow charts discussed herein, and these message flow diagrams, scenarios, and flow charts may becombined with one another, in part or in whole.

[0176] A step or block that represents a processing of information may correspond to circuitry that can be configured to perform the specific logical functions of a herein-described method or technique. Alternatively or additionally, a step or block that represents a processing of information may correspond to a module, a segment, or a portion of program code (including related data). The program code may include one or more instructions executable by a processor for implementing specific logical functions or actions in the method or technique. The program code and / or related data may be stored on any type of computer-readable medium, such as a storage device, including a disk drive, a hard drive, or other storage media.

[0177] The computer-readable medium may also include non-transitory computer-readable media such as computer-readable media that stores data for short periods of time like register memory, processor cache, and / or random access memory (RAM). The computer-readable media may also include non-transitory computer-readable media that stores program code and / or data for longer periods of time, such as secondary or persistent long term storage, like read only memory (ROM), optical or magnetic disks, and / or compact-disc read only memory (CD-ROM), for example. The computer-readable media may also be any other volatile or non-volatile storage systems. A computer-readable medium may be considered a computer-readable storage medium, for example, or a tangible storage device.

[0178] Moreover, a step or block that represents one or more information transmissions may correspond to information transmissions between software and / or hardware modules in the same physical device. However, other information transmissions may be between software modules and / or hardware modules in different physical devices.

[0179] While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope being indicated by the following claims.

Claims

CLAIMSWhat is claimed is:

1. A method for generating a graph to perform a target task, the method comprising:determining, with respect to the target task, a first fitness of a first graph in a set of graphs by evaluating the first graph, wherein the first graph defines a set of processing functions and a pattern of connection of input and output between the processing functions, and wherein at least one processing function of the set of processing functions involves applying an input to a generative machine learning model to generate an output;responsive to determining that the first fitness exceeds a fitness of a second graph in the set of graphs, selecting and mutating the first graph; anddetermining, with respect to the target task, a second fitness of the mutated first graph by evaluating the mutated first graph.

2. The method of claim 1, wherein the set of processing functions of the first graph includes at least one of (i) concatenating two or more input strings together to generate an output string, (ii) concatenating two or more input strings together to generate an output list of strings, (iii) selecting an output from a set of two or more inputs, (iv) determining as an output a most common content of a set of two or more inputs, (v) separating an input string into two or more output strings, (vi) executing an instruction contained in an input string to generate an output, or (vii) extracting a textual or numeric output from an input string.

3. The method of claim 2, wherein mutating the first graph includes at least one of (i) modifying textual content of at least input string of the first graph, (ii) changing which output of the first graph is used as an overall output of the first graph when evaluating the first graph, (iii) changing the pattern of interconnection between the processing functions of the first graph, (iv) adding a processing function to the set of processing functions of the first graph, (v) adding a textual input string to the first graph, or (vi) adding, to the first graph, a sub-graph, wherein the sub-graph defines a set of additional processing functions and a pattern of connection of input and output between the additional processing functions.

4. The method of claim 2, wherein mutating the first graph includes randomly selecting, according to a non-uniform distribution, one of (i) modifying textual content of at least input string of the first graph, (ii) changing which output of the first graph is used as an overall output of the first graph when evaluating the first graph, (iii) changing the pattern of interconnection between the processing functions of the first graph, (iv) adding a processing function to the set of processing functions of the first graph, (v) adding a textual input string to the first graph, or (vi) adding, to the first graph, a sub-graph, wherein the sub-graph defines a set of additional processing functions and a pattern of connection of input and output between the additional processing functions.

5. The method of any of claims 2-4, wherein selecting the output from the set of two or more inputs comprises applying the set of two or more inputs to a machine learning model to select one of the two or more inputs.

6. The method of claim 5, wherein applying the set of two or more inputs to a machine learning model to select one of the two or more inputs comprises determining, by the machine learning model, scores for each of the two or more inputs, and wherein selecting the output from the set of two or more inputs comprises selecting the input of the two or more inputs that has the greatest score.

7. The method of any of claims 2-6, wherein extracting a textual or numeric output from an input string comprises extracting the textual or numeric output from the input string according to a pre-specified formatting.

8. The method of any of claims 3-7, wherein adding, to the first graph, the subgraph comprises adding, to the first graph a sub-graph that includes:a first set of processing functions that apply a common input to at least one generative machine learning model to generate a set of respective generative outputs, andat least one of (i) a processing step that applies the set of generative outputs to a machine learning model to select one of the set of generative outputs as an output of the sub-graph, or (ii) a set of processing steps that extract respective textual or numeric outputs from respective ones of the set of generative outputs and a further processing step that determines, as an outputof the sub-graph, a most common content of the textual or numeric outputs extracted from the set of generative outputs.

9. The method of any preceding claim, wherein the first graph is defined by a first set of nodes that represent respective processing functions of the set of processing functions and a second set of nodes that represent information input to and / or output from processing functions of the set of processing functions, and wherein the first graph is bipartite with respect to the first set of nodes and the second set of nodes.

10. The method of any preceding claim, further comprising:using the mutated first graph to perform the target task by applying, to the mutated first graph, a target input to generate a target output.

11. The method of claim 10, further comprising, prior to using the mutated first graph to perform the target task:selecting the mutated first graph from the set of graphs based on a number of executions of a generative machine learning model that are represented by the mutated first graph.

12. The method of any preceding claim, further comprising:generating an additional graph to perform a second task that differs from the target task by:determining, with respect to the second task, a third fitness of the mutated first graph by evaluating the mutated first graph;responsive to determining that the third fitness exceeds a fitness of a third graph in the set of graphs, selecting and further mutating the mutated first graph; and determining, with respect to the second task, a fourth fitness of the further mutated first graph by evaluating the further mutated first graph.

13. The method of any preceding claim, wherein evaluating the first graph comprises generating an output of the first graph and, based on the output of the first graph, determining the first fitness, wherein determining the first fitness based on the output of the first graph and mutating the first graph are performed by a first computational system, andwherein applying the input to the generative machine learning model to generate the output comprises:transmitting, by the first computational system to a second computational system that is remote to the first computational system, an indication of the input; andreceiving, by the first computational system from the second computational system, an indication of the output generated by the second computational system applying the input to the generative machine learning model14. The method of any preceding claim, further comprising, responsive to determining that the first fitness exceeds the fitness of the second graph in the set of graphs:removing, from the set of graphs, the second graph; andgenerating, within the set of graphs, a replicate of the first graph, wherein mutating the first graph comprises mutating at least one of the first graph or the replicate of the first graph within the set of graphs.

15. The method of any preceding claim, wherein the generative machine learning model comprises at least one of a transformer or a large language model.

16. A method for performing a target task comprising:applying, to a graph generated according to any of claims 1 - 15, a target input to generate a target output.

17. A non-transitory computer readable medium having stored thereon program instructions executable by at least one processor to cause the at least one processor to perform the method of any preceding claim.

18. A system comprising:at least one processor; anda non-transitory computer-readable medium, having stored therein instructions executable by the at least one processor to cause the system to perform the method of any of claims 1-16.

Images