Self-consistency metric for validating outputs of tools controlled by machine-learned models

Abstract

Example aspects of the present disclosure provide an example method. In some implementations, the example method can include receiving a request to perform a task; generating, using a machine-learned model, a plurality of tool paths for performing the task, wherein a respective tool path comprises one or more tool calls generated by the machine- learned model, each tool call configured to cause a corresponding tool to perform a step of the task; selecting a response to the request based on a consistency metric determined by evaluating a plurality of output states respectively from the plurality of tool paths; and returning the response.

Description

SELF-CONSISTENCY METRIC FOR VALIDATING OUTPUTS OF TOOLS CONTROLLED BY MACHINE-LEARNED MODELSFIELD

[0001] The present disclosure relates generally to machine learning processes and machine-learned devices and systems. More particularly, the present disclosure relates to infrastructures for improving performance of tool control by machine-learned models.BACKGROUND

[0002] A computer can receive input(s). The computer can execute instructions to process the input(s) to generate output(s) using a parameterized model. The computer can obtain feedback on its performance in generating the outputs with the model. The computer can generate feedback by evaluating its performance. The computer can receive feedback from an external source. The computer can update parameters of the model based on the feedback to improve its performance. In this manner, the computer can iteratively “learn” to generate the desired outputs. The resulting model is often referred to as a machine-learned model.SUMMARY

[0003] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the embodiments.

[0004] In an aspect, the present disclosure provides a first example method. In some implementations, the first example method includes receiving a request to perform a task. In some implementations, the first example method includes generating, using a machine- learned model, a plurality' of tool paths for performing the task, wherein a respective tool path includes one or more tool calls generated by the machine-learned model, each tool call configured to cause a corresponding tool to perform a step of the task. In some implementations, the first example method includes selecting a response to the request based on a consistency metric determined by evaluating a plurality' of output states respectively from the plurality’ of tool paths. In some implementations, the first example method includes returning the response.

[0005] In an aspect, the present disclosure provides a second example method. In some implementations, the second example method includes transmitting, from a client computing system and to a server computing system, a request to perform a task. In some implementations, the second example method includes receiving, by the client computing system and from the server computing system, a response based on performance of the task. In some implementations, the response is obtained using an implementation of the first example method or the second example method.

[0006] In an aspect, the present disclosure provides an example one or more non- transitory computer-readable media storing instructions that are executable by one or more processors to cause a computing system to perform operations, the operations including an implementation of the first example method or the second example method.

[0007] In an aspect, the present disclosure provides an example computing system. The example computing system can include one or more processors and one or more non- transitory computer-readable media storing instructions that are executable by one or more processors to cause a computing system to perform operations. The operations can include an implementation of the first example method or the second example method.

[0008] Other example aspects of the present disclosure are directed to other systems, methods, apparatuses, tangible non-transitory computer-readable media, and devices for performing functions described herein. These and other features, aspects, and advantages of various implementations will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate implementations of the present disclosure and, together with the description, help explain the related principles.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a block diagram of an example system for validating outputs of a machine-learned task processing system according to example implementations of aspects of the present disclosure.

[0010] Figure 2 is a block diagram of an example tool path for validating outputs of a machine-learned task processing system according to example implementations of aspects of the present disclosure.

[0011] Figure 3A is a block diagram of an example tool path for validating outputs of a machine-learned task processing system according to example implementations of aspects of the present disclosure.

[0012] Figure 3B is a block diagram of an example tool path for validating outputs of a machine-learned task processing system according to example implementations of aspects of the present disclosure.

[0013] Figure 3C is a block diagram of an example tool path for validating outputs of a machine-learned task processing system according to example implementations of aspects of the present disclosure.

[0014] Figure 4 is a block diagram of an example validator for validating outputs of a machine-learned task processing system according to example implementations of aspects of the present disclosure.

[0015] Figure 5 is a flow chart diagram illustrating an example method for validating outputs of a machine-learned task processing system according to example implementations of aspects of the present disclosure.

[0016] Figure 6 is a flow chart diagram illustrating an example method for controlling tools using a machine-learned model according to example implementations of aspects of the present disclosure.

[0017] Figure 7 is a flow chart diagram illustrating an example method for controlling tools using a machine-learned model according to example implementations of aspects of the present disclosure.

[0018] Figure 8 is a flow chart diagram illustrating an example method for training a machine-learned model according to example implementations of aspects of the present disclosure.

[0019] Figure 9 is a block diagram of an example processing flow for using machine- learned model(s) to process input(s) to generate output(s) according to example implementations of aspects of the present disclosure.

[0020] Figure 10 is a block diagram of an example sequence processing model according to example implementations of aspects of the present disclosure.

[0021] Figure 11 is a block diagram of an example technique for populating an example input sequence for processing by a sequence processing model according to example implementations of aspects of the present disclosure.

[0022] Figure 12 is a block diagram of an example model development platform according to example implementations of aspects of the present disclosure.

[0023] Figure 13 is a block diagram of an example training workflow for training a machine-learned model according to example implementations of aspects of the present disclosure.

[0024] Figure 14 is a block diagram of an inference system for operating one or more machine-learned model(s) to perform inference according to example implementations of aspects of the present disclosure.

[0025] Figure 15 is a block diagram of an example networked computing system according to example implementations of aspects of the present disclosure.

[0026] Figure 16 is a block diagram of an example computing device according to example implementations of aspects of the present disclosure.

[0027] Figure 17 is a block diagram of an example computing device according to example implementations of aspects of the present disclosure.DETAILED DESCRIPTION

[0028] Generally, the present disclosure is directed to techniques for applying a consistency metric for validating outputs of tools controlled by machine-learned model(s). For example, machine-learned models can generate instructions for controlling tools to perform a task responsive to a request. The tools can generate outputs responsive to the provided instructions. In some scenarios, due to the stochastic nature of some machine- learned models, the tool outputs might vary. To improve an overall performance on the task, multiple sets of instructions can be obtained from the machine-learned model to afford diverse approaches to performing the task. The end results from the different approaches can be compared to identify features in which the different approaches were consistent. Such consistency can be an indication that the consistently obtained features are responsive to the initial request. Outlier features can be less likely to be responsive to the initial request.

[0029] Some existing processing systems can use machine-learned models to generate instructions for implementing external tools. However, such systems generally fail to employ robust sample sizes of instruction sets. Generally, such systems might only employ one instruction for one tool without exploring alternative solution spaces at query time. Such systems can be susceptible to suboptimal performance, such as in the scenario in which the singular tool instruction either is the wrong instruction for the right tool, the wrong instruction for the wrong tool, the right instruction for the wrong tool, etc. Further, such systems might only obtain a singular output from the singular tool, an approach which can lack certain signals to improve confidence that the output is effective to respond to the task. In such systems, an end user or other downstream system might be responsible for receiving the output, identifying any errors or failures thereof, and repeating the query. This can lead tounnecessary' processing of multiple task repetitions, multiple network communications, failed or abandoned tasks, etc.

[0030] Advantageously, example implementations according to aspects of the present disclosure can sample multiple different sets of instructions, optionally for multiple different tools, to explore potential solution space(s) for performing task(s). By generating multiple outputs using diverse processing paths, example implementations according to aspects of the present disclosure can leverage a self-consistency metric to identify output(s) with features that are shared across multiple of the outputs. In this manner, for instance, example implementations can use the multiple outputs to “vote” on the preferred output. Based on one or more preferred outputs, example implementations can obtain a response to serve in response to the initial request.

[0031] In this manner, for instance, example implementations of the present disclosure can provide for machine-learned model systems that can engage and leverage external tools with improved reliability', consistency, and creativity. The present disclosure applies techniques of self-consistency adapted for tool usage, which can improve the orchestration of single or multi-step tool-based actions. In general, example implementations involve generating multiple sequences of tool paths, executing these paths, and then evaluating the intermediate or output states for consistency. Paths with consistent states can be retained for further processing (e.g., for reluming an output). Paths with state(s) that deviate can be pruned as containing error.

[0032] For example, a machine-learned model can be tasked with performing a file conversion task to convert an input file to a different format. The machine-learned model can be configured to generate commands for controlling a software tool to perform the file conversion. For example, the machine-learned model can be configured to generate textual commands for entry via a command-line interface of a terminal application. Such commands can be modified using various flags to indicate different options. For instance, the syntax~> command - -option value can indicate to perform command with option set to value.

[0033] The machine-learned model can predict which commands to use, which options to set, and which values to apply for those options. However, the model could predict incorrect or invalid configurations. The command execution could fail. The command output could return the wrong result or a result that is not responsive to the initial request.

[0034] One approach to mitigating these failures would be to simply re-run the command until successful. But re-running the command sequentially can lead to undueprocessing delays and suboptimal compute utilization. Additionally, it may be hard to judge when the output is correct even if the command succeeded.

[0035] Advantageously, example implementations of the present disclosure can perform multiple tool call executions in multiple tool paths in parallel. For example, a computing system can sample multiple different outputs from the model to obtain multiple distinct tool calls. The computing system can parse each tool call and initiate execution of each tool call in parallel. The computing system can supervise execution of each tool path to monitor for inoperability, constraint violation, or other failure modes. Tool paths that fail can be pruned or restarted (e.g., by sampling a new tool call from the model).

[0036] The computing system can receive outputs from each of multiple tool paths. The outputs can be compared using a consistency metric to evaluate whether there is majority or plurality alignment on features of the output. For instance, a consistency metric can be evaluated across a sampling of diverse tool paths (e.g., representing diverse approaches to resolving the query ) and corresponding outputs. For example, a set of outputs with diverse tool execution strategies can be polled to obtain a majority or plurality "‘vote"’ on the ultimate output. In this manner, the tool path outputs can self-corroborate to improve the robustness of model output and improve accuracy of the ultimate responses.

[0037] The computing system can evaluate intermediate states for each tool path to confirm that they are progressing towards a valid completion state. An intermediate state can be a tool output of a tool in a multi-tool-call sequence. An intermediate state can include a status update output by a tool. The evaluation of intermediate states can involve the use of heuristics, probabilistic models, the same or additional machine-learned model(s) that can assess the quality and relevance of the intermediate outputs.

[0038] If an intermediate state is determined to be divergent from the desired criteria or exhibits signs of failure, such as inoperability of a tool or tool call, the corresponding tool path can be pruned or corrected. This pruning can be based on redundancy, where similar tool paths yielding deterministic outcomes within a margin of similarity are identified and reduced to a representative subset, or based on explicit failures detected during the execution. This proactive management of tool paths can optimize computational resources and focus efforts on the most promising paths to a successful task completion.

[0039] In addition to consistency measures over output states, the computing system can perform consistency evaluations over intermediate states. For example, such checks can involve comparing intermediate states across different tool paths to identity’ commonalities or discrepancies. Too much commonality' (e.g., identity) can trigger pruning of redundantshapes. By maintaining tool paths having intermediate states that improve the diversity of the set of tool paths, the system can explore a broader solution space and increase the chances of finding an optimal or satisfactory solution. Self-consistency techniques can be applied to various components of the task, such as sections of code (generated by the model for execution), function calls, or API calls, etc.

[0040] Consistency’ metrics can be computed by various different techniques. In some implementations, the system can compare outputs or intermediate states to compute a consistency metric. One method for comparison can be byte-level diffs, which can involve a direct comparison of the binary’ or text-based output data from each tool path. Byte-level diffs can be useful for tasks where the exact output is important, such as in file checksum verification, when comparing output strings for exact identity, etc.

[0041] Another example method for comparing outputs can involve semantic similarity’ comparisons, which can assess the meaning or intent behind the outputs rather than their exact textual or binary' representation. This can be useful for tasks that admit different valid outputs that equivalently fulfill the query’. Semantic similarity can be computed using natural language processing techniques, such as word embeddings, that can capture the context and semantics of the text. In general, vector-based similarity measures, such as cosine similarity’ measures, between vector representations of the output objects can be used to determine a similarity between the output objects.

[0042] For more structured data, such as JSON or XML outputs, tree or graph-based comparison algorithms can be employed to assess structural and content-based consistency. These techniques can include traversing a hierarchical structure of the data and comparing elements and attributes independent of possible variability' in order or structure may not affect the data’s semantic content.

[0043] For examples relating to visual outputs or graphical user interfaces, image comparison techniques such as pixel-by-pixel comparison or more sophisticated image hashing methods can be applied. For example, perceptual hashing can create a hash value based on the visual content of an image, allowing for quick and efficient comparison of visual outputs even if they have been slightly altered or compressed. Similarly, image embeddings can be used to perform a similarity' comparison.

[0044] In some examples, the machine-learned model (that generated the tool call) can itself evaluate a current state of the tool path (e.g., an intermediate state, an output state). For instance, a description of a current state can be input to the model along with the original query’ and an instruction to evaluate the next step for the tool path. The model can output anext action that can be executed in the tool path. If the model determines that the current state is on a likely path to success, it can generate the next tool call to continue the task. If the evaluation indicates potential issues or deviations from the desired outcome, the model can predict corrective actions, such as modifying a parameter or choosing an alternative tool call. If the evaluation indicates a failure mode of the tool path, the model can output a termination or end state as a next action.

[0045] Each tool path can be executed in a different execution environment, which can provide for the isolation and independence of each tool path execution. This can help prevent interference between tool paths. For instance, separate execution environments can be instantiated using virtual machines, containers, or isolated runtime contexts within a single operating system.

[0046] Virtual machines can provide isolation at the hardware level, while containers can provide isolation at the operating system level. Runtime contexts can also be isolated using various programming language features or runtime settings, such as different memory spaces, threads, etc.

[0047] In general, example execution environments can be configured to prevent the executing tool paths from interfering with resources used by other tool paths. The execution environment can enforce resource allocation, access controls, or execution permissions to maintain the integrity and independence of each task instance. For example, resource groups (e.g.. cgroups) or similar mechanisms can be used to control the amount of CPU. memory, and I / O bandwidth that each task can consume. This can help prevent any single task from monopolizing system resources.

[0048] Some resources can be cached for use across multiple tool paths. This can reduce redundant computations and I / O operations, thereby improving the overall efficiency of the system. Caching can be implemented at various levels, including within the execution environments themselves or at a shared layer accessible by all environments. Examples of resources that can be cached include, for instance, common libraries, datasets, intermediate computation results, and configuration files. The caching strategy can implement one or more learned components that can predict which resources are likely to be reused and preemptively cause them to be cached based on the task context or historical usage patterns.

[0049] Example implementations of the present disclosure can provide various technical effects and benefits. For example, by processing different paths in parallel, example implementations can decrease a latency by avoiding naively processing a single instruction, receiving an error, retrying, receiving an error, etc. For instance, example implementations ofthe present disclosure can offer lower latency by effectively deconstructing and front-loading the tool call refinement task. For instance, in lieu of generating, in series, sequentially optimized iterations of a few tool calls to cause a tool to perform a task of interest by incrementing values of the tool call parameters, example implementations of the present disclosure can generate a batch of tool calls with parameter values across a range of the parameters. Generated with sufficient recall, the batch of tool calls can effectively achieve the same output as would be obtained from the time-consuming sequential optimization. However, the batch generation can be highly parallelizable, since each tool call in the batch need not await execution of any prior tool call. Evaluating tool responses from the batch of tool calls can also be processed in parallel.

[0050] In another aspect, example implementations of the present disclosure can provide for improved error checking on the output of tools. For instance, in lieu of naively returning the output of a tool, tool outputs can be collected across a plurality of different tool paths to effectively increase a signal-to-noise ratio. For instance, shared or similar feature values along a particular dimension of the output(s) can be considered to provide good signals. Outlier feature values along that dimension can be considered noise, and the associated output(s) can be ignored or rejected.

[0051] In another aspect, example implementations of the present disclosure can provide for decreased network bandwidth utilization. For instance, by automatically processing different paths to identify quality output(s) in lieu of sequentially generating outputs and awaiting a user’s negative feedback, example implementations can verify internally the qualify of an output before sending, thereby decreasing a likelihood of further iterations of communication(s). This can decrease a number of messages sent over a network to a downstream system and a number of messages received over the network from the downstream system.

[0052] Further, when using probabilistic generative models to control external tools, a technical problem can arise when the model generates outputs that include commands or actions intended for execution by external tools that may be inaccurate or ‘"hallucinated.” This can result in undesirable outcomes, such as the generation of erroneous data, the execution of undesired or unhelpful actions, or complete failure to achieve the task at hand.

[0053] Advantageously, example implementations according to the present disclosure can provide solutions to this and other technical problems. For instance, example implementations of the present disclosure can generate multiple tool paths for performing a requested task. Each tool path can attempt to accomplish at least one step to perform therequested action. The use of multiple tool paths can provide for corroborating the output(s) returned from the tools. This corroboration can increase confidence in the accuracy of the output(s). For example, spurious commands or incorrect outputs may have little corroboration among the set of tool paths. In contrast, correct or desired responses may enjoy higher levels of corroboration. In this manner, for instance, example implementations of the present disclosure can provide solutions to the problem of probabilistic models generating commands that initiate incorrect or inoperable tool executions.

[0054] Another technical problem can arise when multiple processing tasks interfere with one another when accessing or modifying the same resources. For instance, a task can involve adding data to a table. In some situations, if multiple co-executing operations attempt to insert data into the table at the same time, data describing the table obtained by a first operation can be stale due to a background update by a second operation. This can lead to failed operations, data errors, etc.

[0055] Advantageously, example implementations according to the present disclosure can provide solutions to this and other technical problems. For instance, example implementations of the present disclosure can initiate execution of the tool call in an execution environment associated with a corresponding tool path. The execution environment can be sandboxed to prevent the tool path from negatively affecting other tool paths that may be executing in parallel. For example, each tool path can be executed in a separate directory, container, virtual machine, physical machine, etc. In this manner, for example, example implementations of the present disclosure can help facilitate co-execution of multiple different tool paths to perform the same or similar operations with minimal or no interference. Further, by executing tool paths in separate execution environments, example implementations can enable individual control of the amount of computational resources available to each. This can lead to an improved use of computational resources, as each path can be managed and controlled more effectively. For instance, race conditions or failure loops created by a tool path can be individually detected and terminated without allowing the tool path to consume more resources than were originally designated for it.

[0056] Another technical problem can arise when executing tasks in separate environments. In some situations, strict isolation can lead to redundant processing if valid work completed by one task cannot be re-used later. For example, a task downloading a dataset within one virtual machine may be unaw are that the same dataset has already been downloaded by another task executing in a different virtual machine.

[0057] Advantageously, example implementations according to the present disclosure can provide solutions to this and other technical problems. For instance, example implementations of the present disclosure can cache resources for use across multiple tool paths or execution environments. The cache can be read-only to prevent interference across tool paths. Such caching can provide that commonly used data or computational routines do not need to be repeatedly loaded or executed, thereby saving time and reducing the load on the processing and memory resources of the executing machine. The cache can be implemented in hardware (e.g., dedicated memory or storage devices) or software (e.g., dedicated swap files / folders / partition / etc.).

[0058] Another technical problem can arise when executing multiple tasks in parallel. In some situations, certain tasks can fail or begin to diverge from a desired course of action. If such a task fails or diverges early in an execution batch, the processing lane allocated to that task can be suboptimally used (e.g., underutilization of potential processing power, active expenditure of processing power for unusable outputs, etc.).

[0059] Advantageously, example implementations according to the present disclosure can provide solutions to this and other technical problems. For instance, example implementations of the present disclosure can employ machine-learned models or heuristic tools to assess the progression of each tool path during operation. This monitoring can support the pruning of inefficient, failed, or redundant paths, which can help optimize the utilization of computing resources by focusing computational efforts on the most promising avenues for task completion.

[0060] In addition to providing solutions to the above-noted technical challenges, a technical effect of example implementations of the present disclosure is increased energy' efficiency in performing operations using machine-learned models, thereby improving the functioning of computers implementing such models. For instance, example implementations can provide for more energy-efficient runtime execution or inference by carefully managing execution of multiple tool paths to achieve higher-quality results with less redundant processing. In some scenarios, increased energy efficiency can provide for less energy to be used to perform a given task (e.g.. less energy expended to maintain the model in memory, less energy^ expended to perform calculations within the model, etc.). In some scenarios, increased energy' efficiency can provide for more task(s) to be completed for a given energy' budget (e.g., a larger quantity' of tasks, more complex tasks, the same task but with more accuracy or precision, etc.).

[0061] In this manner, for instance, the improved energy efficiency of example implementations of the present disclosure can reduce an amount of pollution or other waste associated with implementing machine-learned models and systems, thereby advancing the field of machine-learning and artificial intelligence as a whole. The amount of pollution can be reduced in toto (e g., an absolute magnitude thereof) or on a normalized basis (e.g., energy per task, per model size, etc.). For example, an amount of CO2 released (e.g., by a power source) in association with training and execution of machine-learned models can be reduced by implementing more energy-efficient training or inference operations. An amount of heat pollution in an environment (e.g., by the processors / storage locations) can be reduced by implementing more energy-efficient training or inference operations.

[0062] Various example implementations are described herein with respect to the accompanying Figures.

[0063] Figure 1 is a block diagram of an example system for using multiple tool paths to perform operations for processing a request. A request 102 can describe a task to be performed. Based on request 102, tool control system(s) 104 can provide an input to machine-learned sequence processing model(s) 106 to cause machine-learned sequence processing model(s) 106 to generate a plurality of outputs that contain instructions for tool calls to perform step(s) in fulfillment of request 102. Tool control system(s) 104 can parse the output of machine-learned sequence processing model(s) 106 to obtain the tool calls. Tool control system(s) 104 can pass the tool calls to tool execution system(s) 108. Each tool call can be executed independently in distinct tool path(s). For instance, N tool calls can be executed to initiate N tool paths 110-1, 110-2, . . . , 110-N. Output values generated by the respective tool paths can be passed back to tool control system(s) 104 for evaluation. This cycle can be iteratively repeated: based on an intermediate output from a tool path, tool control system(s) 104 can generate new tool call(s) to perform a second step of processing. Once tool control system(s) 104 determines a completion of processing sufficient to fulfill request 102 (or determine an inability to fulfill request 102), tool control system(s) 104 can return response 112.

[0064] Request 102 can include a variety of data types and formats. For example, request 102 can contain a textual description of a task. Request 102 can include structured text (e.g., JSON) or unstructured text (e.g., freeform natural language). Request 102 can include image data, audio data, video data, multidimensional data arrays (e.g., point cloud), etc.

[0065] Request 102 can be generated by various sources. For instance, a user can input a request directly through a user interface (e.g., graphical user interface, command line user interface, audio-based user interface, haptic user interface, etc.). In an example, request 102 can include natural language text data transcribed from an input audio signal from a microphone of a user device. Request 102 can be automatically generated by another system or service that requires the execution of a task. Request 102 can be generated by an upstream generative machine-learned model that generates request 102 to initiate performance of a subtask of a larger task.

[0066] Request 102 can also contain metadata that provides additional context or instructions for the task to be performed. This metadata can include priority levels, which can be used to schedule and manage the execution of tasks, or it could specify user preferences, such as the desired output format or quality settings for a conversion task. Security credentials, such as tokens or keys, can also be part of request 102, allowing the system to authenticate the request and access protected resources or tools used for task completion. For example, some tools can provide access to protected resources (e.g., calendar applications, mail applications, etc.). Request 102 can include credentials or a token that can be passed to the tool to authorize execution of the tool to perform the requested task. This information can be in the form of API keys, OAuth tokens, or user credentials, for example.

[0067] Request 102 can be received and processed in real time. Request 102 can be retrieved from a cache or queue for offline processing.

[0068] Request 102 can include configuration parameters to configure fulfillment of the request. Example configuration parameters can be used to configure an operation of tool control system(s) 104, machine-learned model (s) 106. etc. Example configuration parameters can include a listing of tools that can be used in fulfillment of the request. Tool control system 104 can maintain a list of available tools that can be used by the model, or tool control system 104 can use tools that are identified in request 102.

[0069] Tool control system(s) 104 can include any computing system that can use machine-learned model 106 to control operations of one or more tools. Tool control system(s) 104 can host and execute machine-learned model 106. Tool control system(s) 104 can interact with machine-learned model(s) 106 that execute on different external system(s).

[0070] Tool control system(s) 104 can execute processing logic configured to receive request 102 and generate inputs for machine-learned model (s) 106 to perform processing steps in fulfillment of request 102. Tool control system(s) 104 can pass request 102 directly to machine-learned model(s) 106. Tool control system(s) 104 can parse request 102 to extractcontent for input to machine-learned model(s) 106 and generate an input data structure configured for processing by machine-learned model(s) 106.

[0071] For example, tool control system(s) 104 can receive request 102 via an API in a format defined for communication via the API. Some portion of request 102 can include content for input to machine-learned model(s) 106. Some portion of request 102 can include content for communication with tool control system(s) 104 (e.g., configuration parameters, authentication tokens, etc.). Tool control system(s) 104 can generate an input structure for input to machine-learned model 106 based on the content included in request 102 for input to machine-learned model(s) 106.

[0072] Tool control system(s) 104 can preprocess request 102 to identify one or more tasks or subtasks to be performed in fulfillment of request 102 and generate input(s) to machine-learned model(s) 106 to perform one or more of the tasks or subtasks. Tool control system(s) 104 can, for example, use a machine-learned model (e.g., machine-learned model(s) 106) to process request 102 to determine what tasks need to be performed. For instance, the model can return a list of tasks. Tool control system(s) 104 can iterate through the list of tasks, instructing machine-1 earned model 106 to perform each task in turn.

[0073] Tool control system(s) 104 can provide deterministic logic that frames the inputs to and outputs from a probabilistic machine-learned model. The deterministic logic can include structured workflows. For example, different workflows can be configured for different kinds of requests. For instance, a request relating to trip planning can trigger a workflow that includes a predetermined list of steps or stages, such as opening a calendar application, opening a mapping application, etc. The deterministic logic can include output validations. For instance, a JSON validation mode can implement programming logic configured to require the model to only generate valid JSON. For instance, tool control system(s) 104 can monitor the generations of model(s) 106 and constrain output samples to those that adhere to valid JSON syntax (e.g., by prescriptively constraining an output space or by resampling any invalid outputs until a valid sample is obtained). Other output validations can be used to deterministically constrain the operation of the tool to adhere to predefined formats, vocabularies, styles, etc.

[0074] Tool control system(s) 104 can include deterministic logic to constrain tool calls generated by model(s) 106 to valid tool calls. Valid tool calls can include calls to actual, active tools that can operably be called using the commands generated by model 106. Valid tool calls can include calls that contain flags, parameters, or other inputs that conform to a set of defined flags, parameters, or other inputs. Validating tool calls in this manner may notnecessarily cause the call to perform the desired function specified in request 102, but a valid tool call may be less likely to generate errors. Tool control system(s) 104 can detect generation of an output portion that does not conform to a valid tool call (e.g., generation of a flag that is not a defined flag associated with given tool) and initiate resampling of that output portion. Tool control system(s) 104 can manipulate one or more output layers of model 106 to cause the output to be sampled only from a subset of the output vocabulary that would result in a valid tool call for a given inference step.

[0075] Tool control system(s) 104 can include an interface to machine-learned sequence processing model(s) 106. This interface can be designed to handle the model’s input and output formats. For example, tool control system(s) 104 can format request 102 into a representation that is compatible with the machine-learned model and then interpret the model’s output into tool calls. For instance, machine-learned model(s) 106 can output textual content that may not be directly executable in its raw state. Tool control system(s) 104 can parse the output textual content to extract executable tool calls and initiate execution of the tool calls.

[0076] Tool control system(s) 104 can include a scheduling and orchestration layer that manages the execution order and dependencies of the tool calls. This layer can ensure that tool calls are executed in the correct sequence and manage the flow of data between tool calls when there are dependencies or when output from one tool is the input to another. For example, tool control system(s) 104 can identify any requisite packages to install in order to execute a tool called for in a tool call. Tool control system(s) 104 can install or update the packages. Tool control system(s) 104 can alternatively compile a list of available tools and their input schemas and append (or prepend) the list to content from request 102 in the input to machine-learned model(s) 106.

[0077] Tool control system(s) 104 can include monitoring and logging capabilities to track the status and performance of tool calls. These capabilities can assist in debugging, performance optimization, and compliance with audit requirements. For instance, tool control system(s) 104 can log the start time, end time, and outcome of each tool call, along with any errors or warnings that occurred during execution.

[0078] Tool control system(s) 104 can include a failure management system that handles errors and exceptions arising from tool calls. This system can implement retry logic, alternative path execution, or error notifications to ensure robustness in the face of tool failures or unexpected results. For example, if a tool call fails due to a network timeout, the tool control system(s) 104 can automatically retry the call or route the task to a differentnetwork endpoint. If a tool returns an error message, tool control system(s) 104 can automatically request a new tool call from model 106.

[0079] Tool control system(s) 104 can include a resource allocation subsystem that manages the distribution of computational resources among different tool paths. This subsy stem can assign CPU time, memory7, and I / O bandwidth to tool paths in a way that improves overall system throughput and ensures fair resource sharing. For example, tool control system(s) 104 could dynamically allocate more resources to known high-resource operations.

[0080] Tool control system(s) 104 can be configured to build a context history of actions in a tool path. This context history can record tool calls output by model(s) 106 and any outputs or error messages received in response to execution of the tool calls. This context history can be input to model 106 with each new generation to condition the output of model 106 to leam from prior actions, to be aware of the current state of progress, etc. In an example, the context history can be formatted as a dialog or chat sequence of input messages and response messages. In an example, the context history7can have other suitable formats, such as a transcript, record, JSON structure, etc.

[0081] Machine-learned sequence processing model(s) 106 can various different ty pes of machine-learned models configured to process input sequences and generate predictive output sequences. The models can be based on neural network architectures such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs). transformers, etc.

[0082] Machine-learned sequence processing model(s) 106 can include pre-trained components that have been trained on large corpora of text or other data (e.g., in an unsupervised fashion). Machine-learned sequence processing model(s) 106 can include components that have been fine-tuned using a curated dataset that is more closely aligned with the types of tasks the system is expected to perform. For instance, a model could be pretrained on general text corpora and then fine-tuned with curated textual examples of generating tool calls in response to various inputs.

[0083] Machine-learned sequence processing model(s) 106 can use ensemble methods that combine the predictions of multiple models. Machine-learned sequence processing model(s) 106 can use a mixture of experts technique that routes different requests to different sub-models that specialize in various domains. For example, one model might be more accurate at generating file manipulation commands, while another might excel atnetwork-related commands. Using multiple such models can allow machine-learned model(s) 106 to handle a broader range of tasks more effectively.

[0084] Machine-learned sequence processing model(s) 106 can receive updates from a feedback mechanism that allows the system to leam from the outcomes of executed tool paths. This mechanism can use the results of previous tool executions to update the model's parameters. For example, if a certain sequence of tool calls consistently leads to successful outcomes, the model can be updated (e.g.. by adjusting its weights) to favor similar sequences in the future.

[0085] Machine-learned sequence processing model(s) 106 can be configured to process various different kinds of data. Machine-learned sequence processing model (s) 106 can be configured to generate textual commands based on textual and image-based or audiobased inputs. Machine-learned sequence processing model(s) 106 can be configured to generate textual commands based on text or other symbolic inputs alone.

[0086] Machine-learned sequence processing model(s) 106 can output commands for execution to perform at least a step associated with request 102. Machine-learned sequence processing model(s) 106 can text-based commands. For instance, to perform a task or subtask relating to moving a file within a Linux environment, machine-learned sequence processing model(s) 106 can output a string containing a “mv” command. The string can include flags and values associated with the flags. The string can include a name of a target file (e.g., “file.txt”) and a path to a destination folder (e.g., “- / documents7’).

[0087] Machine-learned sequence processing model(s) 106 can output GUI-based commands. For instance, to perform a task of exporting a rendered media file from an application that does not expose a programmatic API, machine-learned sequence processing model(s) 106 can output a string containing a description of actions to be performed using the GUI: e.g., “select, from the ‘File’ menu, the ‘Export’ action.” The string can contain a reference to a GUI-based navigation tool to cause the description of actions to be performed by the GUI-based navigation tool.

[0088] Example GUI-based navigation tools can include machine-learned models and machine-learning techniques that can interact with graphical user interfaces (GUIs) to perform tasks. Example GUI-based navigation tools can include systems that can interact with GUIs using the same modalities as human users: the system can process a rendering of the GUI and output system-level input commands (e.g., taps, clicks, etc.). This can enable automated interactions with substantially any GUI, without requiring custom application programming interfaces for each application. An example system can ingest a naturallanguage instruction and an image-based rendering of a GUI and generate a command for following instruction. Example GUI-based navigation tools can include a pre-trained, sequence-based image processing model (e.g., a model including a vision transformer model) that is adapted to receive instructions for actions to perform with a GUI and generate low- level GUI interaction commands for interacting with a GUI (e.g., click, tap, select, drag, input text, etc.).

[0089] Machine-learned sequence processing model(s) 106 can generate tool calls individually or in groups. For instance, machine-learned sequence processing model(s) 106 can generate a list of tool calls to execute in sequence, and tool control system(s) 104 can proceed to sequentially execute the steps of the sequence. Machine-learned sequence processing model(s) 106 can proceed to instruct each step of a sequence of operations in a tum-by-tum fashion, with each output of a tool responsive to a tool call being fed back to machine-learned sequence processing model (s) 106 with a request for a next step to perform.

[0090] Machine-learned sequence processing model(s) 106 can proceed to instruct each step of a sequence of operations in a tum-by-tum fashion, with a subset of turns used for invoking a tool and a subset of turns obtaining information. In some implementations, decoding can be implemented differently for each subset. For example, the subsets corresponding to tool invocations can be resampled to obtain a plurality of different tool calls (that can be executed in different tool paths). The subsets corresponding to intervening data collection or delivery can be decoded with greedy sampling. This can reduce a level of branching in possible different tool paths while affording a broad array of diverse trials.

[0091] Resampling of tools mid-sequence can be traversed by a depth-first strategy7or a breadth-first strategy. In a depth-first strategy, a tool step can be resampled. The entire trace following from that tool step can be resampled from there for one or multiple times. In a breadth-first strategy, a tool step can be resampled. The sequence up to a next tool step can be resampled for one or multiple times. The first partial trace can be popped up to the next tool step and repeated. A stopping condition can be the first N finished traces.

[0092] In some cases, a subset of turns can correspond to reasoning steps (e.g., chain- of-thought reasoning). These subsets can be sampled with greedy sampling (for efficiency) or with additional resampling to generate additional branching (for diversity).

[0093] Machine-learned sequence processing model(s) 106 can generate outputs in batches. For instance, a batch dimension of an input structure or output structure can provide for contemporaneous usage of a given portion of the model (e.g., a set of weights, a set of cached activations, etc.) to perform inference over multiple input sequences.

[0094] Machine-learned sequence processing model(s) 106 can be configured to generate multiple output sequences distributed along a batch dimension for the same input sequence. For instance, an input sequence can be an initial input generated based on request 102. To generate a plurality of different sampled outputs, machine-learned sequence processing model(s) 106 can autoregressively sample different output values to append to the input sequence. Different sampled values can be appended to different output positions along the batch dimension, such that each different output can be autoregressively generated in parallel. A toy example is illustrated in the following table.Batch Input Output Step 1 Output Step 2 Output Step 31 Write a story: Once upon a2 Write a story: It was raining3 Write a story7: Long ago

[0095] For attention-based models, the computed attention values (e.g.. a KV cache) over the input sequence can be shared over the batches. The computed attention values can be duplicated over the batch dimension. Pointers to the computed attention values can be duplicated over the batch dimension that point to the same computed attention values (e.g., to avoid the memory cost of duplication of the values).

[0096] This technique can be applied to generate a plurality of different tool calls for performing a task requested in an initial input. Furthermore, this technique can be applied in a branching manner: for step of a multi-step tool path, a plurality of next steps can be generated by machine-learned model 106.

[0097] A tool path can be associated with one or more “rows’7or batches. As a tool path is pruned, restarted, or edited, the work generated to that point can be re-used as appropriate. For instance, if only one tool call in the tool path fails, then the sequence state can be rolled back to before that tool call was added, and output generations of a new tool call can proceed from that point. Prior computed attention values can be retained and used for the portion of the sequence that remains.

[0098] Machine-learned sequence processing model(s) 106 can output an output data structure that contains a tool call. Tool control system(s) 104 can parse the output data structure to extract the tool call.

[0099] In an example, machine-learned sequence processing model(s) 106 can be a language processing model and be prompted to generate a tool call. For example, a prompt can provide an example tool call, including a list of tools that can be called.

[0100] A tool call can include a specific command or set of instructions that initiates a single operation or a series of operations within the tool path. These instructions can be formatted as command-line arguments, function calls within a script, or API requests to a web service, as some examples, depending on the nature of the tool being controlled. For instance, a tool call might be a command to start a data analysis tool with certain parameters, such as the location of the input data and the type of analysis to perform.

[0101] A tool call can include parameters and flags that modify the behavior of the tool. These parameters can dictate aspects like the input file paths, output formats, level of detail in the results, or any other configurable options that the tool supports. For example, a tool call to a file compression utility might include flags to set the compression level or to preserve the file’s original creation date.

[0102] An example tool call can include a call to a web search tool. Example parameters for the web search tool can include a number of responses. Example parameters can include a technique for returning results, such as top-K, random sampling, etc. Different sampled outputs can correspond to resampled search queries, different numbers of responses, etc. In this manner, for instance, a plurality of sampled tool calls can generate a plurality of different lists of search results.

[0103] A tool call can include environmental variables or context that is used for the tool to operate. This context can provide the tool with information about its runtime environment, such as the current user, system paths, or network settings. For instance, a tool call might include a database connection string that the tool needs to access a database, or it might include credentials for authenticating with a cloud service.

[0104] A tool call can include dependency information that ensures all required resources are available for the tool. This might involve specify ing the versions of libraries or other tools that the current tool depends on, or it might require the tool execution system(s) 108 to set up certain services before the tool can run. For example, a tool call might specify that a certain Python package must be installed, or it might require a virtual environment to be activated before the tool is executed.

[0105] A tool call can include traceability or logging options that assist in monitoring the execution of the tool and diagnosing issues. These options can direct the tool execution system(s) 108 to capture detailed logs of the tool’s activity. For example, a tool call mightenable verbose logging for a complex operation, ensuring that detailed information is available if the operation does not produce the expected results. This information can be returned in an update so that tool control system 104 (e.g., using model 106) can identify the error and generate a new tool call.

[0106] A tool call can include a unique identifier or transaction ID that links it to a specific instance of a request within the tool execution system(s) 108. This identifier can be used to track the progress of the tool call through the various stages of a tool path and to correlate it with other related operations. For example, each tool call in a multi-step process might carry the same transaction ID to signify' that they are part of the same overall task.

[0107] Tool control system(s) 104 can pass the tool call to tool execution system(s) 108 for execution.

[0108] Tool execution system(s) 108 can be implemented by computing device(s) hosting a runtime environment where the tool calls generated by machine-learned sequence processing model(s) 106 are executed. This environment can be configured to support a variety of software tools, from command-line utilities to full-fledged applications, and can be tailored to provide the necessary computational resources, such as CPU, GPU, memory, I / O. and disk space. For example, tool execution system(s) 108 can include containerized environments for each tool path. Tool execution system(s) 108 can include separate computing devices or systems (real or virtual) for each tool path. Tool execution syslem(s) 108 can execute multiple tool paths on the same system but in separate or sandboxed filesystem sections (e g., different relative root paths).

[0109] Tool execution system(s) 108 can include a dispatcher that allocates tool calls to the appropriate execution environments. This dispatcher can manage a queue of tool calls and assign them to execution slots based on availability and priority. For instance, tool execution system(s) 108 might prioritize urgent tool calls by moving them to the front of the queue and executing them on dedicated high-performance hardware.

[0110] Tool execution system(s) 108 can include monitoring and logging tools that capture the execution details of each tool call. These tools can track the start time, duration, exit status, and resource usage of each call. For example, the logs could be used to identify bottlenecks in the tool execution process or to verify that a particular tool call was executed as intended.[Oi l 1] Tool execution system(s) 108 can include failover and redundancy mechanisms that ensure high availability and reliability of the tool execution process. These mechanisms can automatically reroute tool calls to backup systems or retry failed executions.For instance, if an individual execution environment hosting a particular tool fails, tool execution system(s) 108 could seamlessly transfer the tool calls to an alternate environment without disrupting the overall task flow or requiring a new generation command from tool control system(s) 104.

[0112] Tool execution system(s) 108 can include a resource management subsystem that dynamically adjusts the allocation of computational resources to each tool call or each execution environment. This subsystem can scale resources up or down in response to the demands of the tool calls, optimizing the system’s overall resource utilization. For example, a tool call that requires significant processing power could be allocated additional CPU cores dynamically as needed.

[0113] Tool execution system(s) 108 can include a security layer that enforces strict access controls and isolates the execution of tool calls to prevent interference between separate tool paths. This layer can include mechanisms for setting and releasing read and write access to shared resources, system resources, etc.

[0114] Tool execution system(s) 108 can include an interface for integrating with external services and APIs, enabling the tool calls to interact with a range of external systems and platforms. This interface can support various communication protocols and data formats. For instance, tool execution system(s) 108 can include connectors for cloud storage services or other web-based APIs, enabling tool calls to store, retrieve, or submit data for these services as part of the task execution.

[0115] Tool execution system(s) 108 use separate root directories for each tool path as a method of isolation, ensuring that the execution of one tool path does not interfere with or compromise the integrity of another. Each root directory can act as a standalone environment with its own set of subdirectories and files, mimicking the structure of a complete filesystem. For example, one tool path might operate within a root directory that contains a specific version of a software library, while another tool path might use a different version of the library in its own separate root directory', preventing conflicts betw een the tw o.

[0116] The use of separate root directories in tool execution system(s) 108 can facilitate a clean and controlled execution environment for each tool path, allowing for the customization of dependencies and configurations. By isolating the filesystems, the system can prevent issues such as shared state or resource contention, which can occur in environments where multiple processes have access to the same files. For instance, if one tool path requires writing temporary files to disk, having a separate root directory ensures that these files do not clash with or overwrite the temporary files of another tool path.

[0117] Separate root directories in tool execution system(s) 108 can enhance security by limiting the potential impact of a compromised tool path. If a tool path is designed to execute untrusted code or handle sensitive data, isolating it in its own root directory can prevent malicious code from affecting other tool paths or accessing system-wide resources. In an example, if machine-learned sequence processing model 106 is generating code to be executed, treating all such code as “untrusted" can help prevent erroneously generated code from damaging other areas.

[0118] Tool execution system(s) 108 can use separate root directories to simplify the management of permissions and access controls for each tool path. By assigning different user and group permissions to each root directory, the system can finely control which processes have read, write, or execute access to the files and directories within that environment. For instance, a tool path that requires elevated privileges can be restricted to a root director}' where such privileges are granted, without affecting the security posture of other tool paths. Further, to constrain operations of that tool path to its assigned directory', tool execution system(s) 108 can withhold write permissions from that tool path except for content within its designated root directory.

[0119] The use of separate root directories in tool execution system(s) 108 can make it easier to track resource usage and perform clean-up operations for each tool path. Since each tool path's files and processes can be contained within its own directory, the system can accurately monitor and limit the amount of disk space, memory, and CPU time consumed by the tool path. Additionally, once a tool path completes its execution or if it needs to be terminated, the system can quickly remove its root directory' to free up resources.

[0120] Tool execution system(s) 108 can use containers to isolate the execution of different tool paths. Containers encapsulate a tool's runtime environment — its code, runtime, system tools, libraries, and settings — distinct from other containers and the host system.

[0121] The use of containers in tool execution system(s) 108 can offer a high degree of portability for tool paths, as containers can be easily moved between different host systems without the need for environment-specific configurations. Further, the use of containers for tool execution can provide continuity from run to run. as each time a tool call is initiated, a fresh image can be loaded.

[0122] Tool execution system(s) 108 can use container orchestration tools to manage the lifecycle of containers and automate the deployment, scaling, and operation of containerized tool paths. These orchestration tools can handle tasks such as load balancing, health monitoring, and failover.

[0123] Containers in tool execution system(s) 108 can provide a sandboxed environment for each tool path. Containers can restrict the processes running within them from making unauthorized changes to the host system or accessing resources allocated to other containers.

[0124] The use of containers in tool execution system(s) 108 can simplify resource monitoring and allocation by providing tools to set resource limits and reservations for CPU, memory, and I / O for each container. This can prevent any single container from monopolizing system resources and ensure that each tool path has access to the resources it needs to perform optimally. For example, system administrators can allocate a fixed amount of memory to a container to prevent it from impacting the performance of other containers.

[0125] Tool execution system(s) 108 can benefit from the rapid and repeatability of startup and shutdown capabilities of containers. Containers can be spun up in seconds, or less, allowing for responsive task execution.

[0126] Tool execution system(s) 108 can use virtual machines (VMs) to isolate the execution of different tool paths, providing a robust layer of separation by simulating entire hardware systems on which complete operating systems can run independently. Each VM can operate with its own virtualized hardware, including CPU, memory, and storage resources, allowing for full isolation of the tool path’s execution environment.

[0127] The use of virtual machines in tool execution system(s) 108 can facilitate the management of diverse operating system environments required by different tool paths. Since VMs can run different operating systems, different OS environments that enable different toolsets can be implemented, offering increased diversify in available approaches to performing a given task.

[0128] Virtual machines in tool execution system(s) 108 can also provide a high degree of configurability and control over the computing environment. Virtualization allows for precise allocation of resources to each VM, which can be tailored to the specific needs of a tool path, including custom network configurations, storage setups, and hardware emulation. This means that each tool path can be provided with exactly the resources it needs, optimizing the use of the underlying physical hardware.

[0129] Tool execution system(s) 108 can use virtual machines to enhance security and compliance by leveraging the strong isolation properties of hypervisors, w hich are the software, firmware, or hardw are that create and run VMs. Hypervisors can enforce strict boundaries between VMs, ensuring that activities within one VM, such as processing sensitive data or running experimental code, are completely isolated from other VMs.

[0130] Tool paths 110-1. 110-2, . . . , 110-N can include a series of discrete steps or operations that are generated or instructed by tool control system(s) 104 in fulfillment of request 102. Each tool path can represent a sequence of tool calls, which can include executable commands, function calls, API requests, or any other actions that can be performed by software, firmware, or hardware tools. For example, a tool path can involve a sequence of data retrieval, data transformation, and data analysis operations.

[0131] Response 112 can include the final output generated by tool control system(s) 104 using commands generated by machine-learned model(s) 106 that were executed in tool execution system(s) 108.

[0132] Response 112 can include metadata that provides information about the execution of the request, such as execution time, resources used, or a log of the steps performed. This metadata can be useful for auditing purposes, performance monitoring, or providing insights into the processing of the request. For instance, response 112 might contain details about the completion status of each tool path, including any errors encountered or corrective actions taken. Response 112 can include a confidence score or a measure of accuracy associated with the returned results.

[0133] Response 1 12 can include a link or reference to a location at which the output or results can be accessed, especially if they are too large to be directly included in the response or if they are stored in a cloud service or database. This can be useful for tasks that generate large datasets or files, allowing users to download or interact with the results at their convenience. For example, response 1 12 might provide a URL to a secure location where the user can download the processed files.

[0134] Response 112 can be a direct output from at least one of tool paths 110-1, 110- 2, . . . , 110-N. For example, request 102 can be a data retrieval and summarization task. Response 112 can be the requested summary that was generated using a tool path of the plurality of tool paths.

[0135] Response 112 can be a composite of outputs obtained from multiple tool paths. Individual results from tool paths 110-1, 110-2. . . . , 110-N can be aggregated, compared, or synthesized to form a cohesive final output. This composite output can leverage the diversity of the tool paths to improve the accuracy and reliability of the response by corroborating similar results or by combining different perspectives on the task. For instance, if multiple tool paths were used to analyze a dataset from different angles, response 112 could integrate these analyses into a comprehensive report that offers a more complete understanding of the data. This approach can also be used to cross-validate results.

[0136] In an example, multiple tool paths can perform a data retrieval and summarization task. The outputs from each tool path can be input to a machine-learned model (e.g., model 106) to generate a summary that draws on all the individual outputs.

[0137] Response 1 12 can be selected using a consistency metric as described herein. The outputs can be compared using a consistency metric to evaluate whether there is majority7or plurality alignment on features of the output. For instance, a consistency metric can be evaluated across a sampling of diverse tool paths (e.g., representing diverse approaches to resolving the query ) and corresponding outputs. For example, a set of outputs with diverse tool execution strategies can be polled to obtain a majority7or plurality7“vote” on the ultimate output. In this manner, the tool path outputs can self-corroborate to improve the robustness of model output and improve accuracy of the ultimate responses.

[0138] Figure 2 is a block diagram of an example tool path 200. Tool path 200 can include a sequence of tool executions and responses that are iteratively managed to perform a task or subtask. Tool path can be an example of one of tool paths 110-1, 110-2, . . . , 110-N. Tool control system(s) 104 can issue a tool call 202. Tool execution 204 can be carried out by tool execution system(s) 108 to produce tool response 206. Tool response 206 can include an output of the tool, an error message produced by tool execution system(s) 108 upon failure to execute the tool, etc. Tool execution system(s) 108 can return an update 208 based on tool response 206. Tool control system(s) 104 can provide another tool call 210 to perform next using any output(s) of the prior tool execution 204. Tool call 210 can be executed at tool execution 212 to generate a tool response 214. Tool execution system(s) 108 can return an update 216 based on tool response 214. Tool control system(s) 104 can provide another tool call 218 to perform next using any output(s) of the prior tool execution 212. Tool call 218 can be executed at tool execution 220. This sequence can be repeated until an update returned to tool control system 104 is determined to indicate completion of the task or subtask requested in request 102, or until another stopping condition (e.g., detection of a failure condition).

[0139] Tool call 202 can include a specific command or set of instructions that initiates a single operation or a series of operations within the tool path 200. These instructions can be formatted as command-line arguments, function calls within a script, or API requests to a web service, as some examples, depending on the nature of the tool being controlled. For instance, tool call 202 might be a command to start a data analysis tool with certain parameters, such as the location of the input data and the type of analysis to perform.

[0140] Tool call 202 can include parameters and flags that modify the behavior of the tool execution 204. These parameters can dictate aspects like the input file paths, outputformats, level of detail in the results, or any other configurable options that the tool supports. For example, a tool call to a file compression utility might include flags to set the compression level or to preserve the file’s original creation date.

[0141] Tool call 202 can include environmental variables or context that is used for the tool execution 204 to operate. This context can provide the tool with information about its runtime environment, such as the current user, system paths, or network settings. For instance, tool call 202 might include a database connection string that the tool needs to access a database, or it might include credentials for authenticating with a cloud service.

[0142] Tool call 202 can include dependency information that ensures all required resources are available for the tool execution 204. This might involve specifying the versions of libraries or other tools that the current tool depends on. or it might require the tool execution system(s) 108 to set up certain services before the tool can run. For example, tool call 202 might specify that a certain Python package must be installed, or it might require a virtual environment to be activated before the tool is executed.

[0143] Tool call 202 can include traceability or logging options that assist in monitoring the execution of the tool and diagnosing issues. These options can direct the tool execution system(s) 108 to capture detailed logs of the tool’s activity. For example, tool call 202 might enable verbose logging for a complex operation, ensuring that detailed information is available if the operation does not produce the expected results. This information can be returned in an update so that tool control system 104 (e.g., using model 106) can identify the error and generate a new tool call.

[0144] Tool call 202 can include a unique identifier or transaction ID that links it to a specific instance of a request within the tool execution system(s) 108. This identifier can be used to track the progress of the tool call through the various stages of tool path 200 and to correlate it with other related operations. For example, each tool call in a multi-step process might carry the same transaction ID to signify7that they are part of the same overall task.

[0145] Tool calls 210 and 218 can have various similar attributes to tool call 202.

[0146] Tool execution 204 can include the process of running the specific commands or instructions defined in tool call 202 within the operational environment provided by tool execution system(s) 108. This execution can involve invoking a software application, script, or utility and passing the necessary parameters and data to it. For example, tool execution 204 might involve running a data processing script that takes raw data as input and produces a cleaned dataset as output.

[0147] Tool execution 204 can include the allocation of necessary system resources, such as CPU, memory, and disk space, to ensure the tool has the required capacity to perform its function. This allocation can be dynamically managed by tool execution system(s) 108 based on the needs of the tool and the availability of resources. For instance, a tool that performs resource-intensive computations may be allocated additional processing power during its execution phase.

[0148] Tool execution 204 can include the monitoring of the tooFs performance and health during its operation, capturing metrics such as execution time, resource usage, and throughput. This monitoring can help in identifying any performance bottlenecks or failures that occur during the execution of the tool. For instance, if a tool unexpectedly consumes more memory than anticipated, tool execution 204 might include mechanisms to log this event and take corrective action.

[0149] Tool execution 204 can include the handling of input and output data streams for the tool being executed. This can involve reading data from files, databases, or network sources, and writing output data to the appropriate destinations. For example, tool execution 204 might read input data from a secure cloud storage service and write the processed results to a local file system.

[0150] Tool execution 204 can include the execution of multiple instances of a tool in parallel, particularly when the tool path is part of a distributed system designed for high throughput or redundancy. This parallelism can help in achieving faster processing times and providing fault tolerance. For instance, tool execution 204 may involve running several instances of a data analysis tool across different nodes in a computing cluster.

[0151] Tool execution 204 can include the integration with external services or APIs that provide additional functionality required by the tool. This can involve sending requests to web services, invoking cloud-based functions, or interacting with third-party software through their exposed interfaces. For example, tool execution 204 might include calls to an external API (e.g., external to tool execution system(s) 108).

[0152] Tool executions 212 and 220 can have various similar attributes to tool execution 204.

[0153] Tool response 206 can include the direct output generated by the execution of tool call 202. For example, if tool call 202 instructed a database query, tool response 206 might include the retrieved records or a confirmation of the successful execution of the query and a pointer to the location of the results.

[0154] Tool response 206 can include error messages or diagnostic information produced by tool execution system(s) 108 if the execution of tool call 202 encounters issues. This information can allow tool control system(s) 104 to make informed decisions about how to proceed, whether by ret tying the operation, executing an alternative tool call, or aborting the process by pruning the tool path entirely. For instance, if a network operation failed, tool response 206 could include details about the nature of the network error.

[0155] Tool response 206 can include metadata about the execution of the tool call, such as execution time, system resource usage, or version information of the tool used. For example, tool response 206 might detail the amount of CPU and memory used during the execution, which could be used for optimizing resource allocation in future executions.

[0156] Tool response 206 can include partial results or intermediate data if the tool execution is part of a larger multi-step process within the tool path. These partial results can be used as input for subsequent tool calls, enabling the iterative processing of data or the progressive refinement of results. Tool response 206 can be streamed or computed in toto.

[0157] Tool response 206 can include a quality or confidence score when the tool execution involves predictive models or heuristic evaluations. This score can provide insight into the reliability of the output and can be used to determine whether additional verification or validation is necessary. For example, in a machine learning classification task, tool response 206 might include a confidence score for each class prediction, indicating the model’s certainty in its classification. Tool control system(s) 104 can use the confidence score for evaluating a viability of continuing usage of tool path 200. For instance, tool path 200 can include usage of a probabilistic tool. Another tool path can use a different tool. Based on a confidence score output by the probabilistic tool, tool control system(s) 104 can determine whether to rely on the probabilistic tool output for further computation in tool path 200 or whether to prune tool path 200.

[0158] Tool response 214 can have various similar attributes to tool response 206.

[0159] Update 208 can include data generated by tool call 202 (e.g., tool response206). Update 208 can include status information regarding the execution of tool call 202. This status information can provide insights into whether the tool call was successfully executed, encountered warnings, or resulted in errors. For example, update 208 may contain execution status codes, such as “success” or “error,” along with associated messages that detail the nature of the execution outcome. In scenarios where tool call 202 is part of a batch process or a pipeline, update 208 can also include progress indicators, such as a percentage completionor a step number, to inform the tool control system(s) 104 of the current stage within the tool path.

[0160] Update 208 can include diagnostic data generated during tool execution 204. This diagnostic data can help in troubleshooting issues that arise during the execution of tool call 202. For instance, update 208 might include log files, stack traces, or memory7dumps that were captured if an error occurred. Additionally, update 208 can provide performance metrics such as execution time, memory usage, and CPU load, which can be used to optimize future executions of similar tool calls. In cases where tool execution 204 interacts with external systems, update 208 might also include network diagnostics, such as response times and error codes from those systems.

[0161] Update 208 can include output data or results from tool execution 204. Depending on the nature of the tool and the task it performs, the output data can vary7widely. For example, if the tool call 202 involves data processing, update 208 may contain the processed data or a summary7thereof. If the tool call relates to a query7or a search operation, update 208 might include the retrieved information or a reference to where such information can be accessed. In the context of computational tasks, update 208 could also include the results of calculations or simulations. The format of the output data in update 208 can be structured, such as JSON or XML, or unstructured, such as plain text or binary7data.

[0162] Update 208 can be parsed and interpreted by tool control system 104 to determine the next steps in the tool path. Tool control system 104 can pass update 208 to machined earned sequence processing model 106 and request a next action. If the update indicates a successful execution with valid results, machine-learned sequence processing model 106 can generate a next tool call in the sequence to continue the task progression. In cases where update 208 includes error messages, machine-learned sequence processing model 106 can use this data to identify and troubleshoot issues. For instance, if a tool call results in an error due to missing dependencies, machine-learned sequence processing model 106 can generate instructions to perform corrective actions such as installing the necessarydependencies. Alternatively, if the error is non-recoverable. machine-learned sequence processing model 106 can generate instructions to prune the current tool path and instead initiate a new tool path using a new tool call sampled from machine-learned sequence processing model 106. By providing machine-learned model(s) 106 with intermediate states, machine-learned sequence processing model 106 can generate more accurate and context- aware tool calls that build upon the previous steps, moving the tool path closer to the completion of the task.

[0163] Update 216 can have various similar attributes to update 208.

[0164] Tool path 200 can use data persisted from execution to execution. Tool execution system 108 can employ various data persistence methods to provide that data is retained and made available across different tool executions. One method for persisting data is through the use of temporary' files or directories created on the file system. The output from a tool execution can be written to a file, which is then read by the next tool in the sequence. For instance, a tool that processes images might save the processed image as a file, which can then be accessed by a subsequent tool that performs image recognition.

[0165] Another method involves passing data directly between tool executions using in-memory data structures. When the tools are executed w ithin the same runtime environment or process, data can be stored in variables, arrays, or other data structures that are accessible to subsequent tool calls without the need for reading and writing to disk.

[0166] Databases or data caches can also be used to persist data across tool executions. After a tool completes its execution, the output can be inserted into a database table or stored in a cache. Subsequent tool calls can then query the database or access the cache to retrieve the necessary data.

[0167] Tool paths can be linear. For instance, after initially sampling a plurality of tool calls from an output of machine-learned sequence processing model 106, subsequent tool calls for the tool path can be a single tool call. For instance, tool path 200 contains a linear sequence of tool calls.

[0168] Figures 3A to 3C provides an example illustration of a branching tool path tree 300. Figure 3A illustrates an initial tool call 302 and tool execution 304. The tool response 306 can be returned to tool control systems 104. Tool control systems 104 can process tool response 306 using machine learned sequence processing models 106 to determine a next step.

[0169] Figure 3B illustrates that multiple options can be pursued for evaluating a next step. A tool call 308 can be executed at tool execution 310. Tool execution 310 can be contained or otherwise isolated from a remainder of the tool path tree 300. Tool execution 310 can generate a tool response 312.

[0170] Independently of tool execution 310, tool control system(s) 104 can output another tool call 314 that provides an alternative path branching from tool execution 304. Tool call 314 can be executed at tool execution 316 to generate tool response 318. Tool control system(s) 104 can evaluate both tool response 312 and tool response 318 to determinehow to proceed. For instance, input sequences that respectively include tool response 312 and 318 can be processed by machine-learned model 106 to generate next steps.

[0171] Figure 3C illustrates an implementation of next steps based on tool response 312 and 318. Tool control system 104 can determine to prune the branch including tool execution 310 (e.g., based on determining a failure condition based on tool response 312). Tool control system 104 can determine to initiate a new branch using tool call 320 to process tool response 306 at tool execution 322. Tool control system 104 can further initiate processing of tool response 318 using tool call 324 executed at tool execution 326.

[0172] Tool path tree 300 can further branch. Tool path tree 300 can maintain up to a predetermined maximum number of branches.

[0173] Figure 4 is an example validator 400. Tool control system(s) 104 can compute a consistency metric using validator 400. Validator 400 can process tool path states 402-1, 402-2, and 402-N. Validator 400 can process tool path states 402-1, 402-2, and 402-N in view of request 102. Validator 400 can include one or more probabilistic evaluators 404 and one or more deterministic evaluators 406. Validator 400 can process tool path states 402-1, 402-2, and 402-N to generate an evaluation 408. Evaluation 408 can include a selection of an output for response 112 based on one or more selected tool path states.

[0174] A tool path state can include a tool response or an update generated based thereon. Tool path states 402-1, 402-2, and 402-N can each be drawn from different tool paths or different branches of a same tool path tree.

[0175] A tool path state can include an output of a tool or information returned from a tool. A tool path state can be a completion state (e.g., marking the terminus of a tool path, after which no further tool calls are needed) or an intermediate state (e.g., providing an intermediate result or retrieval).

[0176] Probabilistic evaluators 404 can include one or more machine-learned models that can evaluate a quality of a tool path state. Probabilistic evaluators 404 can include one or more machine-learned embedding models configured to embed tool path states into vector representations that can be quantifiably compared to determine similarity. For example, a vector-based similarity measure can be used to determine agreement or alignment between tool path states. For example, a tool path state can include image data, and embeddings representing the images can be used to compare the semantic content of the images. A consistency metric can be based on a degree of similarity or alignment between the image embeddings. The embeddings can also be used to determine whether the content of the images align with the requested content identified in request 102.

[0177] For instance, a consistency metric for evaluating the quality of each output state can be determined using a machine-learned model. The machine-learned model can either be the same model that was used to generate the plurality of tool paths (e.g., machine- learned sequence processing model(s) 106) or a different model specifically trained or adapted for the evaluation task. For instance, the same machine-learned model that generated the tool paths might also be capable of assessing the quality of text output states. This could be due to the model’s training on a large corpus of text data, enabling it to understand the context and relevance of the outputs. Alternatively, a different machine-learned model could be employed for evaluation if it has been fine-tuned on a dataset that is more closely aligned with the evaluation criteria, such as a dataset of expert-validated text outputs. The evaluation of output states can involve various techniques, including but not limited to, semantic analysis, syntactic comparison, or statistical assessment. For example, the machine-learned model can compute vector-based representations of the text outputs and use similarity measures, such as cosine similarity, to evaluate the quality' of the outputs. In another example, the model could apply classification techniques to categorize the outputs based on predefined quality tiers, with each tier representing a range of quality scores.

[0178] Probabilistic evaluators 404 can include a machine-learned model that is configured to receive an input describing multiple tool path states. Validator 400 can use the machine-learned model to evaluate the quality of each tool path state by processing the plurality of output states together to output a preferred state. A final response (e.g.. response 1 12) or a next intermediate state can be selected using the preferred state. The preferred output state can be one that the model identifies as the most accurate, complete, or reliable based on the training it has received, which may include patterns of consistency, correctness, or other quality indicators relevant to the task at hand. For example, if the task involves generating textual summaries from multiple documents, the machine-learned model can process the various summaries (output states) to identify which one most closely captures the information of the documents with the least amount of noise or irrelevant data. Similarly, for image processing tasks, the model can compare different processed images (output states) to determine which image has the qualities that match the desired output critena.

[0179] Once the machine-learned model identifies the preferred output state, the response to the request can be selected using this state. The final output delivered as the response to the task request can be the one that the machine-learned model has evaluated to be the best among the options generated by the tool paths.

[0180] Probabilistic evaluators 404 can include one or more machine-learned models trained to identify and evaluate salient differences between states.

[0181] Probabilistic evaluators 404 can include one or more machine-learned models that are configured to identify a failure state of a tool path. Probabilistic evaluators 404 can include a classifier model that is configured to classify a response as indicating a failure condition or not indicating a failure condition.

[0182] Probabilistic evaluators 404 can include machine-learned model(s) 106. Machine-learned model(s) 106 can include a powerful multi-task model that can be trained to follow different instructions for performing different tasks. Validator 400 can perform different validation tasks by providing input data to machine-learned model 106 along with one or more instructions for a type of validation to perform. For example, an input prompt to machine-learned model(s) 106 can include data describing the tool path states and an instruction to compare the state data.

[0183] In an example, machine-learned model 106 can process an input describing a tool path state and request 102 and generate an output that predicts an indication of whether the tool path state advances the goal indicated in request 102. The input can be configured to cause the model to generate a new tool call, or generate a pruning instruction, or generate a stop condition, or generate a retry condition. For instance, the input can include textual instructions in a prompt to induce such behavior from the model.

[0184] Deterministic evaluators 406 can include one or more heuristics that can be applied to evaluate states. Heuristics can include a byte-level difference computation that quantifies a difference between two inputs. Heuristics can include a file type or file attribute check. For example, if request 102 requested an operation that results in an output GIF image, then a heuristic can include a file type match operation that matches the output file type to a file type indicated in request 102. A heuristic can include a file size or dimension comparison. Outlier file sizes or files with outlier attributes can be discarded as inconsistent with a prevailing value for the file size or other attribute. A heuristic can include other attribute comparisons. For instance, a consistency metric can be determined by evaluating how many states indicate a particular feature and determining if a majority or plurality of the states have the same feature. Each feature of multiple features can be compared in this manner.

[0185] Validator 400 can use heuristic criteria to quickly filter out output states that fail to meet basic requirements, such as correct file format or complete data fields, before applying probabilistic criteria to further evaluate the remaining outputs for quality. Thislayered evaluation process can improve the efficiency of the system by reducing the number of outputs that need to be processed by the more computationally intensive probabilistic assessment.

[0186] In general, validator 400 can identify one or more features shared by a group of states of the plurality of output states. Validator 400 can cause the system to select a final response 112 from the group (e.g.. as a representative example). The group can be obtained by clustering feature data. Selecting a response from the cluster can include determining a centroid of the cluster and selecting the response based on the centroid.

[0187] The features can be latent or patent. For instance, a latent feature can be a feature computed from the state data using a machine-learned model. A patent feature can be a feature that can be deterministically parsed or extracted from the state data.

[0188] Evaluation 408 can include a selection of data to include in a response (e.g., response 112) based on a consistency metric. Evaluation 408 can include a value for a consistency metric. Evaluation 408 can include a determination, for one or more tool path states, of a failure condition. Evaluation 408 can include a determination that an intermediate state has successfully executed and further processing is warranted.

[0189] Validator 400 can determine that one or more tool paths are redundant. Validator 400 can cause tool control system(s) 104 to prune at least one tool path based on determining a redundancy of the at least one tool path. The redundancy can be determined based on identifying a deterministic outcome of the at least one tool path. The redundancy can be determined based on a margin of similarity. The margin of similarity can be based on a minimum step size associated with a range of parameter values for a tool call.

[0190] Using evaluation 408 from validator 400, tool control system 104 can adjust at least one tool path based on determining a failure condition associated with the at least one tool path. The failure condition can be determined based on receiving a tool response that diverges from desired criteria. The failure condition can be determined based on receiving an error message indicating inoperability of a tool or tool call. Adjusting the tool path can include pruning the at least one tool path. Adjusting the tool path can include restarting the at least one tool path using one or more tool calls newly generated by the machine-learned model. Restarting at least one tool path can include generating a new tool call based on the failure condition. Generating a new tool call based on the failure condition can include inputting data describing the failure condition to the machine-learned model as context to a query for the new tool call.Example Methods

[0191] Figure 5 depicts a flowchart of an example method 500. One or more portion(s) of example method 500 can be implemented by a computing system that includes one or more computing devices such as, for example, computing systems described with reference to the other figures. Each respective portion of example method 500 can be performed by any (or any combination) of one or more computing devices. Moreover, one or more portion(s) of example method 500 can be implemented on the hardware components of the device(s) described herein, for example, to train one or more systems or models. Figure 5 depicts elements performed in a particular order for purposes of illustration and discussion. Those of ordinary' skill in the art, using the disclosures provided herein, will understand that the elements of any of the methods discussed herein can be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. Figure 5 is described with reference to elements / terms described with respect to other systems and figures for exemplary' illustrated purposes and is not meant to be limiting. One or more portions of example method 500 can be performed additionally, or alternatively, by other systems.

[0192] At 502, example method 500 can include receiving a request to perform a task. In some implementations of example method 500, the request can be request 102. In some implementations of example method 500, receiving a request to perform a task can involve obtaining an input that specifies an action or a series of actions to be completed. The request can be in various forms, such as a textual description, a voice command, an image, a set of parameters, or any combination thereof. For instance, the request might be a natural language text input, such as ‘‘convert a JPEG image to a PNG format,” or it could be a more structured input, such as a JSON object describing the task requirements. The request can originate from an end user via a user interface or be programmatically generated by another system or service that requires the execution of a task. The request can also include additional metadata or context that provides further details about the task, such as priority levels, desired output formats, or execution constraints.

[0193] At 504. example method 500 can include generating, using a machine-learned model, a plurality of tool paths for performing the task (e.g., tool path(s) 110), wherein a respective tool path includes one or more tool calls generated by the machine-learned model, each tool call configured to cause a corresponding tool to perform a step of the task. In some implementations of example method 500, generating, using a machine-learned model, a plurality of tool paths for performing the task can involve invoking the machine-learnedmodel to produce a series of instructions or commands that, when executed, are configured to lead to the completion of the task specified in the request. Each tool path can represent a distinct approach or strategy for accomplishing the task, potentially involving different tools, parameters (for the same or different tool(s)), or sequences of operations. The machine- learned model can be configured to predict and output tool calls that are appropriate for the task context and can consider various factors, such as the capabilities of available tools, the efficiency of different approaches, and the likelihood of success. For example, if the task is to convert a file format, the machine-learned model can predict tool calls to different file conversion utilities, each with specific flags and options that affect the conversion process. The plurality of tool paths can be generated in parallel, allowing for concurrent exploration of multiple potential solutions to the task.

[0194] At 506, example method 500 can include selecting a response to the request (e.g., response 112) based on a consistency metric determined by evaluating a plurality of output states respectively from the plurality of tool paths. In some implementations of example method 500, selecting a response to the request based on a consistency metric determined by evaluating a plurality of output states respectively from the plurality of tool paths can involve comparing the results or intermediate outputs obtained from executing the tool calls along the various tool paths. The consistency metric can serve as a measure of agreement or reliability among the outputs. Higher consistency can indicate a stronger signal that the outputs are correctly addressing the task. The evaluation can involve techniques such as vector similarity measures, semantic analysis, or heuristic rules to assess the level of consistency. For instance, if multiple tool paths generate outputs that all contain a certain file attribute or data structure, this consistency can be interpreted as a positive signal, and the response to the request can be based on these consistent features. Conversely, outputs that significantly deviate from the majority can be considered less reliable and may be given less weight or discarded in the selection of the final response.

[0195] At 508, example method 500 can include returning the response. In some implementations of example method 500, returning the response can involve delivering the selected output to the requester or a downstream system. The response can be formatted or packaged in a manner that is suitable for the recipient, such as embedding the output in a message, storing the output in a specified location, or transforming the output into a desired format. For example, the response could be a direct file download link, an API response object, or a visual representation displayed to a user. The returning of the response can also include providing additional information, such as execution details, confidence levels, orexplanatory notes that help the recipient understand or verify the output. The response can be transmitted over a network, saved to a local or remote storage, or presented through a user interface, depending on the requirements of the task and the capabilities of the system executing example method 500.

[0196] In some implementations of the example method, the machine-learned model is a sequence processing model. For example, the machine-learned model can be a sequence processing model 4. The machine-learned model can be configured to receive an input including one or more input portions (in a sequence) and generate an output including one or more output portions (in a sequence) conditioned on the one or more input portions. The output portions can be autoregressively generated. In some implementations of example method 500, the conditioning of the output portions on the input portions can be achieved through various mechanisms within the machine-learned model. For example, the model may use attention mechanisms, recurrent layers, or other architectural features that enable it to consider the input portions while generating the output portions. This can ensure that the output is contextually relevant and maintains coherence with the input. For instance, in a file conversion task where the input portions specify the target file format and conversion options, the output portions may include a file that has been converted according to those specifications, ensuring that the output is conditioned on the input in a meaningful way.

[0197] In some implementations of example method 500, the one or more output portions can be generated in a manner that allows for iterative or recursive processing. An initial output portion can serve as an intermediate result that is then used as part of the input for generating subsequent output portions. For example, in a multi-step computational task, the output from an initial tool execution can be used as input for the next step in the task, with the machine-learned model generating each step’s output based on the cumulative context provided by previous inputs and outputs. This approach can facilitate complex task execution that requires a sequence of interdependent operations, with each output portion being conditioned on the preceding input and output portions.

[0198] In some implementations of example method 500, conditioning the output portions on the input portions can also involve external validation or verification steps. For example, after generating an output portion, the system may use additional tools or models to check the output for accuracy, compliance with the task requirements, or alignment with expected results. This validation step can be informed by the input portions, as it may compare the output against the task specifications provided in the input. If the output passes the validation, it can be returned as part of the response; if not, the system may adjust the toolpaths, re-sample outputs, or take other corrective actions to ensure that the final output portions are properly conditioned on the input portions.

[0199] In some implementations of example method 500, the one or more output portions conditioned on the one or more input portions can be represented in various formats suitable for different ty pes of tasks. For instance, the output portions may be textual data for natural language processing tasks, binary data for file operations, numerical data for computational tasks, or any other data t pe relevant to the specific task being performed. The machine-learned model can be configured to generate these output portions in the required format by conditioning the output generation process on the input portions, which may include format specifications or other relevant data. This flexibility allows the system to adapt to a wide range of tasks and output requirements, providing tailored responses that are conditioned on the provided inputs.

[0200] In some implementations of example method 500, generating the plurality of tool paths using the machine-learned model involves a systematic approach where each tool path is constructed through a series of steps.

[0201] Figure 6 shows an example execution flow for such a systematic approach. In some implementations, each step can include, at 602, the generation of a tool call by the machine-learned model. For instance, In some implementations of the example method, generating, using the machine-learned model, the plurality of tool paths includes, for each respective tool path, and for each respective step of the respective tool path, generating, using the machine-learned model, a tool call. The machine-learned model can be configured to analyze the task request, determine operations to perform the task, and generate a sequence of tool calls that, when executed, will collectively achieve the desired result. Each tool call can include specific commands, parameters, and options that are tailored to the capabilities of the corresponding tool and the requirements of the task. For example, in a file processing task, the machine-learned model can generate tool calls to invoke file manipulation utilities, with each tool call specifying the operation (e.g., copy, move, delete) and the necessary file paths and options (e.g., overwrite flags, recursive operations).

[0202] In some implementations of the example method, generating, using the machine-learned model, the plurality of tool paths includes, for each respective tool path, and for each respective step of the respective tool path, invoking the tool call to initiate execution of a corresponding tool in a tool execution environment (e.g., executing the tool tall at 604). In some implementations of example method 500, invoking the tool call to initiate execution of a corresponding tool in a tool execution environment can include parsing the tool callgenerated by the machine-learned model and executing it within an isolated or dedicated environment where the tool is installed and configured to run. The tool execution environment can be a virtual machine, a container, a sandboxed execution context, or any secure and controlled runtime environment that supports the execution of the tool without interference from or to other processes or tool paths. The invocation of the tool call can be managed by the tool control system(s) 104, which can handle the setup, initiation, monitoring, and teardown of the tool execution environment. For instance, the tool control system(s) can ensure that the necessary dependencies are in place, allocate resources for the execution, and handle any errors or exceptions that arise during the tool’s operation.

[0203] In some implementations of the example method, generating, using the machine-learned model, the plurality of tool paths includes, for each respective tool path, and for each respective step of the respective tool path, obtaining a result from the tool (e.g., at 606). In some implementations of example method 500, obtaining a result from the tool can involve capturing the output, response, or any data produced by the tool upon completion of its execution. The result can be a direct output, such as a processed file, a computed value, or a status message. It can also be an indirect output, such as a log file, an error report, or a performance metric. The tool control system(s) can retrieve this result from the tool execution environment and pass it back to the machine-learned model for further processing. For example, the result might be a converted file that is then used as input for the next tool call in the sequence, or it might be an error message that triggers a retry or an alternative approach.

[0204] In some implementations of the example method, generating, using the machine-learned model, the plurality of tool paths includes, for each respective tool path, and for each respective step of the respective tool path, evaluating the result (e.g., at 608). In some implementations of example method 500, evaluating the result obtained from the tool can include assessing the quality, relevance, or success of the tool’s execution in the context of the task. The evaluation can be performed by the machine-learned model (or a different machine-learned model), which can use the obtained result to determine the next steps in the tool path or to adjust the strategy’ for accomplishing the task. For instance, the machine- learned model can analyze the result to verify that it meets the task requirements, conforms to expected formats or standards, or aligns with the predicted outcomes. If the result is satisfactory’, the machine-learned model can proceed to generate the next tool call in the sequence; if not, it can take corrective actions, such as modity ing parameters, selecting a different tool, or revising the approach altogether. The evaluation process can leverage a variety of techniques, such as comparison against a ground truth, application of heuristicrules, or use of consistency metrics that measure agreement across multiple tool paths or outputs.

[0205] The processing loops shown in Figure 6 can be performed in series or parallel, or combinations thereof. For example, the generations for each tool path can be performed in parallel. For instance, the tool paths may be independent and thus may be able to execute independently in parallel processes. In an example, each step of a given tool path can be generated sequentially.

[0206] Figure 7 shows an example execution flow for generating tool calls. In some implementations of the example method, generating, using the machine-learned model, the tool call includes, at 702, receiving request data describing at least an aspect of the request.

[0207] In some implementations of the example method, generating, using the machine-learned model, the tool call includes, at 704, obtaining an index of available tools. In some implementations of example method 500, obtaining an index of available tools can involve accessing a repository, database, or listing that enumerates the tools that are accessible to the system and capable of being utilized in the execution of tasks. The index can provide comprehensive information about each tool, including its capabilities, usage syntax, available options or flags, versioning, and compatibility with the current system or environment. This index can serve as a reference for the machine-learned model to identify which tools are suitable for the requested task and to generate tool calls that are syntactically correct and functionally appropriate.

[0208] In some implementations of the example method, generating, using the machine-learned model, the tool call includes, at 706, providing the request data and the index to the machine-learned model. The request data and the index can be included, for example, in a textual prompt to the machine-learned model. The prompt can be structured (e.g., in JSON format) or unstructured.

[0209] In some implementations of the example method, generating, using the machine-learned model, the tool call includes, at 708, receiving an output generated by the machine-learned model based on the request data and the index, wherein the output describes the tool call. In an example, the output can be in the form of a textual command, a structured object, or any other format that conveys the model’s recommendation for the tool call. For example, the output might be a string that specifies a command-line instruction with particular flags and arguments, or it could be a JSON object that maps to an API call with defined parameters.

[0210] In some implementations of the example method, generating, using the machine-learned model, the tool call includes, at, 710, parsing the output to obtain the tool call. In some implementations of example method 500, parsing the output to obtain the tool call can involve extracting actionable parts of the model’s output and translating them into a format that can be directly executed or invoked by the tool execution system(s). This parsing step can handle variations in the output format, resolve ambiguities, and ensure that the tool call conforms to the expected syntax and semantics for the tool being used. The parsing can be performed by processing logic that cleans, formats, and validates the tool call before it is sent for execution.

[0211] In some implementations of the example method, the machine-learned model generates tool calls for each step of a sequence of operations in a tum-by-tum manner. In some implementations of the example method, a first subset of turns are used for invoking tools and a second subset of turns are used for obtaining information. In some implementations of the example method, the first subset of turns are resampled to obtain a plurality of different tool calls and the second subset of turns can be decoded with greedy sampling.

[0212] In some implementations, example method 500 includes determining the consistency metric by comparing alignment of the output states. In some implementations of example method 500, determining the consistency metric by comparing alignment of the output states can involve quantitative or qualitative assessments of how closely the results from different tool paths correspond to one another. The comparison can focus on identifying commonalities and discrepancies across the output states, which can be indicative of the reliability and accuracy of the outputs in relation to the task. For example, the consistency metric may be computed using techniques such as byte-level diffs, which compare the binary’ or textual content of outputs, or semantic similarity measures, which evaluate the meaning or intent behind the outputs. Vector-based similarity measures, such as cosine similarity', can also be employed to compare vector representations of the output states within an embedding space. The alignment can be assessed at different granularities, from full outputs to specific features or attributes within the outputs. For instance, if the task involves generating textual summaries, the consistency metric can compare the overlap in key phrases or concepts across the summaries produced by different tool paths.

[0213] In some implementations of example method 500, determining the consistency metric by comparing alignment of the output states can include the use of a voting mechanism where the outputs from multiple tool paths are polled to determine a majority orplurality consensus on certain features or outcomes. For example, if several tool paths generate outputs with varying file attributes after a file conversion task, the consistency metric can be based on how many of the outputs agree on attributes such as file size, resolution, or format. The alignment can be determined by counting the frequency of each attribute value across the outputs and identifying the value with the highest count as the vote winner. The voting mechanism can be weighted based on confidence scores or quality indicators associated w ith each output, giving greater influence to outputs that are deemed more trustworthy or accurate.

[0214] In some implementations, example method 500 includes determining the consistency metric using a machine-learned model to evaluate a qualify of each output state. In some implementations of example method 500, determining the consistency metric by comparing alignment of the output states can involve the use of machine-learned models specifically trained to evaluate the quality and consistency of outputs. For instance, a separate evaluation model can be employed to process the output states and determine the degree of quality or alignment based on learned patterns or features that are indicative of successful task completion. This evaluation model can be trained on a dataset of labeled outputs where the correct and consistent responses are known, allowing it to develop an understanding of what constitutes quality or alignment in the context of various tasks. The evaluation model can output a quality or consistency score for each output state or a combined score for a set of output states, which can then be used to determine the most aligned and consistent (or highest quality) output to be selected as the response.

[0215] In some implementations of example method 500, determining a consistency metric by comparing alignment of the output states can include the application of domainspecific rules or heuristics that are designed to capture the nuances of particular tasks or data ty pes. For example, in tasks involving structured data such as JSON or XML, the comparison can include tree or graph-based algorithms that assess the structural and content-based consistency of the output states, allowing for variations in order or formatting that do not affect the semantic content. In tasks related to visual outputs, image comparison techniques such as feature embedding and comparison or perceptual hashing can be utilized to evaluate the visual alignment of outputs, even when they have been altered or compressed or otherwise differ in ways that do not affect the semantic content.

[0216] In some implementations of example method 500, the machine-learned model used to evaluate the quality of each output state is the same as the machine-learned model used to generate the plurality’ of tool paths. In some implementations of example method 500,the machine-learned model used to evaluate the quality of each output state is different from the machine-learned model used to generate the plurality of tool paths.

[0217] In some implementations of example method 500, the machine-learned model used to evaluate the quality of each output state is trained to process the plurality of output states together to output a preferred output state, wherein the response is selected using the preferred output state.

[0218] In some implementations of example method 500. the consistency metric is based on a similarity of learned embeddings generated using the plurality’ of output states. In some implementations of example method 500, the consistency metric is based on a similarity of learned embeddings generated using the plurality of output states. This can involve transforming the output states into a high-dimensional space where semantically similar outputs are positioned closer to each other. For example, machine-learned models, such as neural networks or embedding models, can be used to generate vector representations — learned embeddings — for each output state. These embeddings can capture the underlying patterns, features, or semantic content of the outputs, allowing for a comparison that goes beyond surface-level textual or binary similarities. The consistency metric can then be computed by evaluating the distance or similarity between these embeddings, with closer embeddings indicating a higher degree of consistency. For instance, cosine similarity or Euclidean distance measures can be used to quantify the similarity between embeddings, with a threshold set to determine when outputs are considered consistent.

[0219] In some implementations of example method 500, the generation of learned embeddings for the consistency metric can be performed by the same machine-learned model that generates the plurality of tool paths or by a separate, specialized embedding model. When using the same model, the embeddings can benefit from the model’s comprehensive understanding of the task and the data, leading to embeddings that are highly tailored to the specific context. Alternatively, a separate embedding model can be employed to provide an independent evaluation of the outputs, which can be useful for cross-validation or when an external perspective is needed. The choice between using the same model or a separate model can depend on factors such as the complexity of the task, the availability of suitable pretrained models, and the desired level of independence in the evaluation process.

[0220] In some implementations of example method 500, the consistency metric is determined by processing the output states using a machine-learned model trained to generate outputs indicative of salient differences between inputs. This machine-learned model, whichcan be referred to as a differential evaluator model, can be specifically trained to recognize and highlight differences that are deemed significant or relevant to the task at hand. The training process can involve exposing the model to pairs or sets of input data along with annotations or labels that indicate the key differences between them. Through this training, the model leams to generate outputs that effectively capture and represent the salient differences, such as discrepancies in data content, structural divergences, or variations in semantic meaning. For example, the model might be trained on a dataset of text pairs with annotations highlighting the differences in sentiment, topic, or factual content, enabling it to generate outputs that clearly delineate these differences in new sets of text data.

[0221] In some implementations of example method 500, the machine-learned model used to determine the consistency metric operates by generating a saliency map or a differential score for each pair or group of output states. This saliency map can visualize or quantify the differences between the outputs, providing a clear indication of areas where the outputs align or diverge. The differential score can be a numerical value that summarizes the extent of the differences, with higher scores indicating greater disparity. For instance, in a task involving image processing, the differential evaluator model might generate saliency maps that highlight regions of images where the outputs from different tool paths show significant variation, such as differences in object detection or segmentation boundaries. These saliency maps and differential scores can then be used to compute the consistency metric, with the system favoring outputs that exhibit lower differential scores and thus higher consistency.

[0222] In some implementations of example method 500, the differential evaluator model can be integrated into the tool control system(s) to provide real-time feedback on the quality of the output states generated by the tool paths. As the tool paths produce outputs, the differential evaluator model can process these outputs to identify salient differences and generate corresponding saliency maps or differential scores. This feedback can inform decisions about pruning suboptimal tool paths, refining tool calls, or selecting the most consistent outputs for the final response.

[0223] In some implementations of example method 500. the differential evaluator model can be employed as part of a larger ensemble of evaluation techniques to determine the consistency metric. In addition to the differential evaluation, other methods such as semantic similarity assessment, structural comparison, or heuristic rule application can be used to evaluate the output states. The consistency metric can be determined by’ combining the signals from these varied evaluation techniques.

[0224] In some implementations of example method 500, the consistency metric includes one or more heuristic criteria and one or more probabilistic criteria, the one or more probabilistic criteria determined using an output of a machine-learned model. In some examples, heuristic criteria can include deterministic rules or patterns that have been engineered based on domain knowledge or empirical observations. These criteria can serve as an interpretable mechanism that can consistently identify output states that meet basic requirements or to flag those that exhibit clear signs of errors or anomalies. For example, heuristic criteria can include checks for specific file attributes in a file conversion task, validation of JSON schema in a data transformation task, or verification of compliance with predefined output formats. The heuristic criteria can be particularly effective in identifying gross inconsistencies or deviations from expected results, providing a straightforward and computationally efficient means of assessing the output states.

[0225] One or more probabilistic criteria can be generated by a machine-learned model. For instance, the model can use probabilistic methods such as likelihood estimation, Bayesian inference, or probabilistic clustering to evaluate the output states. These probabilistic criteria can capture subtleties that heuristic rules may overlook, such as semantic coherence, contextual relevance, or complex patterns that emerge from the data. By combining both heuristic and probabilistic criteria, the consistency metric can leverage the strengths of both approaches.

[0226] In some implementations of example method 500. use of a machine-learned model to determine the probabilistic criteria for the consistency metric can involve various machine learning architectures and techniques suitable for the specific task and data involved. For example, neural networks, support vector machines, decision trees, or ensemble methods can be employed to generate the probabilistic criteria. The model can be pre-trained on large datasets to develop a generalized understanding of task-related features and then fine-tuned on task-specific data to refine its predictions for the consistency evaluation. The probabilistic criteria can be represented as confidence scores, likelihood ratios, or other statistical measures that quantify the degree of consistency across the output states. These probabilistic evaluations can complement the heuristic criteria by providing a data-driven, adaptive, and potentially self-improving mechanism for assessing the quality and consistency of the outputs as part of the consistency metric.

[0227] In some implementations, example method 500 includes selecting the response based on the consistency metric by identifying one or more features shared by a group of the plurality of output states. In some implementations of example method 500, selecting theresponse based on the consistency metric includes identifying one or more features shared by a group of the plurality of output states. This process can involve analyzing the results obtained from the execution of the tool paths to detect common elements or patterns that recur across multiple outputs.

[0228] In some implementations, example method 500 includes selecting the response based on the consistency metric by selecting the response from the group. In some implementations of example method 500, selecting the response from the group based on the identified shared features involves choosing the output state or states that best represent the commonalities found in the group of output states. This selection process can prioritize outputs that exhibit the shared features most prominently or most accurately, depending on the task requirements and the desired quality of the response. For example, among several output states that all include a particular data visualization, the selection may favor the output state in which the visualization is most clearly presented and most closely matches the expected format. The selection can also consider additional criteria, such as the overall completeness of the output, the confidence scores associated with the shared features, or the alignment with user preferences. The chosen output state or states from the group can then be compiled, synthesized, or simply forwarded as the final response to the task.

[0229] In some implementations of example method 500, one or more features include at least one latent feature generated using a machine-learned model. In some implementations of example method 500, the one or more features include at least one latent feature generated using a machine-learned model. An example latent feature can be an attribute, characteristic, or representation that is not explicitly present in the raw data but can be inferred or derived from it through the processing capabilities of a machine-learned model. For example, in the context of image processing, a latent feature could represent an abstract concept such as the style or mood of an image, which a machine-learned model can capture by analyzing patterns and relationships within the pixel data that are not immediately discernible to human observers. Similarly, in natural language processing tasks, a latent feature might represent the underlying sentiment or thematic content of a text, which the machine-learned model can identify by interpreting the subtle cues and contexts conveyed by the choice of words and sentence structures.

[0230] In some implementations, example method 500 includes selecting the response based on the consistency metric by clustering representations of one or more features of the plurality of output states. This clustering process can be executed by employing a machine- learned model or clustering algorithm to analyze the features extracted from the output statesand group them into clusters based on similarity or relatedness. Each cluster represents a set of output states that share common features, indicating a potential alignment in how they address the task. For instance, in a data classification task, output states that assign similar class labels to the same data points can be grouped into the same cluster. The clustering can be based on various algorithms, such as k-means, hierarchical clustering, or density-based clustering, and can leverage different distance metrics, such as Euclidean distance or Jaccard similarity, to determine the closeness of the feature representations.

[0231] In some implementations, example method 500 includes selecting the response based on the consistency metric by identifying a preferred cluster. The preferred cluster can be the one that is deemed most likely to contain the output states that best fulfill the task requirements. The identification can be based on criteria such as the size of the cluster, the centrality of the cluster in the feature space, the density of the cluster, or the quality scores associated with the output states within the cluster. For example, a larger cluster might indicate a higher level of agreement among the outputs, while a cluster with a high density might suggest a strong consensus on certain features. The preferred cluster can also be identified based on external validation or additional consistency checks that confirm the reliability of the outputs it contains. For instance, if a cluster includes outputs that all pass a set of heuristic validation checks, it may be chosen as the preferred cluster for the response selection.

[0232] In some implementations, example method 500 includes selecting the response based on the consistency metric by selecting the response from the cluster. In some implementations of example method 500, selecting the response from the cluster involves choosing an output state from within the preferred cluster to serve as the final response to the task. The system may select the output state that is most central or representative of the preferred cluster. In some implementations of example method 500, selecting the response from the cluster includes determining a centroid of the cluster and selecting the response based on the centroid.

[0233] The system may aggregate information from multiple output states in the cluster to synthesize a composite response. For example, in a task where the cluster represents different textual descriptions of the same event, the system can select the most comprehensive and well-articulated description as the response. Alternatively, the system could combine elements from several descriptions to create a more detailed and informative response.

[0234] In some implementations of example method 500, selecting the response from the cluster includes scoring members of the cluster and selecting the response based on the scores. This process can involve assigning a numerical or categorical score to each output state within the preferred cluster based on one or more evaluation criteria. The criteria can include factors such as the degree of alignment with the task requirements, the presence of desired features or attributes, the quality of the output as determined by a machine-learned model or heuristic evaluation, and the confidence level associated with each output state. The scoring process can be executed by the machine-learned model that generated the output states, by a separate evaluation model, or by a scoring system that integrates multiple evaluation methods. For example, each output state might receive a score that reflects its semantic accuracy in a text summarization task or its precision in a data classification task. The output state with the highest score within the cluster can then be selected as the response to be returned.

[0235] In some implementations of example method 500, the scoring of members within the cluster can be based on a weighted scoring system that considers various aspects of the output states. Each aspect can be assigned a different weight based on its importance or relevance to the task, allowing the system to prioritize certain features over others when calculating the overall score for each output state. The w eighted scores can then be aggregated to produce a composite score for each output state, which can be used to rank the output states within the preferred cluster. The output state with the highest composite score, taking into account the weighted contributions of different aspects, can be selected as the response.

[0236] In some implementations of example method 500, the respective tool path includes one or more nodes, wherein a respective output state of the respective tool path is based on a completion node of the respective tool path. A node can correspond to, for instance, a tool execution (e.g., execution 304). A completion node can be, for instance, a terminal node in a tool path. A node can be a completion node if the tool response, when evaluated, is considered to provide a response to request 102 (or data sufficient to facilitate responding to request 102).

[0237] In some implementations of example method 500, a first node of the one or more nodes is generated by generating, using the machine-learned model, a first tool call to execute a first step of the task. In some implementations of example method 500, a first node of the one or more nodes is generated by initiating execution of the first tool call to obtain a first tool response. In some implementations of example method 500, initiating execution ofthe first tool call to obtain a first tool response involves passing the generated tool call to a tool execution system or environment where the corresponding tool is available for use.

[0238] In some implementations of example method 500, initiating execution of the first tool call includes passing the first tool call to a corresponding tool in an execution environment associated with the corresponding tool. This execution environment can be a dedicated space where the tool is configured to operate, such as a virtual machine, a container, a cloud-based environment, or a specific runtime context within an operating system. The environment can be designed to provide the necessary computational resources, libraries, and access permissions for the tool to function effectively. For instance, if the first tool call is a command to process a dataset using a statistical analysis tool, the execution environment might include the required data science libraries and sufficient memory allocation to handle the dataset. The tool control system(s) can facilitate the passing of the first tool call to the tool execution environment, ensuring that the tool call is formatted correctly and that any required context or parameters are included. The execution environment can be isolated from other processes or tool paths to prevent interference and maintain the integrity of the execution.

[0239] In some implementations of example method 500, the execution environment associated with the corresponding tool can be instantiated dynamically in response to the first tool call. For example, upon receiving the first tool call, the tool control system(s) can trigger the creation of a new execution environment that is tailored to the needs of the tool call. This can involve provisioning a new virtual machine or container, setting up the appropriate runtime settings, and deploying the tool within the environment. The dynamic instantiation of the execution environment can allow for flexible and scalable execution of tool calls, as each tool call can be handled in an environment that is optimized for its specific requirements. Additionally, this approach can support parallel execution of multiple tool calls, each within its own environment, enabling concurrent processing of different steps or tasks. The dynamic environments can be managed by an orchestration layer that allocates resources, monitors the health of the environments, and handles clean-up operations once the tool executions are complete.

[0240] In some implementations of example method 500, the machine-learned model used to generate the tool paths is configured to control operations in the execution environment, such as GUI interactions, command-line controls, write access, read access, etc. The model can be integrated with the execution environment through APIs, command-line interfaces, or other interaction protocols that allow it to exert control over the tool operations.

[0241] In some implementations of example method 500, the execution environment is a virtual machine assigned to a task instance including the machine-learned model used to generate the tool paths.

[0242] In some implementations of example method 500, the execution environment prevents the machine-learned model used to generate the tool paths from interfering with another task instance. In some implementations of example method 500, the execution environment prevents the corresponding tool from interfenng with another tool associated with another tool path. This isolation can be achieved through various mechanisms, such as running each task instance in a separate virtual machine, container, or process that is isolated at the operating system level. For example, the execution environment can utilize virtualization technologies that provide each task instance with its own virtualized set of hardware resources, ensuring that the operations of one task instance do not impact the performance or stability of another. Similarly, containerization can encapsulate each task instance within its own environment that includes the necessary runtime, libraries, and configurations, while maintaining strict boundaries between different instances.

[0243] In some implementations of example method 500. each tool path is executed in a separate execution environment. In some implementations of example method 500, the separate execution environment is instantiated by a virtual machine. In some implementations of example method 500, the separate execution environment is instantiated by separate working directories within a directory tree.

[0244] In some implementations, example method 500 includes caching resources for use across multiple tool paths. In some implementations, example method 500 includes caching resources for use across multiple execution environments.

[0245] In some implementations of example method 500, caching resources for use across multiple tool paths involves storing commonly used data or computational routines in a cache memory or storage to optimize the execution efficiency of the tool paths. The cached resources can include libraries, datasets, intermediate computation results, configuration files, or any other assets that are frequently accessed or required by the tools associated with the tool paths. By caching these resources, the system can reduce redundant computations, minimize disk I / O operations, and decrease the startup time for tool executions. For example, if multiple tool paths utilize the same data analysis library, the library' can be loaded into the cache once and then shared among the tool paths, eliminating the need to repeatedly load the library from disk or retrieve from an external source for each tool path. In some implementations of example method 500, the execution environments can benefit from thecached resources by avoiding redundant operations such as reloading libraries from disk, refetching data from remote servers, or recalculating results that have already been computed. For instance, if multiple tool paths require access to a particular machine learning library, this library can be cached so that subsequent tool paths can access the library from the cache without incurring the overhead of loading it from an external source. This can significantly reduce the startup time for each tool and improve the overall performance of the system by minimizing unnecessary I / O operations and computations.

[0246] In some implementations of example method 500, the caching of resources for multiple tool paths can be managed by a resource management subsystem within the execution environment (e.g., tool execution system(s) 108). This subsystem can monitor the usage patterns of the tools, identify resources that are common across tool paths, and implement caching strategies that balance memory usage with performance gains. The caching mechanism can be dynamic, with the subsystem adding resources to the cache as they are detected to be in high demand and evicting resources that become less frequently used. The cache can be implemented using various technologies, such as in-memory data stores, distributed cache systems, or specialized caching hardware, depending on the scale and requirements of the task execution framework.

[0247] In some implementations of example method 500, the resources cached for use across multiple tool paths can be accessed through a shared layer that is available to all execution environments associated with the tool paths (e.g.. tool execution system(s) 108). This shared layer can serve as a centralized repository for cached resources, providing a consistent and efficient access point for the tools. The shared layer can implement access controls and synchronization mechanisms to ensure that the cached resources are used correctly and that updates to the resources are properly propagated across all tool paths.

[0248] In some implementations of example method 500, the caching strategy for resources used across multiple tool paths can be informed by predictive models or machine learning techniques that anticipate future resource needs based on historical usage patterns or task characteristics. For example, a machine-learned model can analyze the execution history’ of the tool paths, identify patterns in resource usage, and predict which resources are likely to be needed in the near future. The system can then proactively cache these predicted resources, ensuring that they are readily available when required by the tool paths. This predictive caching can reduce latency and improve the responsiveness of the task execution process, as the necessary resources are already in the cache when the tool paths need them.

[0249] In some implementations of example method 500, the caching of resources can be implemented using various caching mechanisms and storage solutions, such as inmemory databases, file-based caches, or distributed caching systems. For example, an inmemory database can provide fast access to cached datasets, while a file-based cache can be suitable for storing large files or assets that are not frequently updated. Distributed caching systems can offer scalability and resilience, allowing the cache to be maintained across multiple machines or nodes in a cluster. The choice of caching mechanism can depend on factors such as the size and type of resources to be cached, the expected access paterns, and the seal ability requirements of the execution environments.

[0250] In some implementations of example method 500, the management of the cache can involve sophisticated policies and algorithms that determine which resources are to be cached, when they should be cached, and how long they should remain in the cache. For instance, a least recently used (LRU) policy can be employed to evict resources that have not been accessed for a certain period of time, making room for more frequently used resources. Alternatively, a predictive caching algorithm can analyze usage paterns and task schedules to pre-emptively cache resources that are likely to be needed in the near future. These caching policies and algorithms can be adjusted and optimized based on real-time monitoring of the execution environments and the performance metrics of the tool paths.

[0251] In some implementations of example method 500, the caching of resources can also include mechanisms for ensuring consistency and integrity of the cached data across multiple execution environments. This can involve version control systems that track different versions of the resources, synchronization protocols that update the cache in response to changes, and validation checks that verify the correctness of the cached resources before they are used by the tools. For example, if a cached dataset is updated by one execution environment, the cache can be synchronized to ensure that all other environments access the latest version of the dataset. These consistency mechanisms can prevent issues such as data staleness, cache incoherence, or conflicts arising from concurrent updates, ensuring that the cached resources remain reliable and up-to-date for all execution environments.

[0252] In some implementations of example method 500, a second node of the one or more nodes is generated by generating, using the machine-learned model and based on the first tool response, a second tool call to execute a second step of the task. In some implementations of example method 500, a second node of the one or more nodes is generated by initiating execution of the second tool call to obtain a second tool response. Forinstance, if the first tool response indicates a successful file transfer, the second tool call might be to a tool that performs file format conversion on the transferred file. The machine- learned model generates the second tool call by analyzing the first tool response, considering the overall task requirements, and selecting the next logical operation to move the task closer to completion.

[0253] In some implementations of example method 500, the plurality of tools paths includes a tool path tree. In some implementations of example method 500, the tool path tree can represent a structured hierarchy or network of execution sequences, each branching out from a common starting point or root based on different decision points, tool selections, or parameter configurations. The structure of the tree allows for the exploration of multiple solution paths in parallel, with each branch representing a distinct approach to accomplishing the task. The branches of the tree can diverge based on the outcomes of tool executions, the availability of resources, or the requirements of the task.

[0254] In some implementations of example method 500, a second respective tool path includes one or more nodes, wherein a respective output state of the second respective tool path is based on a completion node of the second respective tool path. In some implementations of example method 500, a first node of the one or more nodes is generated by generating, using the machine-learned model, a first tool call to execute a first step of the task. In some implementations of example method 500, a first node of the one or more nodes is generated by initiating execution of the first tool call to obtain a first tool response.

[0255] In some implementations, example method 500 includes evaluating a plurality of intermediate states respectively for the plurality' of tool paths, wherein a respective intermediate state of the respective tool path is based on an intermediate node of the respective tool path that precedes the completion node. In some implementations of example method 500, evaluating a plurality of intermediate states respectively for the plurality of tool paths involves assessing the progress and outcomes of various steps leading up to the completion node of each tool path. An intermediate state can correspond to the result or status of an operation performed at an intermediate node along the tool path, such as a step that occurs after the initial node but before the final completion node. The evaluation can include checking whether the intermediate state aligns with the expected progression towards the task goal, verifying that it meets interim criteria or milestones, and determining if it provides a suitable foundation for subsequent steps. For example, an intermediate state might be the output of a data preprocessing tool in a data analysis task, and the evaluation might involve confirming that the preprocessing was successful and that the data is now ready for the nextanalysis step. This evaluation can be performed by the machine-learned model, which can use the intermediate states to guide the generation of future tool calls and to navigate the tool path effectively toward the completion node.

[0256] In some implementations, example method 500 includes pruning at least one of the plurality of tool paths based on a corresponding intermediate state. In some examples, pruning involves discontinuing or discarding a tool path that is determined to be suboptimal, unproductive, or likely to fail based on the evaluation of its intermediate state. This action can be taken when an intermediate state indicates a deviation from the desired course, such as encountering an error, failing to meet a quality threshold, or producing results that are inconsistent with the task requirements. For example, if an intermediate state reveals that a data transformation step resulted in corrupted data, the system might prune the tool path to prevent further processing of the faulty data and to conserve resources. The pruning decision can be informed by heuristic rules, quality scores, or predictive models that assess the likelihood of successful task completion for each tool path. By selectively pruning tool paths, the system can focus its efforts on the most promising avenues for task completion, optimizing resource utilization and increasing the chances of achieving the desired outcome.

[0257] In some implementations, example method 500 includes pruning at least one tool path based on determining a redundancy of the at least one tool path. In some implementations of example method 500, pruning at least one tool path based on determining a redundancy of the at least one tool path involves identifying and eliminating tool paths that are duplicative in the context of the task execution. Redundancy can refer to situations where multiple tool paths are effectively pursuing the same approach, generating similar outputs, or otherwise contributing little additional value to the task completion process. For instance, if two tool paths are executing the same sequence of tool calls with no or only minor variations that do not impact the final result, one of these paths may be deemed redundant and pruned to streamline the execution process. The determination of redundancy can be based on a comparison of the intermediate states or outputs of the tool paths, looking for a high degree of similarity or overlap that suggests the paths are effectively equivalent.

[0258] In some implementations of example method 500. the pruning of redundant tool paths can be performed dynamically, in real-time, as the task execution progresses. As new tool calls are generated and new intermediate states are evaluated, the system can continuously monitor the tool paths for signs of redundancy.

[0259] In some implementations of example method 500. the redundancy is determined based on identifying a deterministic outcome of the at least one tool path. In someimplementations of example method 500, determining redundancy based on identifying a deterministic outcome of at least one tool path involves recognizing when a particular tool path is expected to produce the same result. The system can detect such deterministic behavior by comparing input states of the same tool across multiple executions. In some implementations of example method 500, the redundancy is determined based on a margin of similarity.

[0260] In some implementations of example method 500. the margin of similarity is based on a minimum step size associated with a range of parameter values for a tool call. For example, if a tool call involves adjusting the brightness of an image, the minimum step size might be set to the smallest brightness change that produces a perceptible difference in the image’s appearance. The range of parameter values can encompass the allowable spectrum of values for a given parameter, with the margin of similarity being defined as the span of values within which outcomes are considered effectively equivalent due to the minimum step size. For instance, if the range of brightness values is from 0 to 100 and the minimum step size is 5, then a margin of similarity might be established such that brightness values within 5 units of each other are treated as redundant.

[0261] In some implementations, example method 500 includes adjusting at least one tool path based on determining a failure condition associated with the at least one tool path. In some implementations of example method 500, adjusting at least one tool path based on determining a failure condition associated with the at least one tool path involves identifying issues or errors that arise during the execution of the tool path and taking corrective actions to address these issues. A failure condition can be any event or outcome that deviates from the expected or desired execution flow-, such as an error message returned by a tool, a processing exception, or an output that does not meet predefined quality criteria. The system can detect these failure conditions using monitoring mechanisms, error detection algorithms, or feedback from the machine-learned model that generates the tool paths (e.g., validator 400). Upon detecting a failure condition, the system can adjust the affected tool path by modifying tool calls, changing execution parameters, rerouting the path to alternative tools, or restarting the path from a previous successful state.

[0262] In some implementations of example method 500, the failure condition is determined based on receiving a tool response that diverges from desired criteria. In some implementations of example method 500, the divergence can manifest in various w ays, such as outputs that do not match the expected format, results that fall outside of acceptable quality ranges, or data that fails to meet predefined accuracy or completeness thresholds. The desiredcriteria can be specified as part of the task requirements or can be derived from domain knowledge or prior experience with similar tasks. In some implementations of example method 500, the determination of a failure condition based on a divergent tool response can involve comparing the received tool response against a set of expected outcomes or benchmark results. The comparison can be performed automatically by the tool control system(s), which can use validation algorithms, pattern matching techniques, or machine- learned models to assess the tool response. For instance, the system might use a checksum comparison to verify the integrity of a file output against an expected checksum value; if the values do not match, indicating a divergence, the system can flag this as a failure condition. Alternatively, the system might employ a natural language processing model to evaluate the semantic content of a textual output; if the model detects significant deviations from the task’s semantic criteria, such as missing concepts or incorrect information, this can be identified as a failure condition.

[0263] In some implementations of example method 500, the determination of a failure condition based on a divergent tool response (e.g., tool execution system(s) 108) can also include the use of heunstic rules or expert systems that encode domain-specific knowledge about the task and the expected characteristics of successful outputs. These rules or systems can provide a knowledge-based framework for evaluating the tool response, identifying divergence based on established principles or best practices. For example, in a software compilation task, heuristic rules might specify that a successful output must pass a series of automated tests; if the tool response indicates that the compiled software failed these tests, the system can determine that this divergence constitutes a failure condition.

[0264] In some implementations of example method 500, the failure condition is determined based on receiving an error message indicating inoperability of a tool or tool call. In some implementations of example method 500, adjusting the at least one tool path includes pruning the at least one tool path. In some implementations of example method 500, adjusting the at least one tool path includes restarting the at least one tool path using one or more tool calls newly generated by the machine-learned model. In some implementations of example method 500, restarting the at least one tool path includes generating a new tool call based on the failure condition.

[0265] In some implementations of example method 500, generating the new tool call based on the failure condition includes inputting data describing the failure condition to the machine-learned model as context to a query for the new tool call.

[0266] In some implementations of example method 500, generating the new tool call based on the failure condition includes inputting data describing the failure condition to the machine-learned model as context to a query for the new tool call. This process can include providing the machine-learned model with detailed information about the nature of the failure, such as error codes, diagnostic messages, or descriptions of how the tool path’s execution diverged from the expected outcomes. The failure condition data serves as contextual input that informs the model’s generation of a new tool call. By incorporating this context, the model can make more informed predictions about the most effective course of action to take in order to address the failure and progress the task. For example, if the failure condition data indicates that a file conversion tool failed due to an unsupported file format, the model can use this context to generate a new tool call that specifies a different conversion tool that is known to support the required format. A query might explicitly ask the model to generate a tool call that avoids the issues identified in the failure condition data or to suggest alternative strategies that circumvent the failure.

[0267] In some implementations, example method 500 includes evaluating at least one intermediate state using the machine-learned model. In some implementations, example method 500 includes evaluating at least one intermediate state using a different machine- learned model from the machine-learned model used to generate the tool calls.

[0268] In some implementations, example method 500 includes evaluating at least one intermediate state using the machine-learned model to select a heuristic tool to apply to the at least one intermediate state. In some implementations of example method 500, evaluating at least one intermediate state using the machine-learned model to select a heuristic tool to apply to the at least one intermediate state involves the machine-learned model analyzing the intermediate state to determine an appropriate heuristic tool that can address issues or enhance the quality of the intermediate state. The intermediate state can be an output or status update from a particular operation within the tool path, and the heuristic tool can be a specialized algorithm or routine designed to perform specific checks, transformations, or optimizations on the data. For example, if the intermediate state is a partially processed dataset, the heuristic tool selected by the model might be a data cleaning algorithm that removes outliers or normalizes values.

[0269] In some implementations, example method 500 includes evaluating multiple features of at least one intermediate state. In some implementations of example method 500, evaluating multiple features of at least one intermediate state involves analyzing various aspects or dimensions of data produced during the execution of a tool path prior to reachingthe completion node. In some implementations of example method 500, the evaluation of multiple features of at least one intermediate state can include both quantitative and qualitative assessments.

[0270] In some implementations, example method 500 includes computing a consistency metric over the plurality of intermediate states. In some implementations, example method 500 includes determining, using the request, one or more screening criteria for evaluating the intermediate states. In some implementations, example method 500 includes pruning the at least one tool path based on a failure with respect to the one or more screening criteria.

[0271] In some implementations, example method 500 includes evaluating an intermediate consistency metric over one or more tool calls sampled from the machine- learned model. In some implementations, example method 500 includes selecting a representative tool path based on the consistency metric. In some implementations, example method 500 includes pruning one or more other tool paths.

[0272] In some implementations, example method 500 includes computing a consistency metric over the plurality of intermediate states to identify a cluster of aligned tool paths. In some implementations, example method 500 includes selecting a representative tool path based on the consistency metric, wherein the representative tool path represents the cluster of aligned tool paths. In some implementations, example method 500 includes pruning the other tool paths in the cluster of aligned tool paths.

[0273] In some implementations, example method 500 includes computing a consistency metric over the plurality of intermediate states to identity' a cluster of aligned tool paths. In some implementations, example method 500 includes selecting a representative tool path based on the consistency metric, wherein the representative tool path represents the cluster of aligned tool paths. In some implementations, example method 500 includes pruning the other tool paths in the cluster of aligned tool paths. In some implementations, example method 500 includes starting a new tool path by sampling a new tool call from the machine- learned model.

[0274] In some implementations of example method 500. the tool call includes a textual command configured for entry via a command line interface. In some implementations of example method 500, the tool call includes a textual command configured for entry via a command line interface (CLI). The textual command can be a string of text that represents an instruction or a series of instructions which, when entered into a CLI, are executed by the operating system or a specific application to perform a desiredoperation. The format of the textual command can adhere to the syntax and conventions expected by the CLI. including the use of command names, flags, arguments, and options. The machine-learned model can generate this textual command by predicting the appropriate command structure based on the task requirements and the capabilities of the tool associated with the tool call.

[0275] In some implementations of example method 500, the tool call includes a request to an application programming interface endpoint. In some implementations of example method 500, the tool call includes a request to an application programming interface endpoint over a network. In some implementations of example method 500, the tool call includes a request to an application programming interface (API) endpoint. The request to the API endpoint can be a structured command that conforms to the interface specifications of the API, such as an HTTP request with a specific method (e.g., GET, POST, PUT), a Uniform Resource Identifier (URI) that identifies the API resource, and potentially additional parameters or data payloads that provide the necessary input for the API operation. The API endpoint can be associated with a service or application that performs a particular function relevant to the task, such as a data storage service, a computational engine, or a third-party tool with specialized capabilities.

[0276] In some implementations of example method 500, the API request that constitutes the tool call can include various elements used for invocation of the API endpoint. These elements can include headers that specify metadata such as content type or authentication tokens, query parameters that fine-tune the API operation, and a body payload that contains the input data or instructions for the API service. The API request can be formulated to match the API’s expected request format, ensuring that the sen-ice can correctly interpret and process the request. For instance, if the task involves querying a database via an API, the API request might include query parameters that define the search criteria and a body payload that specifies the fields to be retrieved.

[0277] In some implementations, example method 500 includes generating, using the machine-learned model, a tool including one or more lines of programming code that perform operations related to the requested task. In some implementations, example method 500 includes executing the tool in at least one tool path. The programming code can be written in a variety of languages, such as Python, Java, C++, or any other language suitable for the task and the execution environment.

[0278] In some implementations of example method 500. executing the tool in at least one tool path involves running the generated programming code within a designatedexecution environment that supports the execution of the code. The execution of the tool can be initiated by the tool control system(s), which can compile, interpret, or directly execute the code depending on the language and the nature of the tool. For instance, the tool might be a script that is interpreted by a scripting engine or a compiled binary that is executed directly by the operating system.

[0279] In some implementations, example method 500 includes computing a consistency metric over a plurality of tool calls. In some implementations, example method 500 includes evaluating an intermediate consistency metric over one or more tool calls sampled from the machine-learned model. The evaluation can include assessing the similarity of the tool calls to each other, their alignment with the task requirements, and their adherence to known best practices or successful patterns. For example, if the task involves data extraction from various sources, the intermediate consistency metric might measure the degree to which the sampled tool calls employ similar extraction methods or target similar data structures, suggesting a unified approach to the task. The evaluation of the intermediate consistency metric can be performed by the machine-learned model itself or by a separate evaluation system.

[0280] In some implementations, example method 500 includes selecting one or more tool calls for execution to improve a diversity7of tool paths. In some implementations of example method 500, selecting one or more tool calls for execution to improve a diversity of tool paths involves strategically choosing tool calls that introduce variation and breadth to the range of approaches being explored for task completion.

[0281] In some implementations of example method 500, improving the diversity of tool paths by selecting one or more tool calls can also involve balancing the exploration of new approaches with the exploitation of known successful strategies. The system can employ a multi-armed bandit algorithm, reinforcement learning techniques, or other decision-making frameworks that weigh the trade-offs between trying out novel tool calls and focusing on tool calls that have previously led to positive outcomes. For example, the system might allocate a portion of its computational resources to experimental tool calls that use cutting-edge tools or innovative methods, while dedicating the remaining resources to established tool calls with a proven track record. This balanced approach can enhance the diversity of the tool paths while maintaining a level of performance consistency.

[0282] In some implementations of example method 500, the selection of tool calls to improve diversity can be guided by a diversity metric that quantifies the variation among the tool paths. The diversity metric can be computed based on the heterogeneity of the tools,methods, or data involved in the tool calls, with higher values indicating greater diversity. For instance, the metric might consider the range of different tools being used, the variety of data processing techniques employed, or the spread of parameter settings across the tool calls.

[0283] In some implementations, example method 500 includes clustering tool calls sampled from the machine-learned model. In some implementations, example method 500 includes selecting a representative tool call from each of the top-K clusters for execution. In some implementations of example method 500, selecting a representative tool call from each of the top-K clusters for execution involves identifying a representative tool call within each cluster that represents a typical approach of the cluster’s members. The top-K clusters can be those that have the highest relevance, size, density, or quality scores, indicating that they are the most promising or aligned with the task objectives. The selection of representative tool calls can be based on criteria such as their proximity to the cluster centroid, their coverage of the cluster’s feature space, or their performance in preliminary evaluations.

[0284] In some implementations, example method 500 includes determining, based on the tool call, one or more operations to perform using a graphical user interface. In some implementations of example method 500, determining, based on the tool call, one or more operations to perform using a graphical user interface (GUI) involves interpreting the tool call to identify actions that can be executed within a GUI environment. The operations can include tasks such as opening a file, selecting a menu option, inputting data into a form, or triggering other GUI-based processes. The operations can be determined by parsing an image rendering of the GUI (e g., using a multimodal machine-learned model).

[0285] In some implementations, example method 500 includes generating one or more navigation commands configured to interact with the graphical user interface. In some implementations of example method 500, generating one or more navigation commands configured to interact with the graphical user interface can include creating sequences of instructions that simulate user interactions with the GUI elements. The navigation commands can automate GUI operations, such as clicking buttons, entering text, navigating menus, or dragging items. The commands can be generated in a format that is compatible with GUI automation tools or frameworks that execute the commands within the GUI context. For instance, the machine-learned model can generate a navigation command that emulates the sequence of clicks required to export a chart from a data analysis GUI. The generation of navigation commands can allow the system to perform tasks using GUI-based tools in an automated and efficient manner, expanding the range of tools and operations that can be utilized in the execution of the task.

[0286] In some implementations, example method 500 includes initiating execution of the one or more navigation commands. In some implementations of example method 500. initiating execution of the one or more navigation commands to interact with the graphical user interface can be managed by the tool control system(s) 104 or tool execution environment(s) 108, which can deploy automation engines or virtual input devices to execute the commands within the GUI environment. The initiation process can involve translating the navigation commands into low-level system events, such as mouse movements, keyboard strokes, or touch gestures, that are recognized by the GUI. The execution of the navigation commands can be synchronized with the GUI’s responses. For example, the system can wait for a menu to open before initiating a command to select an item from the menu. The initiation of navigation commands enables the system to leverage GUI-based applications or services as part of the tool paths, providing a means to perform operations that require direct interaction with the GUI.

[0287] In an aspect, the present disclosure provides an example method (e.g., a method that is part of or different from example method 500). In some implementations, the example method includes transmitting, from a client computing system and to a server computing system, a request to perform a task.

[0288] In some implementations, the example method includes receiving, by the client computing system and from the server computing system, a response based on performance of the task.

[0289] In some implementations, the response is obtained using any of the example implementations of example method 500.

[0290] In some implementations of the example method, the server computing system performs the task using any of the example implementations of example method 500.

[0291] In some implementations, the example method includes receiving, by the client computing system and from a user computing system, an initial request. In some implementations, the example method includes obtaining the request to perform the task based on the initial request.

[0292] Figure 8 depicts a flowchart of a method 800 for training one or more machine-learned models according to aspects of the present disclosure. For instance, an example machine-learned model can include a machine-learned model 106.

[0293] One or more portion(s) of example method 800 can be implemented by a computing system that includes one or more computing devices such as, for example, computing systems described with reference to the other figures. Each respective portion ofexample method 800 can be performed by any (or any combination) of one or more computing devices. Moreover, one or more portion(s) of example method 800 can be implemented on the hardware components of the device(s) described herein, for example, to train one or more systems or models. Figure 8 depicts elements performed in a particular order for purposes of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the methods discussed herein can be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. Figure 8 is described with reference to elements / terms described with respect to other systems and figures for exemplary illustrated purposes and is not meant to be limiting. One or more portions of example method 800 can be performed additionally, or alternatively, by other systems.

[0294] At 802, example method 800 can include obtaining a training instance. A set of training data can include a plurality of training instances divided between multiple datasets (e.g., a training dataset, a validation dataset, or testing dataset). A training instance can be labeled or unlabeled. Although referred to in example method 800 as a “training"’ instance, it is to be understood that runtime inferences can form training instances when a model is trained using an evaluation of the model’s performance on that runtime instance (e g., online training / leaming). Example data types for the training instance and various tasks associated therewith are described throughout the present disclosure.

[0295] At 804. example method 800 can include processing, using one or more machine-learned models, the training instance to generate an output. The output can be directly obtained from the one or more machine-learned models or can be a downstream result of a chain of processing operations that includes an output of the one or more machine- learned models.

[0296] At 806, example method 800 can include receiving an evaluation signal associated with the output. The evaluation signal can be obtained using a loss function. Various determinations of loss can be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, contrastive loss, or various other loss functions. The evaluation signal can be computed using known ground-truth labels (e.g., supervised learning), predicted or estimated labels (e.g., semi- or self-supervised learning), or without labels (e.g., unsupervised learning). The evaluation signal can be a reward (e.g., for reinforcement learning). The reward can be computed using a machine-learned reward model configured to generate rewards based on output(s) received. The reward can be computed using feedback data describing human feedback on the output(s).

[0297] At 808, example method 800 can include updating the machine-learned model using the evaluation signal. For example, values for parameters of the machine-learned model(s) can be learned, in some embodiments, using various training or learning techniques, such as, for example, backwards propagation. For example, the evaluation signal can be backpropagated from the output (or another source of the evaluation signal) through the machine-learned model(s) to update one or more parameters of the model(s) (e.g., based on a gradient of the evaluation signal with respect to the parameter value(s)). For example, system(s) containing one or more machine-learned models can be trained in an end-to-end manner. Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations. In some implementations, performing backwards propagation of errors can include performing truncated backpropagation through time. Example method 800 can include implementing a number of generalization techniques (e.g., weight decays, dropouts, etc.) to improve the generalization capability of the models being trained.

[0298] In some implementations, example method 800 can be implemented for training a machine-learned model from an initialized state to a fully trained state (e.g., when the model exhibits a desired performance profile, such as based on accuracy, precision, recall, etc.).

[0299] In some implementations, example method 800 can be implemented for particular stages of a training procedure. For instance, in some implementations, example method 800 can be implemented for pre-training a machine-learned model. Pre-training can include, for instance, large-scale training over potentially noisy data to achieve a broad base of performance levels across a variety of tasks / data types. In some implementations, example method 800 can be implemented for fine-tuning a machine-learned model. Fine-tuning can include, for instance, smaller-scale training on higher-quality (e.g., labeled, curated, etc.) data. Fine-tuning can affect all or a portion of the parameters of a machine-learned model. For example, various portions of the machine-learned model can be “frozen” for certain training stages. For example, parameters associated with an embedding space can be “frozen” during fine-tuning (e.g., to retain information learned from a broader domain(s) than present in the fine-tuning dataset(s)). An example fine-tuning approach includes reinforcement learning. Reinforcement learning can be based on user feedback on model performance during use.Example Machine-Learned Models

[0300] Figure 9 is a block diagram of an example processing flow for using machine- learned model(s) 1 to process input(s) 2 to generate output(s) 3.

[0301] Machine-learned model(s) 1 can be or include one or multiple machine- learned models or model components. Example machine-learned models can include neural networks (e.g., deep neural networks). Example machine-learned models can include nonlinear models or linear models. Example machine-learned models can use other architectures in lieu of or in addition to neural networks. Example machine-learned models can include decision tree based models, support vector machines, hidden Markov models, Bayesian networks, linear regression models, k-means clustering models, etc.

[0302] Example neural networks can include feed-forward neural networks, recurrent neural networks (RNNs), including long short-term memory (LSTM) based recurrent neural networks, convolutional neural networks (CNNs), diffusion models, generative-adversarial networks, or other forms of neural networks. Example neural networks can be deep neural networks. Some example machine-learned models can leverage an attention mechanism such as self-attention. For example, some example machine-learned models can include multiheaded self-attention models.

[0303] Machine-learned model(s) 1 can include a single or multiple instances of the same model configured to operate on data from input(s) 2. Machine-learned model(s) 1 can include an ensemble of different models that can cooperatively interact to process data from input(s) 2. For example, machine-learned model(s) 1 can employ a mixture-of-experts structure. See, e.g., Zhou et al., Mixture-of-Experts with Expert Choice Routing, ARXIV:2202.09368V2 (Oct. 14, 2022).

[0304] Input(s) 2 can generally include or otherwise represent various types of data. Input(s) 2 can include one type or many different types of data. Output(s) 3 can be data of the same type(s) or of different types of data as compared to input(s) 2. Output(s) 3 can include one tope or many different types of data.

[0305] Example data t pes for input(s) 2 or output(s) 3 include natural language text data, software code data (e.g., source code, object code, machine code, or any other form of computer-readable instructions or programming languages), machine code data (e g., binary code, assembly code, or other forms of machine-readable instructions that can be executed directly by a computer’s central processing unit), assembly code data (e.g., low-level programming languages that use symbolic representations of machine code instructions to program a processing unit), genetic data or other chemical or biochemical data, image data.audio data, audiovisual data, haptic data, biometric data, medical data, financial data, statistical data, geographical data, astronomical data, historical data, sensor data generally (e.g., digital or analog values, such as voltage or other absolute or relative level measurement values from a real or artificial input, such as from an audio sensor, light sensor, displacement sensor, etc.), and the like. Data can be raw or processed and can be in any format or schema.

[0306] In multimodal inputs 2 or outputs 3, example combinations of data types include image data and audio data, image data and natural language data, natural language data and software code data, image data and biometric data, sensor data and medical data, etc. It is to be understood that any combination of data types in an input 2 or an output 3 can be present.

[0307] An example input 2 can include one or multiple data types, such as the example data types noted above. An example output 3 can include one or multiple data types, such as the example data types noted above. The data type(s) of input 2 can be the same as or different from the data type(s) of output 3. It is to be understood that the example data types noted above are provided for illustrative purposes only. Data types contemplated within the scope of the present disclosure are not limited to those examples noted above.Example Machine-Learned Sequence Processing Models

[0308] Figure 10 is a block diagram of an example implementation of an example machine-learned model configured to process sequences of information. For instance, an example implementation of machine-learned model(s) 1 can include machine-learned sequence processing model(s) 4. An example system can pass input(s) 2 to sequence processing model(s) 4. Sequence processing model(s) 4 can include one or more machine- learned components. Sequence processing model(s) 4 can process the data from input(s) 2 to obtain an input sequence 5. Input sequence 5 can include one or more input elements 5-1, 5- 2, . . . , 5-M, etc. obtained from input(s) 2. Sequence processing model 4 can process input sequence 5 using prediction layer(s) 6 to generate an output sequence 7. Output sequence 7 can include one or more output elements 7-1, 7-2. . . . , 7 -A, etc. generated based on input sequence 5. The system can generate output(s) 3 based on output sequence 7.

[0309] Sequence processing model(s) 4 can include one or multiple machine-learned model components configured to ingest, generate, or otherwise reason over sequences of information. For example, some example sequence processing models in the text domain are referred to as “Large Language Models / ’ or LLMs. See. e.g., PaLM 2 Technical Report, GOOGLE, https: / / ai.google / static / documents / palm2techreport.pdf (n d ). Other examplesequence processing models can operate in other domains, such as image domains, see, e.g, Dosovitskiy et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, ARXIV:2010.1 1929V2 (Jun. 3, 2021), audio domains, see, e.g., Agostinelli et al., MusicLM: Generating Music From Text, ARXlV:2301.11325vl (Jan. 26, 2023), biochemical domains, see, e.g, Jumper et al., Highly accurate protein structure prediction with AlphaFold, 596 Nature 583 (Aug. 26, 2021), by way of example. Sequence processing model(s) 4 can process one or multiple types of data simultaneously. Sequence processing model(s) 4 can include relatively large models (e.g., more parameters, computationally expensive, etc.), relatively small models (e.g., fewer parameters, computationally lightweight, etc.), or both.

[0310] In general, sequence processing model(s) 4 can obtain input sequence 5 using data from input(s) 2. For instance, input sequence 5 can include a representation of data from input(s) 2 in a format understood by sequence processing model(s) 4. One or more machine- learned components of sequence processing model(s) 4 can ingest the data from input(s) 2, parse the data into pieces compatible with the processing architectures of sequence processing model(s) 4 (e.g., via “tokenization”). and project the pieces into an input space associated with prediction layer(s) 6 (e.g.. via “embedding”).

[0311] Sequence processing model(s) 4 can ingest the data from input(s) 2 and parse the data into a sequence of elements to obtain input sequence 5. For example, a portion of input data from input(s) 2 can be broken down into pieces that collectively represent the content of the portion of the input data. The pieces can provide the elements of the sequence.

[0312] Elements 5-1 , 5-2, . . . , 5- can represent, in some cases, building blocks for capturing or expressing meaningful information in a particular data domain. For instance, the elements can describe “atomic units” across one or more domains. For example, for textual input source(s), the elements can correspond to groups of one or more words or sub-word components, such as sets of one or more characters.

[0313] For example, elements 5-1, 5-2, . . . , 5-M can represent tokens obtained using a tokenizer. For instance, a tokenizer can process a given portion of an input source and output a series of tokens (e.g., corresponding to input elements 5-1. 5-2, . . . , 5-M) that represent the portion of the input source. Various approaches to tokenization can be used. For instance, textual input source(s) can be tokenized using a byte-pair encoding (BPE) technique. See, e.g., Kudo et al., SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing, PROCEEDINGS OF THE 2018 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (System Demonstrations), pages 66-71 (October 31-November 4, 2018),https: / / aclanthology.org / D18-2012.pdf. Image-based input source(s) can be tokenized by extracting and serializing patches from an image.

[0314] In general, arbitrary data types can be serialized and processed into input sequence 5. It is to be understood that element(s) 5-1, 5-2, . . . , 5-M depicted in Figure 10 can be the tokens or can be the embedded representations thereof.

[0315] Prediction layer(s) 6 can predict one or more output elements 7-1, 7-2, . . . , 7- N based on the input elements. Prediction layer(s) 6 can include one or more machine-learned model architectures, such as one or more layers of learned parameters that manipulate and transform the input(s) to extract higher-order meaning from, and relationships between, input element(s) 5-1, 5-2, . . . , 5-M. In this manner, for instance, example prediction layer(s) 6 can predict new output element(s) in view of the context provided by input sequence 5.

[0316] Prediction layer(s) 6 can evaluate associations between portions of input sequence 5 and a particular output element. These associations can inform a prediction of the likelihood that a particular output follows the input context. For example, consider the textual snippet, “The carpenter’s toolbox was small and heavy. It was full of .” Example prediction layer(s) 6 can identify that “It” refers back to “toolbox” by determining a relationship between the respective embeddings. Example prediction layer(s) 6 can also link “It” to the attributes of the toolbox, such as “small” and “heavy.” Based on these associations, prediction layer(s) 6 can, for instance, assign a higher probability to the word “nails” than to the word “sawdust.”

[0317] A transformer is an example architecture that can be used in prediction layer(s) 4. See, e.g., Vaswani et al., Attention Is All You Need, ARXlV:1706.03762v7 (Aug. 2, 2023). A transformer is an example of a machine-learned model architecture that uses an attention mechanism to compute associations between items within a context window. The context window can include a sequence that contains input sequence 5 and potentially one or more output element(s) 7-1, 7-2, . . . , 1-N. A transformer block can include one or more attention layer(s) and one or more post-attention layer(s) (e.g., feedforw ard layer(s), such as a multi-layer perceptron).

[0318] Prediction layer(s) 6 can include other machine-learned model architectures in addition to or in lieu of transformer-based architectures. For example, recurrent neural networks (RNNs) and long short-term memory (LSTM) models can also be used, as well as convolutional neural networks (CNNs). In general, prediction layer(s) 6 can leverage various kinds of artificial neural networks that can understand or generate sequences of information.

[0319] Output sequence 7 can include or otherwise represent the same or different data types as input sequence 5. For instance, input sequence 5 can represent textual data, and output sequence 7 can represent textual data. Input sequence 5 can represent image, audio, or audiovisual data, and output sequence 7 can represent textual data (e.g., describing the image, audio, or audiovisual data). It is to be understood that prediction layer(s) 6, and any other interstitial model components of sequence processing model(s) 4. can be configured to receive a variety of data types in input sequence(s) 5 and output a variety of data types in output sequence(s) 7.

[0320] Output sequence 7 can have various relationships to input sequence 5. Output sequence 7 can be a continuation of input sequence 5. Output sequence 7 can be complementary to input sequence 5. Output sequence 7 can translate, transform, augment, or otherwise modify input sequence 5. Output sequence 7 can answer, evaluate, confirm, or otherwise respond to input sequence 5. Output sequence 7 can implement (or describe instructions for implementing) an instruction provided via input sequence 5.

[0321] Output sequence 7 can be generated autoregressively. For instance, for some applications, an output of one or more prediction layer(s) 6 can be passed through one or more output layers (e.g., softmax layer) to obtain a probability distribution over an output vocabulary' (e.g., a textual or symbolic vocabulary ) conditioned on a set of input elements in a context window. In this manner, for instance, output sequence 7 can be autoregressively generated by sampling a likely next output element, adding that element to the context window, and re-generating the probability distribution based on the updated context window', and sampling a likely next output element, and so forth.

[0322] Output sequence 7 can also be generated non-autoregressively. For instance, multiple output elements of output sequence 7 can be predicted together without explicit sequential conditioning on each other. See, e.g., Saharia et al., Non-Autoregressive Machine Translation with Latent Alignments, ARXlV:2004.07437v3 (NOV. 16, 2020).

[0323] Output sequence 7 can include one or multiple portions or elements. In an example content generation configuration, output sequence 7 can include multiple elements corresponding to multiple portions of a generated output sequence (e.g., a textual sentence, values of a discretized w aveform, computer code, etc.). In an example classification configuration, output sequence 7 can include a single element associated with a classification output. For instance, an output “vocabulary"’ can include a set of classes into which an input sequence is to be classified. For instance, a vision transformer block can pass latent stateinformation to a multilayer perceptron that outputs a likely class value associated with an input image.

[0324] Figure 11 is a block diagram of an example technique for populating an example input sequence 8. Input sequence 8 can include various functional elements that form part of the model infrastructure, such as an element 8-0 obtained from a task indicator 9 that signals to any model(s) that process input sequence 8 that a particular task is being performed (e.g., to help adapt a performance of the model(s) to that particular task). Input sequence 8 can include various data elements from different data modalities. For instance, an input modality 10-1 can include one modality of data. A data-to-sequence model 11-1 can process data from input modality 10-1 to project the data into a format compatible with input sequence 8 (e.g.. one or more vectors dimensioned according to the dimensions of input sequence 8) to obtain elements 8-1, 8-2, 8-3. Another input modality 10-2 can include a different modality of data. A data-to-sequence model 11-2 can project data from input modality 10-2 into a format compatible with input sequence 8 to obtain elements 8-4, 8-5, 8- 6. Another input modality 10-3 can include yet another different modality of data. A data-to- sequence model 11-3 can project data from input modality 10-3 into a format compatible with input sequence 8 to obtain elements 8-7, 8-8, 8-9.

[0325] Input sequence 8 can be the same as or different from input sequence 5. Input sequence 8 can be a multimodal input sequence that contains elements that represent data from different modalities using a common dimensional representation. For instance, an embedding space can have P dimensions. Input sequence 8 can be configured to contain a plurality of elements that have P dimensions. In this manner, for instance, example implementations can facilitate information extraction and reasoning across diverse data modalities by projecting data into elements in the same embedding space for comparison, combination, or other computations therebetween.

[0326] For example, elements 8-0, . . . , 8-9 can indicate particular locations within a multidimensional embedding space. Some elements can map to a set of discrete locations in the embedding space. For instance, elements that correspond to discrete members of a predetermined vocabulary of tokens can map to discrete locations in the embedding space that are associated with those tokens. Other elements can be continuously distributed across the embedding space. For instance, some datatypes can be broken down into continuously defined portions (e.g., image patches) that can be described using continuously distributed locations within the embedding space.

[0327] In some implementations, the expressive power of the embedding space may not be limited to meanings associated with any particular set of tokens or other building blocks. For example, a continuous embedding space can encode a spectrum of high-order information. An individual piece of information (e.g., a token) can map to a particular point in that space: for instance, a token for the word “dog” can be projected to an embedded value that points to a particular location in the embedding space associated with canine-related information. Similarly, an image patch of an image of a dog on grass can also be projected into the embedding space. In some implementations, the projection of the image of the dog can be similar to the projection of the word “dog” while also having similarity to a projection of the word “grass.” while potentially being different from both. In some implementations, the projection of the image patch may not exactly align with any single projection of a single word. In some implementations, the projection of the image patch can align with a combination of the projections of the words “dog” and “grass.” In this manner, for instance, a high-order embedding space can encode information that can be independent of data modalities in which the information is expressed.

[0328] Task indicator 9 can include a model or model component configured to identify a task being performed and inject, into input sequence 8, an input value represented by element 8-0 that signals which task is being performed. For instance, the input value can be provided as a data type associated with an input modality and projected along with that input modality (e.g., the input value can be a textual task label that is embedded along with other textual data in the input; the input value can be a pixel-based representation of a task that is embedded along with other image data in the input; etc.). The input value can be provided as a data type that differs from or is at least independent from other input(s). For instance, the input value represented by element 8-0 can be a learned within a continuous embedding space.

[0329] Input modalities 10-1, 10-2, and 10-3 can be associated with various different data types (e.g., as described above with respect to input(s) 2 and output(s) 3).

[0330] Data-to-sequence models 11-1. 11-2. and 11-3 can be the same or different from each other. Data-to-sequence models 11 -I, 11-2, and 11-3 can be adapted to each respective input modality 10-1, 10-2, and 10-3. For example, a textual data-to-sequence model can subdivide a portion of input text and project the subdivisions into element(s) in input sequence 8 (e.g., elements 8-1. 8-2, 8-3, etc.). An image data-to-sequence model can subdivide an input image and project the subdivisions into element(s) in input sequence 8 (e.g., elements 8-4, 8-5, 8-6, etc.). An arbitrary datatype data-to-sequence model cansubdivide an input of that arbitrary datatype and project the subdivisions into element(s) in input sequence 8 (e.g., elements 8-7. 8-8, 8-9, etc.).

[0331] Data-to-sequence models 1 1-1, 1 1-2, and 11-3 can form part of machine- learned sequence processing model (s) 4. Data-to-sequence models 11-1, 11-2, and 11-3 can be jointly trained with or trained independently from machine-learned sequence processing model(s) 4. Data-to-sequence models 11-1, 11-2, and 11-3 can be trained end-to-end with machine-learned sequence processing model(s) 4.Example Machine-Learned Model Development Platform

[0332] Figure 12 is a block diagram of an example model development platform 12 that can facilitate creation, adaptation, and refinement of example machine-learned models (e.g., machine-learned model(s) 1, sequence processing model(s) 4, etc ). Model development platform 12 can provide a number of different toolkits that developer systems can employ in the development of new or adapted machine-learned models.

[0333] Model development platform 12 can provide one or more model libraries 13 containing building blocks for new models. Model libraries 13 can include one or more pretrained foundational models 13-1, which can provide a backbone of processing power across various tasks. Model libraries 13 can include one or more pre-trained expert models 13-2, which can be focused on performance in particular domains of expertise. Model libraries 13 can include various model primitives 13-3, which can provide low-level architectures or components (optionally pre-trained), which can be assembled in various arrangements as desired.

[0334] Model development platform 12 can receive selections of various model components 14. Model development platform 12 can pass selected model components 14 to a workbench 15 that combines selected model components 14 into a development model 16.

[0335] Workbench 15 can facilitate further refinement and adaptation of development model 16 by leveraging a number of different toolkits integrated with model development platform 12. For example, workbench 15 can facilitate alignment of the development model 16 with a desired performance profile on various tasks using a model alignment toolkit 17.

[0336] Model alignment toolkit 17 can provide a number of tools for causing development model 16 to generate outputs aligned with desired behavioral characteristics. Alignment can include increasing an accuracy, precision, recall, etc. of model outputs.Alignment can include enforcing output styles, schema, or other preferential characteristics of model outputs. Alignment can be general or domain-specific. For instance, a pre-trainedfoundational model 13-1 can begin with an initial level of performance across multiple domains. Alignment of the pre-trained foundational model 13-1 can include improving a performance in a particular domain of information or tasks (e.g., even at the expense of performance in another domain of information or tasks).

[0337] Model alignment toolkit 17 can integrate one or more dataset(s) 17-1 for aligning development model 16. Curated dataset(s) 17-1 can include labeled or unlabeled training data. Dataset(s) 17-1 can be obtained from public domain datasets. Dataset(s) 17-1 can be obtained from private datasets associated with one or more developer system(s) for the alignment of bespoke machine-learned model(s) customized for private use-cases.

[0338] Pre-training pipelines 17-2 can include a machine-learned model training workflow configured to update development model 16 over large-scale, potentially noisy datasets. For example, pre-training can leverage unsupervised learning techniques (e.g., denoising, etc.) to process large numbers of training instances to update model parameters from an initialized state and achieve a desired baseline performance. Pre- training pipelines 17-2 can leverage unlabeled datasets in dataset(s) 17-1 to perform pre-training. Workbench 15 can implement a pre-training pipeline 17-2 to pre-train development model 16.

[0339] Fine-tuning pipelines 17-3 can include a machine-learned model training workflow configured to refine the model parameters of development model 16 with higher- quality data. Fine-tuning pipelines 17-3 can update development model 16 by conducting supervised training with labeled dataset(s) in dataset(s) 17-1. Fine-tuning pipelines 17-3 can update development model 16 by conducting reinforcement learning using reward signals from user feedback signals. Workbench 15 can implement a fine-tuning pipeline 17-3 to finetune development model 16.

[0340] Prompt libraries 17-4 can include sets of inputs configured to induce behavior aligned with desired performance criteria. Prompt libraries 17-4 can include few-shot prompts (e.g., inputs providing examples of desired model outputs for prepending to a desired runtime query), chain-of-thought prompts (e.g., inputs providing step-by-step reasoning within the exemplars to facilitate thorough reasoning by the model), and the like.

[0341] Example prompts can be retrieved from an available repository of prompt libraries 17-4. Example prompts can be contributed by one or more developer systems using workbench 15.

[0342] In some implementations, pre-trained or fine-tuned models can achieve satisfactory performance without exemplars in the inputs. For instance, zero-shot prompts caninclude inputs that lack exemplars. Zero-shot prompts can be within a domain within a training dataset or outside of the training domain(s).

[0343] Prompt libraries 17-4 can include one or more prompt engineering tools. Prompt engineering tools can provide workflows for retrieving or learning optimized prompt values. Prompt engineering tools can facilitate directly learning prompt values (e.g., input element values) based one or more training iterations. Workbench 15 can implement prompt engineering tools in development model 16.

[0344] Prompt libraries 17-4 can include pipelines for prompt generation. For example, inputs can be generated using development model 16 itself or other machine- learned models. In this manner, for instance, a first model can process information about a task and output a input for a second model to process in order to perform a step of the task. The second model can be the same as or different from the first model. Workbench 15 can implement prompt generation pipelines in development model 16.

[0345] Prompt libraries 17-4 can include pipelines for context injection. For instance, a performance of development model 16 on a particular task can improve if provided with additional context for performing the task. Prompt libraries 17-4 can include software components configured to identify desired context, retrieve the context from an external source (e.g., a database, a sensor, etc.), and add the context to the input prompt. Workbench 15 can implement context injection pipelines in development model 16.

[0346] Although various training examples described herein with respect to model development platform 12 refer to ‘'pre-training” and “fine-tuning,” it is to be understood that model alignment toolkit 17 can generally support a wide variety of training techniques adapted for training a wide variety of machine-learned models. Example training techniques can correspond to the example training method 800 described above.

[0347] Model development platform 12 can include a model plugin toolkit 18. Model plugin toolkit 18 can include a variety of tools configured for augmenting the functionality' of a machine-learned model by integrating the machine-learned model w ith other systems, devices, and software components. For instance, a machine-learned model can use tools to increase performance quality where appropriate. For instance, deterministic tasks can be offloaded to dedicated tools in lieu of probabilistically performing the task with an increased risk of error. For instance, instead of autoregressively predicting the solution to a system of equations, a machine-learned model can recognize a tool to call for obtaining the solution and pass the system of equations to the appropriate tool. The tool can be a traditional system of equations solver that can operate deterministically to resolve the system of equations. Theoutput of the tool can be returned in response to the original query. In this manner, tool use can allow some example models to focus on the strengths of machine-learned models — e.g., understanding an intent in an unstructured request for a task — while augmenting the performance of the model by offloading certain tasks to a more focused tool for rote application of deterministic algorithms to a well-defined problem.

[0348] Model plugin toolkit 18 can include validation tools 18-1. Validation tools 18- 1 can include tools that can parse and confirm output(s) of a machine-learned model.Validation tools 18-1 can include engineered heuristics that establish certain thresholds applied to model outputs. For example, validation tools 18-1 can ground the outputs of machine-learned models to structured data sources (e.g., to mitigate ‘‘hallucinations’').

[0349] Model plugin toolkit 18 can include tooling packages 18-2 for implementing one or more tools that can include scripts or other executable code that can be executed alongside development model 16. Tooling packages 18-2 can include one or more inputs configured to cause machine-learned model(s) to implement the tools (e.g., few-shot prompts that induce a model to output tool calls in the proper syntax, etc.). Tooling packages 18-2 can include, for instance, fine-tuning training data for training a model to use a tool.

[0350] Model plugin toolkit 18 can include interfaces for calling external application programming interfaces (APIs) 18-3. For instance, in addition to or in lieu of implementing tool calls or tool code directly with development model 16, development model 16 can be aligned to output instruction that initiate API calls to send or obtain data via external systems.

[0351] Model plugin toolkit 18 can integrate with prompt libraries 17-4 to build a catalog of available tools for use with development model 16. For instance, a model can receive, in an input, a catalog of available tools, and the model can generate an output that selects a tool from the available tools and initiates a tool call for using the tool.

[0352] Model development platform 12 can include a computational optimization toolkit 19 for optimizing a computational performance of development model 16. For instance, tools for model compression 19-1 can allow development model 16 to be reduced in size while maintaining a desired level of performance. For instance, model compression 19-1 can include quantization workflows, weight pruning and sparsification techniques, etc. Tools for hardware acceleration 19-2 can facilitate the configuration of the model storage and execution formats to operate optimally on different hardware resources. For instance, hardware acceleration 19-2 can include tools for optimally sharding models for distributed processing over multiple processing units for increased bandwidth, lower unified memory requirements, etc. Tools for distillation 19-3 can provide for the training of lighter-weightmodels based on the knowledge encoded in development model 16. For instance, development model 16 can be a highly performant, large machine-learned model optimized using model development platform 12. To obtain a lightweight model for running in resource-constrained environments, a smaller model can be a “student model” that learns to imitate development model 16 as a “teacher model.” In this manner, for instance, the investment in learning the parameters and configurations of development model 16 can be efficiently transferred to a smaller model for more efficient inference.

[0353] Workbench 15 can implement one, multiple, or none of the toolkits implemented in model development platform 12. Workbench 15 can output an output model 20 based on development model 16. Output model 20 can be a deployment version of development model 16. Output model 20 can be a development or training checkpoint of development model 16. Output model 20 can be a distilled, compressed, or otherwise optimized version of development model 16.

[0354] Figure 13 is a block diagram of an example training flow for training a machine-learned development model 16. One or more portion(s) of the example training flow can be implemented by a computing system that includes one or more computing devices such as, for example, computing systems described with reference to the other figures. Each respective portion of the example training flow can be performed by any (or any combination) of one or more computing devices. Moreover, one or more portion(s) of the example training flow can be implemented on the hardware components of the device(s) described herein, for example, to train one or more systems or models. FIG. 13 depicts elements performed in a particular order for purposes of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the methods discussed herein can be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. FIG. 13 is described with reference to elements / terms described with respect to other systems and figures for exemplar}' illustrated purposes and is not meant to be limiting. One or more portions of the example training flow can be performed additionally, or alternatively, by other systems.

[0355] Initially, development model 16 can persist in an initial state as an initialized model 21. Development model 16 can be initialized with weight values. Initial weight values can be random or based on an initialization schema. Initial weight values can be based on prior pre-training for the same or for a different model.

[0356] Initialized model 21 can undergo pre-training in a pre-training stage 22. Pretraining stage 22 can be implemented using one or more pre-training pipelines 17-2 over data from dataset(s) 17-1. Pre-training can be omitted, for example, if initialized model 21 is already pre-trained (e.g., development model 16 contains, is, or is based on a pre-trained foundational model or an expert model).

[0357] Pre-trained model 23 can then be a new version of development model 16, which can persist as development model 16 or as a new development model. Pre-trained model 23 can be the initial state if development model 16 was already pre-trained. Pre-trained model 23 can undergo fine-tuning in a fine-tuning stage 24. Fine-tuning stage 24 can be implemented using one or more fine-tuning pipelines 17-3 over data from dataset(s) 17-1. Fine-tuning can be omitted, for example, if a pre-trained model as satisfactory performance, if the model was already fine-tuned, or if other tuning approaches are preferred.

[0358] Fine-tuned model 29 can then be a new version of development model 16, which can persist as development model 16 or as a new development model. Fine-tuned model 29 can be the initial state if development model 16 was already fine-tuned. Fine-tuned model 29 can undergo refinement with user feedback 26. For instance, refinement with user feedback 26 can include reinforcement learning, optionally based on human feedback from human users of fine-tuned model 25. As reinforcement learning can be a form of fine-tuning, it is to be understood that fine-tuning stage 24 can subsume the stage for refining with user feedback 26. Refinement with user feedback 26 can produce a refined model 27. Refined model 27 can be output to downstream system(s) 28 for deployment or further development.

[0359] In some implementations, computational optimization operations can be applied before, during, or after each stage. For instance, initialized model 21 can undergo computational optimization 29-1 (e.g.. using computational optimization toolkit 19) before pre-training stage 22. Pre-trained model 23 can undergo computational optimization 29-2 (e.g., using computational optimization toolkit 19) before fine-tuning stage 24. Fine-tuned model 25 can undergo computational optimization 29-3 (e.g., using computational optimization toolkit 19) before refinement with user feedback 26. Refined model 27 can undergo computational optimization 29-4 (e.g., using computational optimization toolkit 19) before output to downstream system(s) 28. Computational optimization(s) 29-1, . . . , 29-4 can all be the same, all be different, or include at least some different optimization techniques.Example Machine-Learned Model Inference System

[0360] Figure 14 is a block diagram of an inference system for operating one or more machine-learned model(s) 1 to perform inference (e.g., for training, for deployment, etc.). A model host 31 can receive machine-learned model(s) 1. Model host 31 can host one or more model instance(s) 31-1, which can be one or multiple instances of one or multiple models. Model host 31 can host model instance(s) 31-1 using available compute resources 31-2 associated with model host 31.

[0361] Model host 31 can perform inference on behalf of one or more client(s) 32. Client(s) 32 can transmit an input request 33 to model host 31. Using input request 33, model host 31 can obtain input(s) 2 for input to machine-learned model(s) 1. Machine-learned model(s) 1 can process input(s) 2 to generate output(s) 3. Using output(s) 3, model host 31 can return an output payload 34 for responding to input request 33 from client(s) 32. Output payload 34 can include or be based on output(s) 3.

[0362] Model host 31 can leverage various other resources and tools to augment the inference task. For instance, model host 31 can communicate with tool interfaces 35 to facilitate tool use by model instance(s) 31-1. Tool interfaces 35 can include local or remote APIs. Tool interfaces 35 can include integrated scripts or other software functionality. Model host 31 can engage online learning interface(s) 36 to facilitate ongoing improvements to machine-learned model(s) 1. For instance, online learning interface(s) 36 can be used within reinforcement learning loops to retrieve user feedback on inferences served by model host 31. Model host 31 can access runtime data source(s) 37 for augmenting input(s) 2 with additional contextual information. For instance, runtime data source(s) 37 can include a knowledge graph 37-1 that facilitates structured information retrieval for information associated with input request(s) 33 (e.g., a search engine service). Runtime data source(s) 37 can include public or private, external or local database(s) 37-2 that can store information associated with input request(s) 33 for augmenting input(s) 2. Runtime data source(s) 37 can include account data 37-3 which can be retrieved in association with a user account corresponding to a client 32 for customizing the behavior of model host 31 accordingly.

[0363] Model host 31 can be implemented by one or multiple computing devices or systems. Client(s) 2 can be implemented by one or multiple computing devices or systems, which can include computing devices or systems shared with model host 31.

[0364] For example, model host 31 can operate on a server system that provides a machine-learning service to client device(s) that operate client(s) 32 (e.g., over a local or wide-area network). Client device(s) can be end-user devices used by individuals. Clientdevice(s) can be server systems that operate client(s) 32 to provide various functionality as a service to downstream end-user devices.

[0365] In some implementations, model host 31 can operate on a same device or system as client(s) 32. Model host 31 can be a machine-learning service that runs on-device to provide machine-learning functionality to one or multiple applications operating on a client device, which can include an application implementing client(s) 32. Model host 31 can be a part of a same application as client(s) 32. For instance, model host 31 can be a subroutine or method implemented by one part of an application, and client(s) 32 can be another subroutine or method that engages model host 31 to perform inference functions within the application. It is to be understood that model host 31 and client(s) 32 can have various different configurations.

[0366] Model instance(s) 31-1 can include one or more machine-learned models that are available for performing inference. Model instance(s) 31-1 can include weights or other model components that are stored on in persistent storage, temporarily cached, or loaded into high-speed memory. Model instance(s) 31-1 can include multiple instance(s) of the same model (e.g., for parallel execution of more requests on the same model). Model instance(s) 31-1 can include instance(s) of different model(s). Model instance(s) 31-1 can include cached intermediate states of active or inactive model(s) used to accelerate inference of those models. For instance, an inference session with a particular model may generate significant amounts of computational results that can be re-used for future inference runs (e.g., using a KV cache for transformer-based models). These computational results can be saved in association with that inference session so that session can be executed more efficiently when resumed.

[0367] Compute resource(s) 31-2 can include one or more processors (central processing units, graphical processing units, tensor processing units, machine-learning accelerators, etc.) connected to one or more memory devices. Compute resource(s) 31-2 can include a dynamic pool of available resources shared with other processes. Compute resource(s) 31-2 can include memory devices large enough to fit an entire model instance in a single memory instance. Compute resource(s) 31-2 can also shard model instance(s) across multiple memon devices (e.g., using data parallelization or tensor parallelization, etc ). This can be done to increase parallelization or to execute a large model using multiple memory devices which individually might not be able to fit the entire model into memory.

[0368] Input request 33 can include data for input(s) 2. Model host 31 can process input request 33 to obtain input(s) 2. Input(s) 2 can be obtained directly from input request 33or can be retrieved using input request 33. Input request 33 can be submitted to model host 31 via an API.

[0369] Model host 31 can perform inference over batches of input requests 33 in parallel. For instance, a model instance 31-1 can be configured with an input structure that has a batch dimension. Separate input(s) 2 can be distributed across the batch dimension (e.g., rows of an array). The separate input(s) 2 can include completely different contexts. The separate input(s) 2 can be multiple inference steps of the same task. The separate input(s) 2 can be staggered in an input structure, such that any given inference cycle can be operating on different portions of the respective input(s) 2. In this manner, for instance, model host 31 can perform inference on the batch in parallel, such that output(s) 3 can also contain the batch dimension and return the inference results for the batched input(s) 2 in parallel. In this manner, for instance, batches of input request(s) 33 can be processed in parallel for higher throughput of output payload(s) 34.

[0370] Output payload 34 can include or be based on output(s) 3 from machine- learned model(s) 1. Model host 31 can process output(s) 3 to obtain output payload 34. This can include chaining multiple rounds of inference (e.g.. iteratively, recursively, across the same model(s) or different model(s)) to arrive at a final output for a task to be returned in output payload 34. Output payload 34 can be transmitted to client(s) 32 via an API.

[0371] Online learning interface(s) 36 can facilitate reinforcement learning of machine-learned model(s) 1. Online learning interface(s) 36 can facilitate reinforcement learning with human feedback (RLHF). Online learning interface(s) 36 can facilitate federated learning of machine-learned model(s) 1.

[0372] Model host 31 can execute machine-learned model(s) 1 to perform inference for various tasks using various t pes of data. For example, various different input(s) 2 and output(s) 3 can be used for various different tasks. In some implementations, input(s) 2 can be or otherwise represent image data. Machine-learned model(s) 1 can process the image data to generate an output. As an example, machine-learned model(s) 1 can process the image data to generate an image recognition output (e.g., a recognition of the image data, a latent embedding of the image data, an encoded representation of the image data, a hash of the image data, etc.). As another example, machine-learned model(s) 1 can process the image data to generate an image segmentation output. As another example, machine-learned model(s) 1 can process the image data to generate an image classification output. As another example, machine-learned model(s) 1 can process the image data to generate an image data modification output (e.g., an alteration of the image data, etc.). As another example, machine-learned model(s) 1 can process the image data to generate an encoded image data output (e.g., an encoded and / or compressed representation of the image data, etc.). As another example, machine-learned model(s) 1 can process the image data to generate an upscaled image data output. As another example, machine-learned model(s) 1 can process the image data to generate a prediction output.

[0373] In some implementations, the task is a computer vision task. In some cases, input(s) 2 includes pixel data for one or more images and the task is an image processing task. For example, the image processing task can be image classification, where the output is a set of scores, each score corresponding to a different object class and representing the likelihood that the one or more images depict an object belonging to the object class. The image processing task may be object detection, where the image processing output identifies one or more regions in the one or more images and, for each region, a likelihood that region depicts an object of interest. As another example, the image processing task can be image segmentation, where the image processing output defines, for each pixel in the one or more images, a respective likelihood for each category in a predetermined set of categories. For example, the set of categories can be foreground and background. As another example, the set of categories can be object classes. As another example, the image processing task can be depth estimation, where the image processing output defines, for each pixel in the one or more images, a respective depth value. As another example, the image processing task can be motion estimation, where the network input includes multiple images, and the image processing output defines, for each pixel of one of the input images, a motion of the scene depicted at the pixel between the images in the network input.

[0374] In some implementations, input(s) 2 can be or otherwise represent natural language data. Machine-learned model(s) 1 can process the natural language data to generate an output. As an example, machine-learned model(s) 1 can process the natural language data to generate a language encoding output. As another example, machine-learned model(s) 1 can process the natural language data to generate a latent text embedding output. As another example, machine-learned model(s) 1 can process the natural language data to generate a translation output. As another example, machine-learned model(s) 1 can process the natural language data to generate a classification output. As another example, machine-learned model(s) 1 can process the natural language data to generate a textual segmentation output. As another example, machine-learned model(s) 1 can process the natural language data to generate a semantic intent output. As another example, machine-learned model(s) 1 can process the natural language data to generate an upscaled text or natural language output(e.g., text or natural language data that is higher quality than the input text or natural language, etc.). As another example, machine-learned model(s) 1 can process the natural language data to generate a prediction output (e.g., one or more predicted next portions of natural language content).

[0375] In some implementations, input(s) 2 can be or otherwise represent speech data (e.g., data describing spoken natural language, such as audio data, textual data, etc.). Machine-learned model(s) 1 can process the speech data to generate an output. As an example, machine-learned model(s) 1 can process the speech data to generate a speech recognition output. As another example, machine-learned model(s) 1 can process the speech data to generate a speech translation output. As another example, machine-learned model(s) 1 can process the speech data to generate a latent embedding output. As another example, machine-learned model(s) 1 can process the speech data to generate an encoded speech output (e.g., an encoded and / or compressed representation of the speech data, etc.). As another example, machine-learned model(s) 1 can process the speech data to generate an upscaled speech output (e.g., speech data that is higher quality than the input speech data, etc.). As another example, machine-learned model(s) 1 can process the speech data to generate a textual representation output (e g., a textual representation of the input speech data, etc.). As another example, machine-learned model(s) 1 can process the speech data to generate a prediction output.

[0376] In some implementations, input(s) 2 can be or otherwise represent latent encoding data (e.g., a latent space representation of an input, etc ). Machine-learned model(s) 1 can process the latent encoding data to generate an output. As an example, machine- learned model(s) 1 can process the latent encoding data to generate a recognition output. As another example, machine-learned model(s) 1 can process the latent encoding data to generate a reconstruction output. As another example, machine-learned model(s) 1 can process the latent encoding data to generate a search output. As another example, machine- learned model(s) 1 can process the latent encoding data to generate a reclustering output. As another example, machine-learned model(s) 1 can process the latent encoding data to generate a prediction output.

[0377] In some implementations, input(s) 2 can be or otherwise represent statistical data. Statistical data can be, represent, or otherwise include data computed and / or calculated from some other data source. Machine-learned model(s) 1 can process the statistical data to generate an output. As an example, machine-learned model(s) 1 can process the statistical data to generate a recognition output. As another example, machine-learned model(s) 1 canprocess the statistical data to generate a prediction output. As another example, machine- learned model(s) 1 can process the statistical data to generate a classification output. As another example, machine-learned model(s) 1 can process the statistical data to generate a segmentation output. As another example, machine-learned model(s) 1 can process the statistical data to generate a visualization output. As another example, machine-learned model(s) 1 can process the statistical data to generate a diagnostic output.

[0378] In some implementations, input(s) 2 can be or otherwise represent sensor data. Machine-learned model(s) 1 can process the sensor data to generate an output. As an example, machine-learned model(s) 1 can process the sensor data to generate a recognition output. As another example, machine-learned model(s) 1 can process the sensor data to generate a prediction output. As another example, machine-learned model(s) 1 can process the sensor data to generate a classification output. As another example, machine-learned model(s) 1 can process the sensor data to generate a segmentation output. As another example, machine-learned model(s) 1 can process the sensor data to generate a visualization output. As another example, machine-learned model(s) 1 can process the sensor data to generate a diagnostic output. As another example, machine-learned model(s) 1 can process the sensor data to generate a detection output.

[0379] In some implementations, machine-learned model(s) 1 can be configured to perform a task that includes encoding input data for reliable and / or efficient transmission or storage (and / or corresponding decoding). For example, the task may be an audio compression task. The input may include audio data and the output may include compressed audio data. In another example, the input includes visual data (e.g., one or more images or videos), the output includes compressed visual data, and the task is a visual data compression task. In another example, the task may include generating an embedding for input data (e.g., input audio or visual data). In some cases, the input includes audio data representing a spoken utterance and the task is a speech recognition task. The output may include a text output which is mapped to the spoken utterance. In some cases, the task includes encrypting or decrypting input data. In some cases, the task includes a microprocessor performance task, such as branch prediction or memory address translation.

[0380] In some implementations, the task is a generative task, and machine-learned model(s) 1 can be configured to output content generated in view of input(s) 2. For instance, input(s) 2 can be or otherwise represent data of one or more modalities that encodes context for generating additional content.

[0381] In some implementations, the task can be a text completion task. Machine- learned model(s) 1 can be configured to process input(s) 2 that represent textual data and to generate output(s) 3 that represent additional textual data that completes a textual sequence that includes input(s) 2. For instance, machine-learned model(s) 1 can be configured to generate output(s) 3 to complete a sentence, paragraph, or portion of text that follows from a portion of text represented by input(s) 2.

[0382] In some implementations, the task can be an instruction following task. Machine-learned model(s) 1 can be configured to process input(s) 2 that represent instructions to perform a function and to generate output(s) 3 that advance a goal of satisfying the instruction function (e.g., at least a step of a multi-step procedure to perform the function). Output(s) 3 can represent data of the same or of a different modality as input(s) 2. For instance, input(s) 2 can represent textual data (e.g., natural language instructions for a task to be performed) and machine-learned model(s) 1 can process input(s) 2 to generate output(s) 3 that represent textual data responsive to the instructions (e.g., natural language responses, programming language responses, machine language responses, etc.). Input(s) 2 can represent image data (e.g.. image-based instructions for a task to be performed, optionally accompanied by textual instructions) and machine-learned model(s) 1 can process input(s) 2 to generate output(s) 3 that represent textual data responsive to the instructions (e.g., natural language responses, programming language responses, machine language responses, etc.). One or more output(s) 3 can be iteratively or recursively generated to sequentially process and accomplish steps toward accomplishing the requested functionality. For instance, an initial output can be executed by an external system or be processed by machine-learned model(s) 1 to complete an initial step of performing a function. Multiple steps can be performed, with a final output being obtained that is responsive to the initial instructions.

[0383] In some implementations, the task can be a question answering task. Machine- learned model(s) 1 can be configured to process input(s) 2 that represent a question to answer and to generate output(s) 3 that advance a goal of returning an answer to the question (e.g., at least a step of a multi-step procedure to perform the function). Output(s) 3 can represent data of the same or of a different modality as input(s) 2. For instance, input(s) 2 can represent textual data (e.g., natural language instructions for a task to be performed) and machine- learned model(s) 1 can process input(s) 2 to generate output(s) 3 that represent textual data responsive to the question (e.g., natural language responses, programming language responses, machine language responses, etc.). Input(s) 2 can represent image data (e.g., image-based instructions for a task to be performed, optionally accompanied by textualinstructions) and machine-learned model(s) 1 can process input(s) 2 to generate output(s) 3 that represent textual data responsive to the question (e.g., natural language responses, programming language responses, machine language responses, etc.). One or more output(s) 3 can be iteratively or recursively generated to sequentially process and accomplish steps toward answering the question. For instance, an initial output can be executed by an external system or be processed by machine-learned model(s) 1 to complete an initial step of obtaining an answer to the question (e.g., querying a database, performing a computation, executing a script, etc.). Multiple steps can be performed, with a final output being obtained that is responsive to the question.

[0384] In some implementations, the task can be an image generation task. Machine- learned model(s) 1 can be configured to process input(s) 2 that represent context regarding a desired portion of image content. The context can include text data, image data, audio data, etc. Machine-learned model(s) 1 can be configured to generate output(s) 3 that represent image data that depicts imagery7related to the context. For instance, machine-learned model(s) 1 can be configured to generate pixel data of an image. Values for channel (s) associated with the pixels in the pixel data can be selected based on the context (e.g.. based on a probability determined based on the context).

[0385] In some implementations, the task can be an audio generation task. Machine- learned model(s) 1 can be configured to process input(s) 2 that represent context regarding a desired portion of audio content. The context can include text data, image data, audio data, etc. Machine-learned model(s) 1 can be configured to generate output(s) 3 that represent audio data related to the context. For instance, machine-learned model (s) 1 can be configured to generate waveform data in the form of an image (e.g., a spectrogram). Values for channel(s) associated with pixels of the image can be selected based on the context. Machine- learned model(s) 1 can be configured to generate waveform data in the form of a sequence of discrete samples of a continuous waveform. Values of the sequence can be selected based on the context (e.g., based on a probability7determined based on the context).

[0386] In some implementations, the task can be a data generation task. Machine- learned model(s) I can be configured to process input(s) 2 that represent context regarding a desired portion of data (e.g., data from various data domains, such as sensor data, image data, multimodal data, statistical data, etc.). The desired data can be, for instance, synthetic data for training other machine-learned models. The context can include arbitrary data type(s). Machine-learned model(s) 1 can be configured to generate output(s) 3 that represent data that aligns with the desired data. For instance, machine-learned model (s) 1 can be configured togenerate data values for populating a dataset. Values for the data object(s) can be selected based on the context (e.g., based on a probability determined based on the context).Example Computing Systems and Devices

[0387] Figure 15 is a block diagram of an example networked computing system that can perform aspects of example implementations of the present disclosure. The system can include a number of computing devices and systems that are communicatively coupled over a network 49. An example computing device 50 is described to provide an example of a computing device that can perform any aspect of the present disclosure (e.g., implementing model host 31, client(s) 32, or both). An example server computing system 60 is described as an example of a server computing system that can perform any aspect of the present disclosure (e.g., implementing model host 31, client(s) 32, or both). Computing device 50 and server computing system(s) 60 can cooperatively interact (e.g., over network 49) to perform any aspect of the present disclosure (e.g., implementing model host 31, client(s) 32, or both). Model development platform system 70 is an example system that can host or serve model development platform(s) 12 for development of machine-learned models. Third-party system(s) 80 are example system(s) with which any of computing device 50, server computing system(s) 60, or model development platform system(s) 70 can interact in the performance of various aspects of the present disclosure (e.g., engaging third-party tools, accessing third-party databases or other resources, etc.).

[0388] Network 49 can be any type of communications network, such as a local area network (e.g., intranet), wide area network (e.g., Internet), or some combination thereof and can include any number of wired or wireless links. In general, communication over network 49 can be carried via any type of wired or wireless connection, using a wide variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), or protection schemes (e.g., VPN, secure HTTP, SSL). Network 49 can also be implemented via a system bus. For instance, one or more devices or systems of Figure 15 can be co-located with, contained by, or otherwise integrated into one or more other devices or systems.

[0389] Computing device 50 can be any type of computing device, such as, for example, a personal computing device (e.g., laptop or desktop), a mobile computing device (e.g., smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, a server computing device, a virtual machine operating on a host device, or any other type of computing device. Computing device 50 can be a clientcomputing device. Computing device 50 can be an end-user computing device. Computing device 50 can be a computing device of a service provided that provides a service to an end user (who may use another computing device to interact with computing device 50).

[0390] Computing device 50 can include one or more processors 51 and a memory 52. Processor(s) 51 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. Memory 52 can include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory' devices, magnetic disks, etc., and combinations thereof. Memory 52 can store data 53 and instructions 54 which can be executed by processor(s) 51 to cause computing device 50 to perform operations. The operations can implement any one or multiple features described herein. The operations can implement example methods and techniques described herein.

[0391] Computing device 50 can also include one or more input components that receive user input. For example, a user input component can be a touch-sensitive component (e.g.. a touch-sensitive display screen or a touch pad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component can serve to implement a virtual keyboard. Other example user input components include a microphone, camera, LIDAR, a physical keyboard or other buttons, or other means by which a user can provide user input.

[0392] Computing device 50 can store or include one or more machine-learned models 55. Machine-learned models 55 can include one or more machine-learned model(s) 1, such as a sequence processing model 4. Machine-learned models 55 can include one or multiple model instance(s) 31-1. Machine-learned model(s) 55 can be received from server computing system(s) 60, model development platform system 70, third party system(s) 80 (e.g., an application distribution platform), or developed locally on computing device 50. Machine-learned model(s) 55 can be loaded into memory 52 and used or otherwise implemented by processor(s) 51. Computing device 50 can implement multiple parallel instances of machine-learned model(s) 55.

[0393] Server computing system(s) 60 can include one or more processors 61 and a memory 62. Processor(s) 61 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. Memory’ 62 can include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM,EEPROM, EPROM, flash memory' devices, magnetic disks, etc., and combinations thereof. Memory 62 can store data 63 and instructions 64 which can be executed by processor(s) 61 to cause server computing system(s) 60 to perform operations. The operations can implement any one or multiple features described herein. The operations can implement example methods and techniques described herein.

[0394] In some implementations, server computing system 60 includes or is otherwise implemented by one or multiple server computing devices. In instances in which server computing system 60 includes multiple server computing devices, such server computing devices can operate according to sequential computing architectures, parallel computing architectures, or some combination thereof.

[0395] Server computing system 60 can store or otherwise include one or more machine-learned models 65. Machine-learned model(s) 65 can be the same as or different from machine-learned model(s) 55. Machine-learned models 65 can include one or more machine-learned model(s) 1, such as a sequence processing model 4. Machine-learned models 65 can include one or multiple model instance(s) 31-1. Machine-learned model(s) 65 can be received from computing device 50, model development platform system 70. third party system(s) 80, or developed locally on server computing system(s) 60. Machine-learned model(s) 65 can be loaded into memory' 62 and used or otherwise implemented by processor(s) 61. Server computing system(s) 60 can implement multiple parallel instances of machine-learned model(s) 65.

[0396] In an example configuration, machine-learned models 65 can be included in or otherwise stored and implemented by server computing system 60 to establish a client-server relationship with computing device 50 for serving model inferences. For instance, server computing system(s) 60 can implement model host 31 on behalf of client(s) 32 on computing device 50. For instance, machine-learned models 65 can be implemented by server computing system 60 as a portion of a web service (e.g., remote machine-learned model hosting service, such as an online interface for performing machine-learned model operations over a network on server computing system(s) 60). For instance, server computing system(s) 60 can communicate with computing device 50 over a local intranet or internet connection. For instance, computing device 50 can be a workstation or endpoint in communication with server computing system(s) 60, with implementation of machine-learned models 65 being managed by server computing system(s) 60 to remotely perform inference (e g., for runtime or training operations), with output(s) returned (e.g., cast, streamed, etc.) to computing device50. Machine-learned models 65 can work cooperatively or interoperatively with machine- learned models 55 on computing device 50 to perform various tasks.

[0397] Model development platform system(s) 70 can include one or more processors 71 and a memory 72. Processor(s) 71 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. Memory 72 can include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. Memory 72 can store data 73 and instructions 74 which can be executed by processor(s) 71 to cause model development platform system(s) 70 to perform operations. The operations can implement any one or multiple features described herein. The operations can implement example methods and techniques described herein. Example operations include the functionality described herein with respect to model development platform 12. This and other functionality7can be implemented by developer tool(s) 75.

[0398] Third-party system(s) 80 can include one or more processors 81 and a memory 82. Processor(s) 81 can be any suitable processing device (e.g.. a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. Memory 82 can include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. Memory 82 can store data 83 and instructions 84 which can be executed by processor(s) 81 to cause third-party system(s) 80 to perform operations. The operations can implement any one or multiple features described herein. The operations can implement example methods and techniques described herein. Example operations include the functionality described herein with respect to tools and other external resources called when training or performing inference with machine-learned model(s) 1, 4, 16, 20, 55, 65, etc. (e.g., third-party resource(s) 85).

[0399] Figure 15illustrates one example arrangement of computing systems that can be used to implement the present disclosure. Other computing system configurations can be used as well. For example, in some implementations, one or both of computing system 50 or server computing system(s) 60 can implement all or a portion of the operations of model development platform system 70. For example, computing system 50 or server computing system(s) 60 can implement developer tool(s) 75 (or extensions thereof) to develop, update / train, or refine machine-learned models 1, 4, 16, 20, 55, 65, etc. using one or moretechniques described herein with respect to model alignment toolkit 17. In this manner, for instance, computing system 50 or server computing system(s) 60 can develop, update / train, or refine machine-learned models based on local datasets (e.g., for model personalization / customization, as permitted by user data preference selections).

[0400] Figure 16 is a block diagram of an example computing device 98 that performs according to example embodiments of the present disclosure. Computing device 98 can be a user computing device or a server computing device (e.g., computing device 50, server computing system(s) 60, etc ). Computing device 98 can implement model host 31. For instance, computing device 98 can include a number of applications (e.g., applications 1 through N). Each application can contain its own machine learning library and machine- learned model(s). For example, each application can include a machine-learned model. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. As illustrated in Figure 1 , each application can communicate with a number of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, or additional components. In some implementations, each application can communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to that application.

[0401] Figure 17 is a block diagram of an example computing device 99 that performs according to example embodiments of the present disclosure. Computing device 99 can be the same as or different from computing device 98. Computing device 99 can be a user computing device or a server computing device (e.g., computing device 50, server computing system(s) 60. etc.). Computing device 98 can implement model host 31. For instance, computing device 99 can include a number of applications (e.g., applications 1 through N). Each application can be in communication with a central intelligence layer. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In some implementations, each application can communicate with the central intelligence layer (and model(s) stored therein) using an API (e.g., a common API across all applications).

[0402] The central intelligence layer can include a number of machine-learned models. For example, as illustrated in Figure 17, a respective machine-learned model can be provided for each application and managed by the central intelligence layer. In other implementations, two or more applications can share a single machine-learned model. For example, in some implementations, the central intelligence layer can provide a single modelfor all of the applications. In some implementations, the central intelligence layer is included within or otherwise implemented by an operating system of computing device 99.

[0403] The central intelligence layer can communicate with a central device data layer. The central device data layer can be a centralized repository of data for computing device 99. As illustrated in Figure 17, the central device data layer can communicate with a number of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).Additional Disclosure

[0404] The technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems, as well as actions taken and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and divisions of tasks and functionality between and among components. For instance, processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0405] While the present subject matter has been described in detail with respect to various specific example embodiments thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations or additions to the present subject matter as w ould be readily apparent to one of ordinary skill in the art. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure cover such alterations, variations, and equivalents.

[0406] Aspects of the disclosure have been described in terms of illustrative embodiments thereof. Any and all features in the following claims can be combined or rearranged in any way possible, including combinations of claims not explicitly enumerated in combination together, as the example claim dependencies listed herein should not be readas limiting the scope of possible combinations of features disclosed herein. Accordingly, the scope of the present disclosure is by way of example rather than by way of limitation, and the subject disclosure does not preclude inclusion of such modifications, variations or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. Moreover, terms are described herein using lists of example elements joined by conjunctions such as “and.” “or,” “but,” etc. It should be understood that such conjunctions are provided for explanatory purposes only. Clauses and other sequences of items joined by a particular conjunction such as “or,” for example, can refer to “and / or,” “at least one of’, “any combination of’ example elements listed therein, etc. Terms such as “based on” should be understood as “based at least in part on.”

[0407] The term “can” should be understood as referring to a possibility of a feature in various implementations and not as prescribing an ability7that is necessarily present in every7implementation. For example, the phrase “X can perform Y” should be understood as indicating that, in various implementations, X has the potential to be configured to perform Y, and not as indicating that in every’ instance X must always be able to perform Y. It should be understood that, in various implementations. X might be unable to perform Y and remain within the scope of the present disclosure.

[0408] The term “may” should be understood as referring to a possibility' of a feature in various implementations and not as prescribing an ability that is necessarily present in every implementation. For example, the phrase “X may perform Y” should be understood as indicating that, in various implementations, X has the potential to be configured to perform Y, and not as indicating that in every' instance X must always be able to perform Y. It should be understood that, in various implementations, X might be unable to perform Y and remain within the scope of the present disclosure.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method for validating outputs of a machine-learned task processing system, comprising: receiving a request to perform a task; generating, using a machine-learned model, a plurality of tool paths for performing the task, wherein a respective tool path comprises one or more tool calls generated by the machine-learned model, each tool call configured to cause a corresponding tool to perform a step of the task; selecting a response to the request based on a consistency metric determined by evaluating a plurality of output states respectively from the plurality of tool paths; and returning the response.

2. The computer-implemented method of claim 1, wherein the machine-learned model is a sequence processing model configured to receive an input comprising one or more input portions and generate an output comprising one or more output portions conditioned on the one or more input portions.

3. The computer-implemented method of any preceding claim, wherein generating, using the machine-learned model, the plurality of tool paths comprises: for each respective tool path: for each respective step of the respective tool path: generating, using the machine-learned model, a tool call; invoking the tool call to initiate execution of a corresponding tool in a tool execution environment; obtaining a result from the tool; and evaluating the result.

4. The computer-implemented method of any preceding claim, wherein generating, using the machine-learned model, the tool call comprises: receiving request data describing at least an aspect of the request; obtaining an index of available tools; providing the request data and the index to the machine-learned model;receiving an output generated by the machine-learned model based on the request data and the index, wherein the output describes the tool call; and parsing the output to obtain the tool call.

5. The computer-implemented method of any preceding claim, wherein: the machine-learned model generates tool calls for each step of a sequence of operations in atum-by-tum manner, wherein a first subset of turns are used for invoking tools and a second subset of turns are used for obtaining information; and the first subset of turns are resampled to obtain a plurality of different tool calls and the second subset of turns can be decoded with greedy sampling.

6. The computer-implemented method of any preceding claim, comprising: determining the consistency metric by comparing alignment of the output states.

7. The computer-implemented method of any preceding claim, comprising: determining the consistency metric using a machine-learned model to evaluate a quality of each output state, wherein: the machine-learned model used to evaluate the uality of each output state is the same as the machine-learned model used to generate the plurality of tool paths; or the machine-learned model used to evaluate the quality of each output state is different from the machine-learned model used to generate the plurality of tool paths.

8. The computer-implemented method of any preceding claim, wherein the machine-learned model used to evaluate the quality of each output state is trained to process the plurality of output states together to output a preferred output state, wherein the response is selected using the preferred output state.

9. The computer-implemented method of any preceding claim, wherein the consistency metric is based on a similarity of learned embeddings generated using the plurality of output states.

10. The computer-implemented method of any preceding claim, wherein the consistency metric is determined by processing the output states using a machine-learned model trained to generate outputs indicative of salient differences between inputs.

11. The computer-implemented method of any preceding claim, wherein the consistency metric comprises one or more heuristic criteria and one or more probabilistic criteria, the one or more probabilistic criteria determined using an output of a machine- learned model.

12. The computer-implemented method of any preceding claim, comprising: selecting the response based on the consistency metric by: identifying one or more features shared by a group of the plurality of output states; and selecting the response from the group.

13. The computer-implemented method of any preceding claim, wherein the one or more features comprise at least one latent feature generated using a machine-learned model.

14. The computer-implemented method of any preceding claim, comprising: selecting the response based on the consistency metric by: clustering representations of one or more features of the plurality of output states; identifying a preferred cluster; and selecting the response from the cluster.

15. The computer-implemented method of any preceding claim, wherein selecting the response from the cluster comprises scoring members of the cluster and selecting the response based on the scores.

16. The computer-implemented method of any preceding claim, wherein selecting the response from the cluster comprises determining a centroid of the cluster and selecting the response based on the centroid.

17. The computer-implemented method of any preceding claim, wherein: the respective tool path comprises one or more nodes, wherein a respective output state of the respective tool path is based on a completion node of the respective tool path; and a first node of the one or more nodes is generated by: generating, using the machine-learned model, a first tool call to execute a first step of the task; and initiating execution of the first tool call to obtain a first tool response.

18. The computer-implemented method of any preceding claim, wherein initiating execution of the first tool call comprises passing the first tool call to a corresponding tool in an execution environment associated with the corresponding tool.

19. The computer-implemented method of any preceding claim, wherein the machine-learned model used to generate the tool paths is configured to control operations in the execution environment.

20. The computer-implemented method of any preceding claim, wherein the execution environment is a virtual machine assigned to a task instance comprising the machine-learned model used to generate the tool paths.

21. The computer-implemented method of any preceding claim, wherein the execution environment prevents the machine-learned model used to generate the tool paths from interfering with another task instance.

22. The computer-implemented method of any preceding claim, wherein the execution environment prevents the corresponding tool from interfering with another tool associated with another tool path.

23. The computer-implemented method of any preceding claim, wherein each tool path is executed in a separate execution environment.

24. The computer-implemented method of any preceding claim, wherein the separate execution environment is instantiated by a virtual machine.

25. The computer-implemented method of any preceding claim, wherein the separate execution environment is instantiated by separate working directories within a directory tree.

26. The computer-implemented method of any preceding claim, comprising: caching resources for use across multiple tool paths.

27. The computer-implemented method of any preceding claim, comprising: caching resources for use across multiple execution environments.

28. The computer-implemented method of any preceding claim, wherein: a second node of the one or more nodes is generated by: generating, using the machine-learned model and based on the first tool response, a second tool call to execute a second step of the task; and initiating execution of the second tool call to obtain a second tool response.

29. The computer-implemented method of any preceding claim, wherein: the plurality of tools paths comprises a tool path tree; a second respective tool path comprises one or more nodes, wherein a respective output state of the second respective tool path is based on a completion node of the second respective tool path; and a first node of the one or more nodes is generated by: generating, using the machine-learned model, a first tool call to execute a first step of the task; and initiating execution of the first tool call to obtain a first tool response.

30. The computer-implemented method of any preceding claim, comprising: evaluating a plurality of intermediate states respectively for the plurality' of tool paths, wherein a respective intermediate state of the respective tool path is based on an intermediate node of the respective tool path that precedes the completion node; andpruning at least one of the plurality of tool paths based on a corresponding intermediate state.

31. The computer-implemented method of any preceding claim, comprising: pruning at least one tool path based on determining a redundancy of the at least one tool path.

32. The computer-implemented method of any preceding claim, wherein the redundancy is determined based on identifying a deterministic outcome of the at least one tool path.

33. The computer-implemented method of any preceding claim, wherein the redundancy is determined based on a margin of similarity'.

34. The computer-implemented method of any preceding claim, wherein the margin of similarity is based on a minimum step size associated with a range of parameter values for a tool call.

35. The computer-implemented method of any preceding claim, comprising: adjusting at least one tool path based on determining a failure condition associated with the at least one tool path.

36. The computer-implemented method of any preceding claim, wherein the failure condition is determined based on receiving a tool response that diverges from desired criteria.

37. The computer-implemented method of any preceding claim, wherein the failure condition is determined based on receiving an error message indicating inoperability of a tool or tool call.

38. The computer-implemented method of any preceding claim, wherein adjusting the at least one tool path comprises: pruning the at least one tool path.

39. The computer-implemented method of any preceding claim, wherein adjusting the at least one tool path comprises: restarting the at least one tool path using one or more tool calls newly generated by the machine-learned model.

40. The computer-implemented method of any preceding claim, wherein restarting the at least one tool path comprises: generating a new tool call based on the failure condition.

41. The computer-implemented method of any preceding claim, wherein generating the new tool call based on the failure condition comprises: inputting data describing the failure condition to the machine-learned model as context to a query for the new tool call.

42. The computer-implemented method of any preceding claim, comprising: evaluating at least one intermediate state using the machine-learned model.

43. The computer-implemented method of any preceding claim, comprising: evaluating at least one intermediate state using a different machine-learned model from the machine-learned model used to generate the tool calls.

44. The computer-implemented method of any preceding claim, comprising: evaluating at least one intermediate state using the machine-learned model to select a heuristic tool to apply to the at least one intermediate state.

45. The computer-implemented method of any preceding claim, comprising: evaluating multiple features of at least one intermediate state.

46. The computer-implemented method of any preceding claim, comprising: computing a consistency metric over the plurality of intermediate states; selecting a representative tool path based on the consistency metric; and pruning one or more other tool paths.

47. The computer-implemented method of any preceding claim, comprising: computing a consistency metric over the plurality of intermediate states to identify a cluster of aligned tool paths; selecting a representative tool path based on the consistency metric, wherein the representative tool path represents the cluster of aligned tool paths; and pruning the other tool paths in the cluster of aligned tool paths.

48. The computer-implemented method of any preceding claim, comprising: computing a consistency metric over the plurality of intermediate states to identify a cluster of aligned tool paths; selecting a representative tool path based on the consistency metric, wherein the representative tool path represents the cluster of aligned tool paths; pruning the other tool paths in the cluster of aligned tool paths; and starting anew tool path by sampling a new tool call from the machine-learned model.

49. The computer-implemented method of any preceding claim, wherein the tool call comprises a textual command configured for entry via a command line interface.

50. The computer-implemented method of any preceding claim, wherein the tool call comprises a request to an application programming interface endpoint.

51. The computer-implemented method of any preceding claim, wherein the tool call comprises a request to an application programming interface endpoint over a network.

52. The computer-implemented method of any preceding claim, comprising: generating, using the machine-learned model, a tool comprising one or more lines of programming code that perform operations related to the requested task; and executing the tool in at least one tool path.

53. The computer-implemented method of any preceding claim, comprising: generating, using a machine-learned code generation model, a tool comprising one or more lines of programming code that perform operations related to the requested task; and executing the tool in at least one tool path.

54. The computer-implemented method of any preceding claim, comprising: determining, using the request, one or more screening criteria for evaluating the intermediate states; and pruning the at least one tool path based on a failure with respect to the one or more screening criteria.

55. The computer-implemented method of any preceding claim, comprising: evaluating an intermediate consistency metric over one or more tool calls sampled from the machine-learned model.

56. The computer-implemented method of any preceding claim, comprising: computing a consistency metric over a plurality of tool calls.

57. The computer-implemented method of any preceding claim, comprising: selecting one or more tool calls for execution to improve a diversity7of tool paths.

58. The computer-implemented method of any preceding claim, comprising: clustering tool calls sampled from the machine-learned model; and selecting a representative tool call from each of the top-K clusters for execution.

59. The computer-implemented method of any preceding claim, comprising: determining, based on the tool call, one or more operations to perform using a graphical user interface; generating one or more navigation commands configured to interact with the graphical user interface; and initiating execution of the one or more navigation commands.

60. A computer-implemented method, comprising: transmitting, from a client computing system and to a server computing system, a request to perform a task; and receiving, by the client computing system and from the server computing system, a response based on performance of the task; wherein the response is obtained using the method of any of the preceding claims.

61. The computer-implemented method of claim 60, wherein: the server computing system performs the task using the method of any of the preceding claims.

62. The computer-implemented method of claim 60 or 61. comprising: receiving, by the client computing system and from a user computing system, an initial request; and obtaining the request to perform the task based on the initial request.

63. One or more non-transitory computer-readable media storing instructions that are executable by one or more processors to cause a computing system to perform operations, the operations comprising the method of any one or more of the preceding claims.

64. A computing system, comprising: one or more processors; and one or more non-transitory computer-readable media storing instructions that are executable by one or more processors to cause a computing system to perform operations, the operations comprising the method of any one or more of the preceding claims.

Images