Self-consistency metrics for validating outputs of tools controlled by machine-learned models
By generating multiple tool paths to execute in parallel and using a consistency metric to evaluate the output, the suboptimal performance problem caused by a single instruction in the prior art is solved, achieving more efficient and reliable tool control and reducing latency and resource interference.
Patent Information
- Application Number
- CN202480090895.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-21
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies, when using machine learning-based model control tools, are susceptible to suboptimal performance from single instructions, lack signals to improve the confidence of effective output responses, and may lead to unnecessary task duplication, network communication disruptions, and resource interference.
By generating multiple tool paths to execute in parallel, evaluating the output using consistency metrics, pruning faulty paths, optimizing resource utilization and cache sharing, the consistency and accuracy of the output are ensured.
It improves the reliability and accuracy of tool output, reduces latency and network bandwidth utilization, optimizes computing resource usage, and reduces energy consumption and environmental pollution.
Smart Images

Figure CN123076871A_ABST
Abstract
Description
Technical Field
[0001] This disclosure generally relates to machine learning processes and apparatuses and systems that utilize machine learning. More specifically, this disclosure relates to infrastructure for improving the performance of tools controlled by machine learning models. Background Technology
[0002] A computer can receive input. It can execute instructions to process the input using a parameterized model to generate output. The computer can obtain feedback on its performance in generating output using that model. The computer can generate feedback by evaluating its performance. The computer can receive feedback from external sources. Based on this feedback, the computer can update the model's parameters to improve its performance. In this way, the computer can iteratively "learn" to generate the desired output. The resulting model is often referred to as a machine learning model. Summary of the Invention
[0003] Various aspects and advantages of embodiments of this disclosure will be set forth in part in the description which follows, or may be learned from the description, or may be learned by practice of these embodiments.
[0004] In one aspect, this disclosure provides a first example method. In some implementations, the first example method includes: receiving a request to perform a task. In some implementations, the first example method includes: generating multiple tool paths for performing the task using a machine learning-based model, wherein each tool path includes one or more tool calls generated by the machine learning model, each tool call being configured to cause a corresponding tool to perform steps of the task. In some implementations, the first example method includes: selecting a response to the request based on a consistency metric determined by evaluating multiple output states from the multiple tool paths, respectively. In some implementations, the first example method includes: returning the response.
[0005] In one aspect, this disclosure provides a second example method. In some implementations, the second example method includes: transmitting a request from a client computing system to a server computing system for performing a task. In some implementations, the second example method includes: the client computing system receiving a response from the server computing system based on the execution of the task. In some implementations, the response is obtained using an implementation of either the first example method or the second example method.
[0006] In one aspect, this disclosure provides examples of one or more non-transitory computer-readable media storing instructions that can be executed by one or more processors to cause a computing system to perform operations, including implementations of the first example method or the second example method.
[0007] In one aspect, this disclosure provides an example computing system. The example computing system may include one or more processors and one or more non-transitory computer-readable media storing instructions that can be executed by the one or more processors to cause the computing system to perform operations. These operations may include implementations of a first example method or a second example method.
[0008] Other exemplary aspects of this disclosure relate to other systems, methods, apparatuses, tangible non-transitory computer-readable media, and devices for performing the functions described herein. These and other features, aspects, and advantages of the various implementations will be better understood with reference to the following description and the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate implementations of this disclosure and, together with the description, help to explain the relevant principles. Attached Figure Description
[0009] Figure 1 This is a block diagram of an example system for verifying the output of a machine learning-based task processing system, based on exemplary implementations of various aspects of this disclosure.
[0010] Figure 2 This is a block diagram of an example tool path for verifying the output of a machine learning-based task processing system, based on example implementations of various aspects of this disclosure.
[0011] Figure 3A This is a block diagram of an example tool path for verifying the output of a machine learning-based task processing system, based on example implementations of various aspects of this disclosure.
[0012] Figure 3B This is a block diagram of an example tool path for verifying the output of a machine learning-based task processing system, based on example implementations of various aspects of this disclosure.
[0013] Figure 3C This is a block diagram of an example tool path for verifying the output of a machine learning-based task processing system, based on example implementations of various aspects of this disclosure.
[0014] Figure 4 This is a block diagram of an example validator for validating the output of a machine learning-based task processing system, based on example implementations of various aspects of this disclosure.
[0015] Figure 5 This is a flowchart illustrating an example method for verifying the output of a machine learning-based task processing system, according to example implementations of various aspects of this disclosure.
[0016] Figure 6This is a flowchart illustrating an example method for controlling a tool using a machine learning-based model, according to exemplary implementations of various aspects of this disclosure.
[0017] Figure 7 This is a flowchart illustrating an example method for controlling a tool using a machine learning-based model, according to exemplary implementations of various aspects of this disclosure.
[0018] Figure 8 This is a flowchart illustrating an example method for training a machine learning model, according to example implementations of various aspects of this disclosure.
[0019] Figure 9 This is a block diagram of an example processing flow for using a machine learning-based model to process inputs to generate outputs, based on example implementations of various aspects of this disclosure.
[0020] Figure 10 This is a block diagram of an example sequence processing model based on example implementations of various aspects of this disclosure.
[0021] Figure 11 This is a block diagram of an example technique for filling example input sequences for processing by a sequence processing model, based on example implementations of various aspects of this disclosure.
[0022] Figure 12 This is a block diagram of an example model development platform based on example implementations of various aspects of this disclosure.
[0023] Figure 13 This is a block diagram of an example training workflow for training a machine learning model, based on example implementations of various aspects of this disclosure.
[0024] Figure 14 This is a block diagram of an inference system for operating one or more machine learning-based models to perform inference, based on example implementations of various aspects of this disclosure.
[0025] Figure 15 This is a block diagram of an example networked computing system based on exemplary implementations of various aspects of this disclosure.
[0026] Figure 16 This is a block diagram of an example computing device that implements various aspects of this disclosure.
[0027] Figure 17 This is a block diagram of an example computing device that implements various aspects of this disclosure. Detailed Implementation
[0028] In general, this disclosure relates to techniques for applying consistency metrics to verify the output of a tool controlled by a machine learning model. For example, a machine learning model can generate instructions to control a tool to perform a task in response to a request. The tool can generate output in response to the provided instructions. In some scenarios, the tool output may vary due to the stochastic nature of some machine learning models. To improve overall performance on the task, multiple sets of instructions can be obtained from the machine learning model to provide diverse approaches to performing the task. The final results from different approaches can be compared to identify consistent features between them. Such consistency can indicate that consistently obtained features respond to the initial request. Outlier features are less likely to respond to the initial request.
[0029] Some existing processing systems can use machine learning models to generate instructions for implementing external tools. However, such systems often fail to adopt a robust instruction set sample size. Typically, such systems may only use one instruction for a single tool without exploring the alternative solution space at query time. Such systems may be susceptible to suboptimal performance, such as in scenarios where the single tool instruction is either a wrong instruction for the correct tool, a wrong instruction for the wrong tool, or a correct instruction for the wrong tool. Furthermore, such systems may only obtain a single output from that single tool, which may lack certain signals to increase the confidence that the output effectively responds to the task. In such systems, the end user or other downstream systems may be responsible for receiving the output, identifying any errors or failures in the output, and repeating the query. This can lead to unnecessary processing of multiple task repetitions, multiple network communications, failed or abandoned tasks, etc.
[0030] Advantageously, example implementations according to various aspects of this disclosure can sample multiple different instruction sets, optionally targeting multiple different tools, to explore the potential solution space for performing tasks. By generating multiple outputs through diverse processing paths, example implementations according to various aspects of this disclosure can utilize self-consistency metrics to identify outputs that share characteristics among multiple outputs. In this way, for example, the example implementations can use these multiple outputs to "vote" for a preferred output. Based on one or more preferred outputs, the example implementations can obtain a response for responding to an initial request.
[0031] In this manner, for example, the exemplary implementations of this disclosure can provide a machine learning-based model system that can access and utilize external tools with improved reliability, consistency, and creativity. This disclosure applies self-consistency techniques suitable for tool use, which can improve the orchestration of single-step or multi-step tool-based actions. Generally, the exemplary implementations involve generating multiple sequences of tool paths, executing these paths, and then evaluating the consistency of intermediate or output states. Paths with consistent states can be retained for further processing (e.g., for returning output). Paths with deviating states can be pruned due to containing errors.
[0032] For example, a machine learning model can be assigned to perform file conversion tasks to transform input files into different formats. The machine learning model can be configured to generate commands that control software tools to perform file conversions. For instance, a machine learning model can be configured to generate text commands for input via a command-line interface of a terminal application. Various flags can be used to modify such commands to indicate different options. For example, syntax...
[0033] ~> command --option value
[0034] You can instruct the execution of a command, where the option is set to the value.
[0035] Machine learning models can predict which commands to use, which options to set, and which values to apply to those options. However, the model may predict incorrect or invalid configurations. Command execution may fail. Command output may return incorrect results or fail to respond to the initial request.
[0036] One way to mitigate these failures is to simply rerun the command until it succeeds. However, rerunning the command sequentially can lead to inappropriate processing delays and suboptimal computational utilization. Furthermore, even if the command succeeds, it can be difficult to determine when the output is correct.
[0037] Advantageously, the example implementations of this disclosure allow for the parallel execution of multiple tool calls across multiple tool paths. For example, the computational system can sample multiple distinct outputs from the model to obtain multiple distinct tool calls. The computational system can parse each tool call and initiate the execution of each tool call in parallel. The computational system can monitor the execution of each tool path to detect inoperability, constraint violations, or other failure modes. Failed tool paths can be pruned or restarted (e.g., by sampling new tool calls from the model).
[0038] The computational system can receive output from each of multiple toolpaths. Consistency measures can be used to compare the outputs to assess whether there is a majority or relative majority alignment on the output features. For example, a consistency measure can be evaluated for sampling the diverse toolpaths (e.g., representing diverse approaches to resolving a query) and their corresponding outputs. For instance, a set of outputs with diverse tool execution strategies can be aggregated to obtain a majority or relative majority "vote" regarding the final output. In this way, toolpath outputs can self-validate to improve the robustness of the model output and the accuracy of the final response.
[0039] The computational system can evaluate the intermediate states of each tool path to confirm that they are progressing toward a valid completion state. Intermediate states can be tool outputs from tools in a multi-tool call sequence. Intermediate states can include state updates from tool outputs. Evaluation of intermediate states can involve using heuristics, probabilistic models, or the same or additional machine learning models that can assess the quality and relevance of intermediate outputs.
[0040] If an intermediate state deviates from the expected criteria or exhibits signs of failure, such as inoperability of a tool or tool call, the corresponding tool path can be pruned or corrected. This pruning can be based on redundancy, where similar tool paths that produce deterministic results within a similarity margin are identified and reduced to a representative subset, or on explicit failures detected during execution. This proactive management of tool paths optimizes computational resources and focuses efforts on the paths most likely to succeed in completing the task.
[0041] Beyond measuring consistency of output states, computational systems can perform consistency evaluations on intermediate states. For example, such checks can involve comparing intermediate states across different toolpaths to identify commonalities or differences. Too much commonality (e.g., identity) can trigger pruning of redundant forms. By maintaining toolpaths with intermediate states that offer diversity for improving that set of toolpaths, the system can explore a broader solution space and increase the chances of finding optimal or satisfactory solutions. Self-consistency techniques can be applied to various components of a task, such as code segments (generated by the model for execution), function calls, or API calls.
[0042] Consistency metrics can be computed using various techniques. In some implementations, the system can compare outputs or intermediate states to compute the consistency metric. One approach for comparison can be byte-level differences, which can involve a direct comparison of binary or text-based output data from each toolpath. Byte-level differences can be useful for tasks where exact output matters, such as in file checksums and verifications, when comparing the exact identity of output strings, etc.
[0043] Another example approach for comparing outputs could involve semantic similarity comparisons, which assess the meaning or intent behind the outputs rather than their exact textual or binary representations. This can be useful for tasks that allow for different, valid outputs that equivalently satisfy a query. Semantic similarity can be computed using natural language processing techniques such as word embeddings, which can capture the context and semantics of the text. In general, similarity between output objects can be determined using vector-based similarity measures between their vector representations, such as cosine similarity measures.
[0044] For more structured data, such as JSON or XML output, tree- or graph-based comparison algorithms can be used to assess structural consistency and content-based consistency. These techniques can involve traversing the hierarchical structure of the data and comparing elements and attributes, regardless of possible changes in order or structure that may not affect the semantic content of the data.
[0045] For examples involving visual output or graphical user interfaces, image comparison techniques, such as pixel-by-pixel comparison or more sophisticated image hashing methods, can be applied. For instance, perceptual hashing can create hash values based on the visual content of an image, making it possible to quickly and efficiently compare visual outputs, even if they have been slightly altered or compressed. Similarly, image embeddings can be used to perform similarity comparisons.
[0046] In some examples, the machine learning model (that generates tool calls) itself can evaluate the current state of the tool path (e.g., intermediate state, output state). For example, a description of the current state can be fed into the model along with the original query and instructions for evaluating the next step of the tool path. The model can output the next action that can be performed in the tool path. If the model determines that the current state is on a potentially successful path, it can generate the next tool call to continue the task. If the evaluation indicates a potential problem or deviation from the expected result, the model can predict corrective actions, such as modifying parameters or selecting an alternative tool call. If the evaluation indicates a failure mode for the tool path, the model can output a termination or end state as the next action.
[0047] Each toolpath can execute in a different execution environment, which provides isolation and independence for each toolpath's execution. This helps prevent interference between toolpaths. For example, separate execution environments can be instantiated using isolated runtime contexts within virtual machines, containers, or a single operating system.
[0048] Virtual machines can provide isolation at the hardware level, while containers can provide isolation at the operating system level. Runtime contexts can also be isolated using various programming language features or runtime settings, such as different memory spaces and threads.
[0049] Generally, a sample execution environment can be configured to prevent executing tool paths from interfering with the resources used by other tool paths. The execution environment can enforce resource allocation, access control, or execution permissions to maintain the integrity and independence of each task instance. For example, resource groups (e.g., cgroups) or similar mechanisms can be used to control the amount of CPU, memory, and I / O bandwidth that each task can consume. This can help prevent any single task from monopolizing system resources.
[0050] Some resources can be cached for use across multiple toolpaths. This reduces redundant computations and I / O operations, thereby improving the overall efficiency of the system. Caching can be implemented at various levels, including within the execution environment itself or at a shared layer accessible to all environments. Examples of resources that can be cached include, for instance, public libraries, datasets, intermediate computation results, and configuration files. Caching strategies can implement one or more learned components that can predict which resources are likely to be reused and pre-cache them based on task context or historical usage patterns.
[0051] The example implementations of this disclosure can provide various technical effects and benefits. For example, by processing different paths in parallel, the example implementations can reduce latency by avoiding simply processing single instructions, receiving errors, retries, and so on. For example, the example implementations of this disclosure can provide lower latency by efficiently deconstructing and pre-processing tool call refinement tasks. For example, instead of sequential optimization iterations that serially generate a small number of tool calls by incrementing the values of the tool call parameters to make the tool perform the task of interest, the example implementations of this disclosure can generate batches of tool calls with parameter values spanning a range of parameters. Generated with sufficient recall, this batch of tool calls can efficiently achieve the same output as that obtained from time-consuming sequential optimization. However, batch generation can be highly parallelized because each tool call in the batch does not need to wait for the execution of any previous tool calls. The evaluation of tool responses from this batch of tool calls can also be processed in parallel.
[0052] On the other hand, the example implementations of this disclosure can provide improved error checking of tool output. For example, instead of simply returning the tool output, tool output can be collected across multiple different tool paths to effectively increase the signal-to-noise ratio. For example, shared or similar eigenvalues along a specific dimension of the output can be considered to provide a good signal. Outlier eigenvalues along that dimension can be considered noise, and the associated output can be ignored or rejected.
[0053] On the other hand, the example implementations of this disclosure can reduce network bandwidth utilization. For example, by automatically processing different paths to identify high-quality output, instead of sequentially generating output and waiting for negative user feedback, the example implementations can internally verify the quality of output before sending, thereby reducing the likelihood of further communication iterations. This can reduce the number of messages sent to downstream systems over the network and the number of messages received from downstream systems over the network.
[0054] Furthermore, when using probabilistic generative models to control external tools, technical problems can arise when the model generates outputs that include commands or actions intended to be performed by the external tools, which may be inaccurate or "illusionary." This can lead to undesirable results, such as the generation of erroneous data, the execution of unintended or unhelpful actions, or the complete inability to achieve the current task.
[0055] Advantageously, the exemplary implementations of this disclosure can provide solutions to this and other technical problems. For example, the exemplary implementations of this disclosure can generate multiple tool paths for performing a requested task. Each tool path can attempt to complete at least one step for performing the requested action. Using multiple tool paths can provide verification of the output returned from these tools. This verification can increase the confidence in the accuracy of the output. For example, a false command or incorrect output may have little verification in this set of tool paths. In contrast, a correct or expected response may enjoy a higher level of verification. In this way, for example, the exemplary implementations of this disclosure can provide a solution to the problem of probabilistic model generation initiating commands that result in incorrect or inoperable tool execution.
[0056] Another technical problem can arise when multiple processing tasks interfere with each other while accessing or modifying the same resource. For example, a task might involve adding data to a table. In some cases, if multiple concurrently performed operations attempt to insert data into a table, the data describing the table obtained by the first operation might become outdated due to a background update by the second operation. This can lead to failed operations, data errors, etc.
[0057] Advantageously, the exemplary implementations of this disclosure can provide solutions to this and other technical problems. For example, the exemplary implementations of this disclosure can initiate the execution of tool calls within an execution environment associated with the corresponding tool path. The execution environment can be sandboxed to prevent the tool path from negatively impacting other tool paths that may execute in parallel. For example, each tool path can execute in a separate directory, container, virtual machine, physical machine, etc. In this way, for example, the exemplary implementations of this disclosure can help facilitate the co-execution of multiple different tool paths to perform the same or similar operations with minimal or no interference. Furthermore, by executing tool paths in separate execution environments, the exemplary implementations can achieve individual control over the amount of computational resources available to each tool path. This can lead to improved computational resource utilization because each path can be managed and controlled more effectively. For example, race conditions or failure loops created by a tool path can be detected and terminated individually, preventing the tool path from consuming more resources than originally specified for it.
[0058] Executing tasks in separate environments can present another technical challenge. In some cases, strict isolation can lead to redundant processing if the effective work completed by a task cannot be reused later. For example, a task downloading a dataset within a virtual machine might be unaware that the same dataset has already been downloaded by another task executing in a different virtual machine.
[0059] Advantageously, the exemplary implementations of this disclosure can provide solutions to this and other technical problems. For example, the exemplary implementations of this disclosure can cache resources for use across multiple tool paths or execution environments. The cache can be read-only to prevent interference across tool paths. Such a cache can provide that frequently used data or computational routines do not need to be repeatedly loaded or executed, thereby saving time and reducing the load on the processing and memory resources of the execution machine. The cache can be implemented in hardware (e.g., dedicated memory or storage device) or software (e.g., dedicated swap file / folder / partition, etc.).
[0060] Another technical problem may arise when multiple tasks are executed in parallel. In some cases, certain tasks may fail or begin to deviate from their expected action path. If such a task fails or deviates early in an execution batch, the processing lane allocated to that task may be used suboptimally (e.g., underutilization of potential processing capacity, deliberate expenditure of processing capacity for unavailable output, etc.).
[0061] Advantageously, the exemplary implementations of this disclosure can provide solutions to this and other technical problems. For example, the exemplary implementations of this disclosure can employ machine learning models or heuristic tools to evaluate the progress of each tool path during operation. Such monitoring can support the pruning of inefficient, failing, or redundant paths, which can help optimize the utilization of computational resources by focusing computational efforts on the most promising task-completion paths.
[0062] In addition to providing solutions to the aforementioned technical challenges, the technical effect of the exemplary implementations disclosed herein is to improve the energy efficiency of performing operations using machine learning-based models, thereby improving the capabilities of computers implementing such models. For example, the exemplary implementations can achieve more energy-efficient runtime execution or inference by carefully managing the execution of multiple toolpaths, thus obtaining higher quality results with less redundant processing. In some scenarios, improved energy efficiency can reduce the energy consumed in performing a given task (e.g., reducing the energy consumed in maintaining the model in memory, reducing the energy consumed in performing computations within the model, etc.). In other scenarios, improved energy efficiency can accomplish more tasks within a given energy budget (e.g., more tasks, more complex tasks, or completing the same task with higher accuracy or precision, etc.).
[0063] In this way, for example, the improved energy efficiency of the exemplary implementations of this disclosure can reduce the amount of pollution or other waste associated with implementing machine learning-based models and systems, thereby advancing the field of machine learning and artificial intelligence as a whole. The amount of pollution can be reduced overall (e.g., its absolute value) or on a normalized basis (e.g., energy per task, per model size, etc.). For example, by implementing more energy-efficient training or inference operations, CO2 emissions (e.g., emissions from power sources) associated with training and executing machine learning models can be reduced. By implementing more energy-efficient training or inference operations, the amount of thermal pollution in the environment (e.g., caused by processor / storage locations) can be reduced.
[0064] This article describes various example implementations with reference to the accompanying drawings.
[0065] Figure 1This is a block diagram of an example system for executing operations to process a request using multiple toolpaths. Request 102 may describe the task to be performed. Based on request 102, tool control system 104 may provide input to machine learning-based sequence processing model 106 to cause machine learning-based sequence processing model 106 to generate multiple outputs containing instructions for tool invocation to execute steps that satisfy request 102. Tool control system 104 may parse the outputs of machine learning-based sequence processing model 106 to obtain tool invocations. Tool control system 104 may pass tool invocations to tool execution system 108. Each tool invocation may be executed independently in a different toolpath. For example, N A tool call can be executed to initiate N Tool paths 110-1, 110-2, ..., 110- N The output value generated by the corresponding tool path can be passed back to the tool control system 104 for evaluation. This loop can be repeated iteratively: based on the intermediate output from the tool path, the tool control system 104 can generate a new tool call to perform a second processing step. Once the tool control system 104 determines that the processing has been completed sufficiently to satisfy request 102 (or determines that request 102 cannot be satisfied), the tool control system 104 can return a response 112.
[0066] Request 102 may include various data types and formats. For example, Request 102 may contain a text description of the task. Request 102 may include structured text (e.g., JSON) or unstructured text (e.g., free-form natural language). Request 102 may include image data, audio data, video data, multidimensional data arrays (e.g., point clouds), etc.
[0067] Request 102 can be generated from various sources. For example, a user can directly input the request through a user interface (e.g., a graphical user interface, a command-line user interface, an audio-based user interface, a haptic user interface, etc.). In one example, request 102 may include natural language text data transcribed from an input audio signal from a microphone on a user's device. Request 102 can be automatically generated by another system or service that needs to perform a task. Request 102 can be generated by an upstream generative machine learning model that generates request 102 to initiate the execution of a subtask of a larger task.
[0068] Request 102 may also include metadata that provides additional context or instructions for the task to be performed. This metadata may include priority levels that can be used to schedule and manage the execution of the task, or it may specify user preferences, such as the desired output format or quality settings for the task. Security credentials, such as tokens or keys, may also be part of Request 102, enabling the system to authenticate the request and access the protected resources or tools used to complete the task. For example, some tools may provide access to protected resources (e.g., calendar applications, email applications, etc.). Request 102 may include credentials or tokens that can be passed to a tool to authorize its execution to perform the requested task. This information may be in the form of an API key, an OAuth token, or user credentials.
[0069] Request 102 can be received and processed in real time. Request 102 can also be retrieved from a cache or queue for offline processing.
[0070] Request 102 may include configuration parameters for configuring the request to satisfy it. Example configuration parameters may be used to configure the operation of the tool control system 104, the machine learning model 106, etc. Example configuration parameters may include a list of tools that can be used to satisfy the request. The tool control system 104 may maintain a list of available tools that can be used by the model, or the tool control system 104 may use tools identified in request 102.
[0071] The tool control system 104 may include any computing system that can use a machine learning-based model 106 to control the operation of one or more tools. The tool control system 104 may host and execute the machine learning-based model 106. The tool control system 104 may interact with the machine learning-based model 106 executed on different external systems.
[0072] The tool control system 104 can execute processing logic configured to receive request 102 and generate input for the machine learning model 106 to perform processing steps that satisfy request 102. The tool control system 104 can also directly pass request 102 to the machine learning model 106. Furthermore, the tool control system 104 can parse request 102 to extract content for input to the machine learning model 106 and generate an input data structure configured for processing by the machine learning model 106.
[0073] For example, the tool control system 104 may receive request 102 via an API, in a format defined for communication via the API. A portion of request 102 may include content for input to the machine learning model 106. A portion of request 102 may include content for communication with the tool control system 104 (e.g., configuration parameters, authentication tokens, etc.). The tool control system 104 may generate an input structure for input to the machine learning model 106 based on the content included in request 102 for input to the machine learning model 106.
[0074] The tool control system 104 can preprocess request 102 to identify one or more tasks or subtasks to be performed to satisfy request 102 and generate input to a machine learning model 106 to perform the one or more tasks or subtasks. For example, the tool control system 104 can use a machine learning model (e.g., machine learning model 106) to process request 102 to determine which tasks need to be performed. For example, the model can return a list of tasks. The tool control system 104 can iterate through the task list, instructing the machine learning model 106 to perform each task in sequence.
[0075] The tool control system 104 can provide deterministic logic that frameworks the inputs and outputs of a probabilistic machine learning model. This deterministic logic can include structured workflows. For example, different workflows can be configured for different types of requests. For instance, a request related to trip planning might trigger a workflow that includes a list of predetermined steps or stages, such as opening a calendar application, opening a map application, etc. This deterministic logic can include output validation. For example, JSON validation patterns can implement programming logic configured to require the model to generate only valid JSON. For example, the tool control system 104 can monitor the generation of model 106 and constrain output samples to those that conform to valid JSON syntax (e.g., by prescriptively constraining the output space or by resampling any invalid output until valid samples are obtained). Other output validations can be used to deterministically constrain the operation of the tool to conform to predefined formats, vocabularies, styles, etc.
[0076] The tool control system 104 may include deterministic logic to constrain tool calls generated by model 106 to be valid tool calls. Valid tool calls may include calls to actual, active tools that can be operationally invoked using commands generated by model 106. Valid tool calls may include calls containing flags, parameters, or other inputs that conform to a defined set of flags, parameters, or other inputs. Validating a tool call in this manner may not necessarily cause the call to perform the desired function specified in request 102, but valid tool calls are unlikely to generate errors. The tool control system 104 may detect the generation of output portions that do not conform to valid tool calls (e.g., generating flags that are not defined flags associated with a given tool) and initiate resampling of that output portion. The tool control system 104 may manipulate one or more output layers of model 106 so that the output is sampled only from a subset of the output vocabulary that would result in a valid tool call for a given inference step.
[0077] The tool control system 104 may include an interface to the machine learning-enabled sequence processing model 106. This interface may be designed to handle the model's input and output formats. For example, the tool control system 104 may format request 102 into a representation compatible with the machine learning model and then interpret the model's output as a tool invocation. For instance, the machine learning model 106 may output text content that may not be directly executable in its raw state. The tool control system 104 may parse the output text content to extract executable tool invocations and initiate the execution of the tool invocations.
[0078] The tool control system 104 may include a scheduling and orchestration layer that manages the execution order and dependencies of tool calls. This layer can ensure that tool calls are executed in the correct sequence and manage data flow between tool calls when dependencies exist or when the output from one tool is the input to another. For example, the tool control system 104 can identify any necessary software packages that need to be installed to execute the tools invoked in the tool calls. The tool control system 104 can install or update these software packages. Alternatively, the tool control system 104 can compile a list of available tools and their input patterns and append this list to (or add before) the content from request 102 as input to the machine learning model 106.
[0079] The tool control system 104 may include monitoring and logging capabilities to track the status and performance of tool calls. These capabilities can aid in debugging, performance optimization, and compliance with auditing requirements. For example, the tool control system 104 may log the start time, end time, and result of each tool call, along with any errors or warnings that occur during execution.
[0080] The tool control system 104 may include a failure management system that handles errors and exceptions caused by tool invocations. This system can implement retry logic, alternative path execution, or error notification to ensure robustness in the face of tool failures or unexpected results. For example, if a tool invocation fails due to a network timeout, the tool control system 104 can automatically retry the invocation or route the task to a different network endpoint. If the tool returns an error message, the tool control system 104 can automatically request a new tool invocation from model 106.
[0081] The tool control system 104 may include a resource allocation subsystem that manages the distribution of computing resources across different tool paths. This subsystem can allocate CPU time, memory, and I / O bandwidth to tool paths in a manner that improves overall system throughput and ensures fair resource sharing. For example, the tool control system 104 can dynamically allocate more resources to known resource-intensive operations.
[0082] The tool control system 104 can be configured to construct a context history of actions in the toolpath. This context history can record tool calls output by model 106, as well as any output or error messages received in response to the execution of those tool calls. This context history can be fed into model 106 with each new generation, conditioned on the output of model 106, allowing the model to learn from previous actions, understand its current progress, etc. In one example, the context history can be formatted as a dialogue or chat sequence of input and response messages. In another example, the context history can have other suitable formats, such as transcribed text, records, JSON structures, etc.
[0083] The machine learning-based sequence processing model 106 can be any type of machine learning model configured to process input sequences and generate predictive output sequences. These models can be based on neural network architectures such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), Transformers, etc.
[0084] The machine learning-based sequence processing model 106 may include pre-trained components that have been trained (e.g., in an unsupervised manner) on a large text corpus or other data corpus. The machine learning-based sequence processing model 106 may also include components that have been fine-tuned using a selected dataset more closely aligned with the type of task the system is expected to perform. For example, the model may be pre-trained on a general text corpus and then fine-tuned using selected text examples in response to calls from various input generation tools.
[0085] The machine learning-based sequence processing model 106 can use an ensemble approach that combines predictions from multiple models. The machine learning-based sequence processing model 106 can also use expert fusion techniques, which route different requests to different sub-models specializing in various domains. For example, one model might be more accurate at generating file manipulation commands, while another might excel at network-related commands. Using multiple such models allows the machine learning-based model 106 to handle a wider range of tasks more effectively.
[0086] The machine learning-based sequence processing model 106 can receive updates from a feedback mechanism that allows the system to learn from the results of previously executed tool paths. This mechanism can use the results of previous tool executions to update the model's parameters. For example, if a tool call sequence continues to produce successful results, the model can be updated (e.g., by adjusting its weights) to favor similar sequences in the future.
[0087] The machine learning-based sequence processing model 106 can be configured to process various types of data. The machine learning-based sequence processing model 106 can be configured to generate text commands based on text input and image-based or audio-based input. The machine learning-based sequence processing model 106 can also be configured to generate text commands based solely on text input or other symbolic input.
[0088] The machine learning-based sequence processing model 106 can output commands for performing at least the steps associated with request 102. The machine learning-based sequence processing model 106 can be a text-based command. For example, to perform a task or subtask related to moving files within a Linux environment, the machine learning-based sequence processing model 106 can output a string containing the command "mv". This string may include flags and values associated with those flags. The string may include the name of the target file (e.g., "file.txt") and the path to the destination folder (e.g., "~ / documents").
[0089] The machine learning-based sequence processing model 106 can output GUI-based commands. For example, to perform a task of exporting rendered media files from an application that does not expose a programming API, the machine learning-based sequence processing model 106 can output a string containing a description of the action to be performed using the GUI: for example, "Select the 'Export' action from the 'File' menu". This string may contain a reference to a GUI-based navigation tool to cause the GUI-based navigation tool to perform the description of the action.
[0090] Example GUI-based navigation tools can include machine learning models and techniques that can interact with a graphical user interface (GUI) to perform tasks. Example GUI-based navigation tools can include systems that can interact with a GUI using the same modality as a human user: the system can handle the rendering of the GUI and output system-level input commands (e.g., taps, clicks, etc.). This enables automated interaction with virtually any GUI without requiring customized application programming interfaces for each application. Example systems can ingest natural language instructions and image-based rendering of the GUI and generate commands to follow those instructions. Example GUI-based navigation tools can include pre-trained, sequence-based image processing models (e.g., models including visual Transformer models) adapted to receive instructions for actions performed using the GUI and generate low-level GUI interaction commands for interacting with the GUI (e.g., clicks, taps, selections, drags, text input, etc.).
[0091] The machine learning-based sequence processing model 106 can generate tool calls individually or in groups. For example, the machine learning-based sequence processing model 106 can generate a list of tool calls to be executed sequentially, and the tool control system 104 can continue to execute the steps of the sequence sequentially. The machine learning-based sequence processing model 106 can continue to instruct each step of the operation sequence in a round-robin manner, wherein each output of a tool in response to a tool call is fed back to the machine learning-based sequence processing model 106 with a request for the next step to be executed.
[0092] The machine learning-based sequence processing model 106 can continue to instruct each step of the operation sequence in a round-by-round manner, where a round subset is used to enable the tool, and a round subset is used to acquire information. In some implementations, decoding can be implemented differently for each subset. For example, the subset corresponding to tool activation can be resampled to obtain multiple different tool calls (which can be executed in different tool paths). The subset corresponding to data collection or delivery in between can be decoded using greedy sampling. This can reduce the degree of branching in possible different tool paths while providing a wide range of diversity attempts.
[0093] Resampling of tools along a sequence can be performed using either a depth-first or breadth-first search strategy. In a depth-first strategy, tool steps can be resampled. The entire trajectory following a tool step can be resampled once or multiple times. In a breadth-first strategy, tool steps can be resampled. The sequence up to the next tool step can be resampled once or multiple times. The first local trajectory can be popped until the next tool step and repeated. The stopping condition can be the first N completed trajectories.
[0094] In some cases, a subset of rounds can correspond to reasoning steps (e.g., thought chain reasoning). These subsets can be sampled using greedy sampling (for efficiency) or additional resampling to generate additional branches (for diversity).
[0095] The sequence processing model 106, which utilizes machine learning, can generate outputs in batches. For example, the batch dimension of the input or output structure allows a given part of the model (e.g., a set of weights, a set of cached activations, etc.) to be used simultaneously to perform inference on multiple input sequences.
[0096] The machine learning-based sequence processing model 106 can be configured to generate multiple output sequences distributed along the batch dimension for the same input sequence. For example, the input sequence could be an initial input generated based on request 102. To generate multiple distinct sampled outputs, the machine learning-based sequence processing model 106 can autoregressively sample different output values to append to the input sequence. Different sampled values can be appended to different output positions along the batch dimension, allowing each distinct output to be generated autoregressively in parallel. A simplified example is shown below.
[0097]
[0098] For attention-based models, attention values computed on the input sequence (e.g., key-value caches) can be shared across batches. Computed attention values can be copied along the batch dimension. Pointers to computed attention values can also be copied along the batch dimension, with these pointers pointing to the same computed attention values (e.g., to avoid the memory cost of copying these values).
[0099] This technique can be applied to generate multiple different tool calls to perform the task requested in the initial input. Furthermore, the technique can be applied in a branching manner: for steps in a multi-step tool path, multiple subsequent steps can be generated by the machine learning model 106.
[0100] A toolpath can be associated with one or more "rows" or batches. As a toolpath is pruned, restarted, or edited, the work generated up to that point can be reused as appropriate. For example, if only one tool call in the toolpath fails, the sequence state can be rolled back to before that tool call was added, and output generation for new tool calls can continue from that point. Previously computed attention values can be retained and used for the remainder of the sequence.
[0101] The machine learning-based sequence processing model 106 can output an output data structure containing tool calls. The tool control system 104 can parse the output data structure to extract the tool calls.
[0102] In one example, the machine learning-based sequence processing model 106 could be a language processing model and prompts for tool calls. For example, the prompts could provide example tool calls, including a list of available tools.
[0103] Tool invocations can include specific commands or sets of instructions that initiate a single operation or a series of operations within a tool path. Depending on the nature of the controlled tool, these instructions may be formatted as command-line arguments, function calls within a script, or API requests to a web service. For example, a tool invocation might be a command used to launch a data analysis tool with certain parameters, such as the location of the input data and the type of analysis to be performed.
[0104] Tool calls can include parameters and flags that modify the tool's behavior. These parameters can specify things like the input file path, output format, level of detail in the results, or any other configurable options supported by the tool. For example, a tool call to a file compression utility might include flags for setting the compression level or preserving the file's original creation date.
[0105] Example tool calls can include calls to web search tools. Example parameters for a web search tool can include the number of responses. Example parameters can include techniques used to return results, such as Top-K, random sampling, etc. Different sampled outputs can correspond to resampled search queries, different numbers of responses, etc. In this way, for example, multiple sampled tool calls can generate multiple different lists of search results.
[0106] Tool calls can include environment variables or context used by the tool's operations. This context can provide the tool with information about its runtime environment, such as the current user, system paths, or network settings. For example, a tool call might include a database connection string required for the tool to access a database, or it might include credentials used to authenticate with a cloud service.
[0107] Tool invocations can include dependency information to ensure that all required resources are available to the tool. This might involve specifying versions of libraries or other tools that the current tool depends on, or it might require the tool execution system 108 to set up certain services before the tool can run. For example, a tool invocation might specify that a certain Python package must be installed, or it might require that a virtual environment be activated before the tool is executed.
[0108] Tool calls may include traceability options or logging options to assist in monitoring tool execution and diagnosing problems. These options can guide tool execution system 108 to capture detailed logs of tool activity. For example, tool calls can enable detailed logging for complex operations, ensuring detailed information is available when operations do not produce the expected results. This information can be returned in updates, allowing tool control system 104 (e.g., using model 106) to identify errors and generate new tool calls.
[0109] Tool calls may include a unique identifier or transaction ID that links them to a specific instance of the request within the tool execution system 108. This identifier can be used to track the progress of a tool call at various stages of the tool path and to associate it with other related operations. For example, each tool call in a multi-step process may carry the same transaction ID to indicate that they are part of the same overall task.
[0110] The tool control system 104 can pass tool calls to the tool execution system 108 for execution.
[0111] The tool execution system 108 can be implemented by a computing device hosting a runtime environment in which tool calls generated by a machine learning-based sequence processing model 106 are executed. This environment can be configured to support a variety of software tools, from command-line utilities to full applications, and can be customized to provide the necessary computing resources, such as CPU, GPU, memory, I / O, and disk space. For example, the tool execution system 108 may include a containerized environment for each tool path. The tool execution system 108 may include a separate computing device or system (real or virtual) for each tool path. The tool execution system 108 can execute multiple tool paths on the same system but in separate or sandboxed file system portions (e.g., different relative root paths).
[0112] The tool execution system 108 may include a scheduler that allocates tool calls to appropriate execution environments. This scheduler can manage a queue of tool calls and assign them to execution slots based on availability and priority. For example, the tool execution system 108 can prioritize urgent tool calls by moving them to the front of the queue and executing them on dedicated high-performance hardware.
[0113] The tool execution system 108 may include monitoring and logging tools that capture the execution details of each tool call. These tools can track the start time, duration, exit status, and resource usage of each call. For example, logs can be used to identify bottlenecks in the tool execution process or to verify that a particular tool call is executed as expected.
[0114] The tool execution system 108 may include failover and redundancy mechanisms to ensure high availability and reliability of the tool execution process. These mechanisms can automatically reroute tool calls to backup systems or retry failed executions. For example, if an execution environment hosting a particular tool fails, the tool execution system 108 can seamlessly transfer tool calls to an alternative environment without interrupting the overall task flow or requiring new build commands from the tool control system 104.
[0115] The tool execution system 108 may include a resource management subsystem that dynamically adjusts the allocation of computing resources to each tool call or execution environment. This subsystem can scale resources up or down in response to the demands of a tool call, thereby optimizing the overall resource utilization of the system. For example, additional CPU cores can be dynamically allocated as needed for tool calls that require significant processing power.
[0116] The tool execution system 108 may include a security layer that implements strict access control and isolates the execution of tool calls to prevent interference between individual tool paths. This layer may include mechanisms for setting and revoking read and write access to shared resources, system resources, etc.
[0117] The tool execution system 108 may include interfaces for integration with external services and APIs, enabling tool calls to interact with a range of external systems and platforms. These interfaces may support various communication protocols and data formats. For example, the tool execution system 108 may include connectors for cloud storage services or other web-based APIs, allowing tool calls to store, retrieve, or submit data to these services as part of task execution.
[0118] The tool execution system 108 uses a separate root directory for each tool path as an isolation method to ensure that the execution of one tool path does not interfere with or compromise the integrity of another. Each root directory can act as an independent environment with its own set of subdirectories and files, thus mimicking the structure of a complete file system. For example, one tool path can operate within a root directory containing a specific version of a software library, while another tool path can use a different version of the library in its own separate root directory, preventing conflicts between the two.
[0119] Using a separate root directory in the tool execution system 108 promotes a clean and controlled execution environment for each tool path, allowing for customization of dependencies and configurations. By isolating the file system, the system can prevent problems such as shared state or resource contention, which can occur in environments where multiple processes can access the same files. For example, if a tool path needs to write temporary files to disk, having a separate root directory ensures that these files will not conflict with or overwrite temporary files in another tool path.
[0120] A separate root directory within the tool execution system 108 can enhance security by limiting the potential impact of compromised tool paths. If a tool path is designed to execute untrusted code or process sensitive data, isolating it within its own root directory can prevent malicious code from affecting other tool paths or accessing system-wide resources. In one example, if a machine learning-based sequence processing model 106 is generating code to be executed, treating all of that code as "untrusted" can help prevent incorrectly generated code from corrupting other areas.
[0121] The tool execution system 108 can use separate root directories to simplify the management of permissions and access control for each tool path. By assigning different user and group permissions to each root directory, the system can finely control which processes have read, write, or execute access to files and directories within the environment. For example, a tool path requesting elevated privileges can be restricted to the root directory that grants such privileges without affecting the security posture of other tool paths. Furthermore, to constrain the operations of a tool path to its assigned directory, the tool execution system 108 can deny write permissions to that tool path except for content within its designated root directory.
[0122] Using a separate root directory in the tool execution system 108 makes it easier to track resource usage and perform cleanup operations for each tool path. Since the files and processes of each tool path can be contained within their own directories, the system can accurately monitor and limit the amount of disk space, memory, and CPU time consumed by each tool path. Furthermore, once a tool path completes its execution or if it needs to be terminated, the system can quickly remove its root directory to release resources.
[0123] The tool execution system 108 can use containers to isolate the execution of different tool paths. Containers encapsulate the runtime environment of a tool (its code, runtime, system tools, libraries, and settings), making that environment distinct from other containers and the host system.
[0124] Using containers in the tool execution system 108 provides high portability for the tool path because containers can be easily moved between different host systems without requiring environment-specific configurations. Furthermore, using containers for tool execution provides continuity between runs because a new image can be loaded each time a tool call is initiated.
[0125] The tool execution system 108 can use container orchestration tools to manage the lifecycle of containers and automate the deployment, scaling, and operation of containerized toolpaths. These orchestration tools can handle tasks such as load balancing, health monitoring, and failover.
[0126] Containers in the tool execution system 108 can provide a sandboxed environment for each tool path. Containers can restrict processes running within them from making unauthorized changes to the host system or accessing resources allocated to other containers.
[0127] Using containers in the tool execution system 108 simplifies resource monitoring and allocation by providing tools for setting resource limits and reservations for CPU, memory, and I / O for each container. This prevents any single container from monopolizing system resources and ensures that each tool path has access to the resources it needs to execute optimally. For example, a system administrator can allocate a fixed amount of memory to a container to prevent it from impacting the performance of other containers.
[0128] The tool execution system 108 can benefit from the speed and repeatability of container startup and shutdown capabilities. Containers can start in seconds or less, enabling fast-response task execution.
[0129] The tool execution system 108 can use virtual machines (VMs) to isolate the execution of different tool paths, providing a robust separation layer by simulating a complete hardware system on which a complete operating system can run independently. Each VM can operate using its own virtualized hardware (including CPU, memory, and storage resources), thereby enabling complete isolation of the execution environment of the tool path.
[0130] Using virtual machines in the tool execution system 108 helps manage the diverse operating system environments required for different tool paths. Because VMs can run different operating systems, different OS environments can be enabled for different toolsets, thereby increasing the diversity of available pathways when performing a given task.
[0131] The virtual machines in the tool execution system 108 also provide a high degree of configurability and control over the computing environment. Virtualization allows for the precise allocation of resources to each VM, which can be customized according to the specific needs of the tool path, including customized network configurations, storage settings, and hardware emulation. This means that each tool path can be provided with exactly the resources it needs, thereby optimizing the use of the underlying physical hardware.
[0132] The tool execution system 108 can use virtual machines to enhance security and compliance by leveraging the strong isolation properties of a hypervisor, which is the software, firmware, or hardware that creates and runs the VMs. The hypervisor can enforce strict boundaries between VMs, ensuring that activities within one VM (such as processing sensitive data or running experimental code) are completely isolated from other VMs.
[0133] Tool paths 110-1, 110-2, ..., 110-N may include a series of discrete steps or operations generated or instructed by the tool control system 104 to fulfill request 102. Each tool path may represent a sequence of tool calls, which may include executable commands, function calls, API requests, or any other actions that can be performed by software tools, firmware tools, or hardware tools. For example, a tool path may involve a sequence of data acquisition, data transformation, and data analysis operations.
[0134] Response 112 may include the final output produced by tool control system 104 using commands generated by machine learning model 106, which are executed in tool execution system 108.
[0135] Response 112 may include metadata providing information about the execution of the request, such as execution time, resources used, or logs of the steps performed. This metadata can be used for auditing purposes, performance monitoring, or to provide insights into the processing of the request. For example, Response 112 may contain details about the completion status of each toolpath, including any errors encountered or corrective actions taken. Response 112 may include a confidence score or accuracy metric associated with the returned result.
[0136] Response 112 may include a link or reference to a location where the output or results can be accessed, especially when the output or results are too large to be included directly in the response, or when the output or results are stored in a cloud service or database. This can be useful for tasks that generate large datasets or files, allowing users to download or interact with the results at their convenience. For example, response 112 could provide a URL to a secure location where users can download the processed file.
[0137] Response 112 can be a direct output from at least one of the tool paths 110-1, 110-2, ..., 110-N. For example, request 102 can be a data acquisition and summarization task. Response 112 can be the requested summary generated using one of the tool paths.
[0138] Response 112 can be a composite of outputs from multiple toolpaths. Individual results from toolpaths 110-1, 110-2, ..., 110-N can be aggregated, compared, or synthesized to form a coherent final output. This composite output can leverage the diversity of toolpaths to improve the accuracy and reliability of the response by confirming similar results or by combining different perspectives on the task. For example, if multiple toolpaths are used to analyze a dataset from different angles, response 112 can integrate these analyses into a comprehensive report that provides a more complete understanding of the data. This approach can also be used for cross-validation of results.
[0139] In one example, multiple toolpaths can perform data acquisition and summarization tasks. The output from each toolpath can be fed into a machine learning model (e.g., model 106) to generate a summary that utilizes all the individual outputs.
[0140] Response 112 can be selected using a consistency metric as described herein. A consistency metric can be used to compare outputs to assess whether there is a majority or relative majority alignment on output features. For example, a consistency metric can be evaluated for sampling of diverse toolpaths (e.g., representing diverse approaches to resolving a query) and their corresponding outputs. For example, a set of outputs with diverse tool execution strategies can be aggregated to obtain a majority or relative majority "vote" regarding the final output. In this way, toolpath outputs can self-validate to improve the robustness of the model output and improve the accuracy of the final response.
[0141] Figure 2This is a block diagram of an example tool path 200. Tool path 200 may include a sequence of tool executions and responses managed iteratively to perform tasks or subtasks. A tool path may be an example of one of tool paths 110-1, 110-2, ..., 110-N. Tool control system 104 may issue tool call 202. Tool execution 204 may be executed by tool execution system 108 to produce tool response 206. Tool response 206 may include tool output, error messages generated by tool execution system 108 in case of tool execution failure, etc. Tool execution system 108 may return an update 208 based on tool response 206. Tool control system 104 may provide another tool call 210 to be executed next, which uses any output of the previous tool execution 204. Tool call 210 may be executed at tool execution 212 to generate tool response 214. Tool execution system 108 may return an update 216 based on tool response 214. The tool control system 104 can provide another tool call 218 to be executed next, which uses any output of the previous tool execution 212. Tool call 218 can be executed at tool execution 220. This sequence can be repeated until the completion of the task or subtask requested in the update instruction request 102 returned to the tool control system 104 is determined, or until another stopping condition is met (e.g., a failure condition is detected).
[0142] Tool invocation 202 may include a specific set of commands or instructions that initiate a single operation or a series of operations within tool path 200. Depending on the nature of the controlled tool, these instructions may be formatted as command-line arguments, function calls within a script, or API requests to a web service. For example, tool invocation 202 might be a command to launch a data analysis tool with certain parameters, such as the location of the input data and the type of analysis to be performed.
[0143] Tool invocation 202 may include parameters and flags that modify the behavior of tool execution 204. These parameters may specify things like input file paths, output formats, the level of detail in the results, or any other configurable options supported by the tool. For example, a tool invocation for a file compression utility might include flags for setting the compression level or preserving the original creation date of the file.
[0144] Tool call 202 may include environment variables or context used by the tool to perform the 204 operation. This context can provide the tool with information about its runtime environment, such as the current user, system paths, or network settings. For example, tool call 202 may include a database connection string required for the tool to access a database, or it may include credentials used to authenticate with a cloud service.
[0145] Tool invocation 202 may include dependency information to ensure that all required resources are available for tool execution 204. This may involve specifying versions of libraries or other tools that the current tool depends on, or it may require the tool execution system 108 to set up certain services before the tool can run. For example, tool invocation 202 may specify that a certain Python package must be installed, or it may require that a virtual environment be activated before the tool is executed.
[0146] Tool invocation 202 may include traceability options or logging options to assist in monitoring tool execution and diagnosing problems. These options can guide tool execution system 108 to capture detailed logs of tool activity. For example, tool invocation 202 can enable detailed logging for complex operations, ensuring detailed information is available when operations do not produce the expected results. This information can be returned in an update, allowing tool control system 104 (e.g., using model 106) to identify errors and generate new tool invocations.
[0147] Tool invocation 202 may include a unique identifier or transaction ID that links it to a specific instance of the request within tool execution system 108. This identifier can be used to track the progress of the tool invocation at various stages of tool path 200 and associate it with other related operations. For example, each tool invocation in a multi-step process may carry the same transaction ID to indicate that they are part of the same overall task.
[0148] Tool calls 210 and 218 can have various properties similar to those of tool call 202.
[0149] Tool execution 204 may include the process of running specific commands or instructions defined in tool invocation 202 within the operating environment provided by tool execution system 108. This execution may involve invoking a software application, script, or utility and passing it necessary parameters and data. For example, tool execution 204 may involve running a data processing script that takes raw data as input and produces a cleaned dataset as output.
[0150] Tool execution 204 may include the allocation of necessary system resources, such as CPU, memory, and disk space, to ensure that the tool has the required capacity to perform its functions. This allocation can be dynamically managed by the tool execution system 108 based on the tool's needs and resource availability. For example, a tool performing resource-intensive computations may be allocated additional processing power during its execution phase.
[0151] Tool execution 204 may include monitoring the tool's performance and health during tool operation, capturing metrics such as execution time, resource usage, and throughput. This monitoring can help identify any performance bottlenecks or failures that occur during tool execution. For example, if the tool unexpectedly consumes more memory than expected, tool execution 204 may include mechanisms to log the event and take corrective action.
[0152] Tool execution 204 may include processing the input and output data streams of the tool being executed. This may involve reading data from files, databases, or network sources, and writing the output data to appropriate destinations. For example, tool execution 204 may read input data from a secure cloud storage service and write the processed results to a local file system.
[0153] Tool execution 204 can include the parallel execution of multiple instances of a tool, especially when the tool path is part of a distributed system designed for high throughput or redundancy. This parallelism can help achieve faster processing times and provide fault tolerance. For example, tool execution 204 might involve running several instances of a data analysis tool across different nodes in a computing cluster.
[0154] Tool execution 204 may include integration with external services or APIs that provide the additional functionality required by the tool. This may involve sending requests to web services, invoking cloud functions, or interacting with third-party software through exposed interfaces. For example, tool execution 204 may include calls to external APIs (e.g., APIs outside of tool execution system 108).
[0155] Tool executions 212 and 220 can have various properties similar to those of tool execution 204.
[0156] Tool response 206 may include direct output generated by the execution of tool call 202. For example, if tool call 202 indicates a database query, tool response 206 may include the records retrieved or confirmation of successful query execution, as well as a pointer to the result location.
[0157] Tool response 206 may include error messages or diagnostic information generated by tool execution system 108 when a problem occurs during the execution of tool call 202. This information enables tool control system 104 to make informed decisions about how to continue, whether by retrying the operation, executing an alternative tool call, or aborting the process by completely pruning the tool path. For example, if a network operation fails, tool response 206 may include details about the nature of the network error.
[0158] Tool response 206 may include metadata related to the tool's execution, such as execution time, system resource usage, or the version information of the tool used. For example, tool response 206 may detail the amount of CPU and memory used during execution, which can be used to optimize resource allocation in future executions.
[0159] If the tool execution is part of a larger, multi-step process within the tool path, the tool response 206 may include partial results or intermediate data. These partial results can be used as input for subsequent tool calls, enabling iterative data processing or incremental refinement of results. The tool response 206 can be streamed or computed entirely.
[0160] When tool execution involves predictive modeling or heuristic evaluation, tool response 206 may include a quality score or confidence score. This score can provide insight into the reliability of the output and can be used to determine whether additional verification or validation is needed. For example, in a machine learning classification task, tool response 206 might include a confidence score for each class prediction, indicating the model's certainty in its classification. Tool control system 104 can use the confidence score to assess the feasibility of continuing with tool path 200. For example, tool path 200 may include the use of probabilistic tools. Another tool path may use different tools. Based on the confidence score output by the probabilistic tool, tool control system 104 can determine whether to rely on the probabilistic tool output and perform further computation in tool path 200, or to prune tool path 200.
[0161] Tool response 214 can have various properties similar to tool response 206.
[0162] Update 208 may include data generated by tool call 202 (e.g., tool response 206). Update 208 may include status information about the execution of tool call 202. This status information can provide insight into whether the tool call was executed successfully, encountered warnings, or resulted in errors. For example, update 208 may include execution status codes such as "success" or "error," along with relevant messages detailing the nature of the execution result. In scenarios where tool call 202 is part of a batch or pipeline, update 208 may also include progress indicators, such as percentage of completion or step number, to inform the tool system 104 of the current stage within the tool path.
[0163] Update 208 may include diagnostic data generated during tool execution 204. This diagnostic data can help troubleshoot problems that occurred during tool invocation 202. For example, update 208 may include log files, stack traces, or memory dumps captured when errors occurred. Additionally, update 208 may provide performance metrics such as execution time, memory usage, and CPU load, which can be used to optimize future executions of similar tool invocations. In cases where tool execution 204 interacts with external systems, update 208 may also include network diagnostics, such as response times and error codes from those systems.
[0164] Update 208 may include output data or results from tool execution 204. The output data can vary greatly depending on the nature of the tool and the task it performs. For example, if tool call 202 involves data processing, update 208 may contain the processed data or a summary thereof. If the tool call involves a query or search operation, update 208 may include information obtained or references to where such information can be accessed. In the context of a computational task, update 208 may also include the results of a computation or simulation. The format of the output data in update 208 can be structured, such as JSON or XML, or unstructured, such as plain text or binary data.
[0165] Update 208 can be parsed and interpreted by the tool control system 104 to determine the next steps in the tool path. The tool control system 104 can pass Update 208 to the machine learning-based sequence processing model 106 and request the next action. If the update instruction is executed successfully and the result is valid, the machine learning-based sequence processing model 106 can generate the next tool call in the sequence to continue advancing the task. In cases where Update 208 includes an error message, the machine learning-based sequence processing model 106 can use this data to identify and troubleshoot problems. For example, if a tool call results in an error due to a missing dependency, the machine learning-based sequence processing model 106 can generate instructions to perform corrective actions (such as installing necessary dependencies). Alternatively, if the error is unrecoverable, the machine learning-based sequence processing model 106 can generate instructions to prune the current tool path and, alternatively, initiate a new tool path using new tool calls sampled from the machine learning-based sequence processing model 106. By providing intermediate states to the machine learning-based model 106, the machine learning-based sequence processing model 106 can generate more accurate and context-aware tool calls that build upon previous steps, bringing the tool path closer to task completion.
[0166] Update 216 can have a variety of properties similar to those of Update 208.
[0167] Tool path 200 can use data persisted between executions. Tool execution system 108 can employ various data persistence methods to ensure data is retained and available between different tool executions. One method for persisting data is by using temporary files or directories created on the file system. Output from a tool execution can be written to a file, which can then be read by the next tool in the sequence. For example, an image processing tool can save the processed image as a file, which can then be accessed by a subsequent tool performing image recognition.
[0168] Another approach involves using in-memory data structures to pass data directly between tool executions. While tools execute within the same runtime environment or process, data can be stored in variables, arrays, or other data structures that can be accessed by subsequent tool calls without requiring disk reads and writes.
[0169] Databases or data caches can also be used to persist data between tool executions. After a tool completes its execution, the output can be inserted into a database table or stored in a cache. Subsequent tool calls can then query the database or access the cache to retrieve the necessary data.
[0170] Tool paths can be linear. For example, after initially sampling multiple tool calls from the output of a machine learning-based sequence processing model 106, subsequent tool calls for that tool path can be a single tool call. For example, tool path 200 contains a linear sequence of tool calls.
[0171] Figures 3A to 3C Example instructions for the branching tool path tree 300 are provided. Figure 3A The initial tool call 302 and tool execution 304 are illustrated. The tool response 306 can be returned to the tool control system 104. The tool control system 104 can process the tool response 306 using a machine learning-based sequence processing model 106 to determine the next step.
[0172] Figure 3B This demonstrates that multiple options can be explored to evaluate the next step. Tool call 308 can be executed at tool execution 310. Tool execution 310 can be included or otherwise isolated from the rest of the tool path tree 300. Tool execution 310 can generate a tool response 312.
[0173] Independent of tool execution 310, tool control system 104 can output another tool call 314, which provides an alternative path from the branch of tool execution 304. Tool call 314 can be executed at tool execution 316 to generate tool response 318. Tool control system 104 can evaluate both tool response 312 and tool response 318 to determine how to proceed. For example, the input sequence including tool responses 312 and 318, respectively, can be processed by machine learning model 106 to generate the next steps.
[0174] Figure 3C The implementation of subsequent steps based on tool responses 312 and 318 is illustrated. The tool control system 104 may determine to prune the branch including tool execution 310 (e.g., based on determining a failure condition according to tool response 312). The tool control system 104 may determine to initiate a new branch using tool call 320 to process tool response 306 at tool execution 322. The tool control system 104 may further initiate processing of tool response 318 using tool call 324 executed at tool execution 326.
[0175] The tool path tree 300 can branch further. The tool path tree 300 can maintain up to a predetermined maximum number of branches.
[0176] Figure 4 This is an example validator 400. The tool control system 104 can use validator 400 to calculate a consistency metric. Validator 400 can process tool path states 402-1, 402-2, and 402-N. Validator 400 can process tool path states 402-1, 402-2, and 402-N in response to request 102. Validator 400 may include one or more probabilistic evaluators 404 and one or more deterministic evaluators 406. Validator 400 can process tool path states 402-1, 402-2, and 402-N to generate an evaluation 408. Evaluation 408 may include selecting an output for response 112 based on one or more selected tool path states.
[0177] Tool path states can include tool responses or updates generated based on them. Tool path states 402-1, 402-2, and 402-N can each come from different tool paths or different branches of the same tool path tree.
[0178] Tool path status can include the tool's output or information returned from the tool. Tool path status can be a completed state (e.g., marking the end of the tool path after which no further tool calls are needed) or an intermediate state (e.g., providing intermediate results or fetching information).
[0179] The probabilistic evaluator 404 may include one or more machine learning models capable of evaluating the quality of tool path states. The probabilistic evaluator 404 may include one or more machine learning embedding models configured to embed tool path states into vector representations that can be quantized for comparison to determine similarity. For example, vector-based similarity measures may be used to determine consistency or alignment between tool path states. For example, tool path states may include image data, and embeddings representing images may be used to compare the semantic content of the images. Consistency measures may be based on the degree of similarity or alignment between image embeddings. Embeddings may also be used to determine whether the content of an image aligns with the requested content identified in request 102.
[0180] For example, a consistency metric used to evaluate the quality of each output state can be determined using a machine learning model. This machine learning model could be the same model used to generate the multiple toolpaths (e.g., a machine learning sequence processing model 106), or a different model specifically trained or adapted for the evaluation task. For instance, the same machine learning model generating the toolpaths might also be able to rate the quality of the text output states. This could be due to the model being trained on a large corpus of text data, enabling it to understand the context and relevance of the output. Alternatively, if different machine learning models have been fine-tuned on a dataset more closely aligned with the evaluation criteria (such as a dataset of expert-validated text outputs), those different machine learning models can be used for evaluation. Evaluation of output states can involve a variety of techniques, including but not limited to semantic analysis, syntactic comparison, or statistical evaluation. For example, a machine learning model could compute a vector-based representation of the text output and use a similarity metric (such as cosine similarity) to evaluate the quality of the output. In another example, the model could apply classification techniques to categorize the outputs based on predefined quality tiers, each tier representing a range of quality scores.
[0181] The probabilistic evaluator 404 may include a machine learning-enabled model configured to receive inputs describing multiple toolpath states. The validator 400 may use the machine learning-enabled model to evaluate the quality of each toolpath state by processing the multiple output states together to output a preferred state. The final response (e.g., response 112) or the next intermediate state may be selected using the preferred state. The preferred output state may be the state that the model identifies as the most accurate, complete, or reliable based on its received training, which may include patterns of consistency, correctness, or other quality metrics relevant to the current task. For example, if the task involves generating text summaries from multiple documents, the machine learning-enabled model may process various summaries (output states) to identify which summary most tightly captures the document's information with minimal noise or irrelevant data. Similarly, for image processing tasks, the model may compare different processed images (output states) to determine which image has characteristics that match the desired output criteria.
[0182] Once the machine learning model identifies the preferred output state, that state can be used to select the response to the request. The final output delivered as a response to the task request can be the output that the machine learning model has evaluated as the best among the options generated by the toolpath.
[0183] The probabilistic evaluator 404 may include one or more machine learning models trained to identify and evaluate significant differences between states.
[0184] The probabilistic evaluator 404 may include one or more machine learning models configured to identify failure states of the tool path. The probabilistic evaluator 404 may include a classifier model configured to classify responses as indicating a failure state or not indicating a failure state.
[0185] The probabilistic evaluator 404 may include a machine learning model 106. The machine learning model 106 may include a robust multi-task model that can be trained to follow different instructions to perform different tasks. The validator 400 can perform different verification tasks by providing input data to the machine learning model 106 along with one or more instructions for the type of verification to be performed. For example, input prompts to the machine learning model 106 may include data describing the state of the toolpath and instructions for comparing the state data.
[0186] In one example, a machine learning model 106 can process inputs describing the toolpath state and request 102, and generate outputs an indication predicting whether the toolpath state advances the goal indicated in request 102. The inputs can be configured to cause the model to generate new tool calls, pruning instructions, stopping conditions, or retry conditions. For example, the inputs could include textual instructions in a prompt to cause the model to behave in this way.
[0187] The deterministic evaluator 406 may include one or more heuristics that can be applied to evaluate states. Heuristics may include byte-level difference calculations that quantify the difference between two inputs. Heuristics may include file type or file attribute checks. For example, if request 102 requests an operation to produce an output GIF image, the heuristic may include a file type matching operation that matches the output file type with the file type indicated in request 102. Heuristics may include file size or dimension comparisons. Outlier file sizes or files with outlier attributes may be discarded because they are inconsistent with the majority value of file size or other attributes. Heuristics may include other attribute comparisons. For example, a consistency measure can be determined by evaluating how many states indicate a specific feature and determining whether a majority or relative majority of states have the same feature. Each of multiple features can be compared in this way.
[0188] The validator 400 can use heuristic criteria to quickly filter out output states that fail to meet basic requirements (such as correct file format or complete data fields) before applying probabilistic criteria to further evaluate the quality of the remaining outputs. This hierarchical evaluation process can improve system efficiency by reducing the number of outputs that need to be processed by computationally intensive probabilistic evaluations.
[0189] Generally, validator 400 can identify one or more features shared by a set of states among the multiple output states. Validator 400 enables the system to select a final response 112 from this set (e.g., as a representative example). This set can be obtained by clustering the feature data. Selecting a response from this cluster can include determining the centroid of the cluster and selecting a response based on that centroid.
[0190] Features can be latent or explicit. For example, latent features can be features computed from state data using a machine learning model. Explicit features can be features that can be resolved or extracted from state data in a deterministic manner.
[0191] Evaluation 408 may include selecting data to include in the response (e.g., response 112) based on a consistency metric. Evaluation 408 may include the value of the consistency metric. Evaluation 408 may include determining failure conditions for one or more toolpath states. Evaluation 408 may include determining that an intermediate state has been successfully executed and further processing is necessary.
[0192] Verifier 400 can determine that one or more tool paths are redundant. Verifier 400 can cause tool control system 104 to prune at least one tool path based on the determination of its redundancy. Redundancy can be determined based on deterministic results of identifying the at least one tool path. Redundancy can be determined based on a similarity margin. The similarity margin can be based on a minimum step size associated with the range of parameter values for tool calls.
[0193] Using evaluation 408 from validator 400, tool control system 104 can adjust at least one tool path based on determining failure conditions associated with it. Failure conditions can be determined based on receiving a tool response that deviates from expected criteria. Failure conditions can be determined based on receiving an error message indicating inoperability of a tool or tool call. Adjusting a tool path may include pruning the at least one tool path. Adjusting a tool path may include restarting the at least one tool path using one or more newly generated tool calls from a machine learning model. Restarting at least one tool path may include generating new tool calls based on failure conditions. Generating new tool calls based on failure conditions may include inputting data describing the failure conditions as contextual input to a query for the new tool call into the machine learning model.
[0194] Example Method
[0195] Figure 5 A flowchart of the example method 500 is depicted. One or more parts of the example method 500 can be implemented by a computing system including one or more computing devices, such as the computing system described with reference to other figures. Each corresponding part of the example method 500 can be executed by any computing device (or any combination thereof) of the one or more computing devices. Furthermore, one or more parts of the example method 500 can be implemented on hardware components of the apparatus described herein, for example, for training one or more systems or models. Figure 5 For illustrative and discussion purposes, elements executed in a specific order are depicted. Those skilled in the art will understand using the disclosure provided herein that elements of any method discussed herein can be adapted, rearranged, extended, omitted, combined, or modified in various ways without departing from the scope of this disclosure. Figure 5The elements / terms described are for illustrative purposes only and with reference to other systems and figures, and are not intended to be limiting. One or more portions of example method 500 may be performed additionally or alternatively by other systems.
[0196] At 502, example method 500 may include receiving a request for performing a task. In some implementations of example method 500, this request may be request 102. In some implementations of example method 500, receiving a request for performing a task may involve obtaining input specifying an action or a series of actions to be performed. The request may take various forms, such as a text description, a voice command, an image, a set of parameters, or any combination thereof. For example, the request may be natural language text input, such as “convert JPEG image to PNG format”, or it may be more structured input, such as a JSON object describing the task requirements. The request may come from an end user via a user interface or be generated programmatically by another system or service that needs to perform the task. The request may also include additional metadata or context providing further details about the task, such as priority level, expected output format, or execution constraints.
[0197] At 504, example method 500 may include generating multiple tool paths (e.g., tool path 110) for performing a task using a machine learning model, wherein each tool path includes one or more tool calls generated by the machine learning model, each tool call being configured to cause the corresponding tool to perform steps of the task. In some implementations of example method 500, generating multiple tool paths for performing a task using a machine learning model may involve enabling the machine learning model to generate a series of instructions or commands that, when executed, are configured to result in the completion of the task specified in the request. Each tool path may represent a different approach or strategy for completing the task, potentially involving different tools, parameters (for the same or different tools), or sequences of operations. The machine learning model may be configured to predict and output tool calls appropriate to the task context and may take into account various factors such as the capabilities of available tools, the efficiency of different approaches, and the probability of success. For example, if the task is to convert a file format, the machine learning model may predict tool calls for different file conversion utilities, each with specific flags and options that affect the conversion process. These multiple tool paths may be generated in parallel, allowing for concurrent exploration of multiple potential solutions to the task.
[0198] At 506, example method 500 may include selecting a response to a request (e.g., response 112) based on a consistency metric determined by evaluating multiple output states from the plurality of tool paths, respectively. In some implementations of example method 500, selecting a response to a request based on a consistency metric determined by evaluating multiple output states from the plurality of tool paths may involve comparing results or intermediate outputs obtained from executing tool calls along various tool paths. A consistency metric can be used as a measure of consistency or reliability among outputs. Higher consistency can be a stronger signal that the output is correctly processing the task. Evaluation may involve techniques such as vector similarity measures, semantic analysis, or heuristic rules to assess the level of consistency. For example, if multiple tool paths generate outputs that all contain a certain file attribute or data structure, this consistency can be interpreted as a positive signal, and the response to the request can be based on these consistency characteristics. Conversely, outputs that significantly deviate from the majority may be considered less reliable and may be given less weight or discarded in the selection of the final response.
[0199] At 508, example method 500 may include returning a response. In some implementations of example method 500, returning a response may involve delivering selected output to the requesting party or a downstream system. The response may be formatted or packaged in a manner suitable for the recipient, such as embedding the output in a message, storing the output in a specified location, or transforming the output into a desired format. For example, the response may be a direct file download link, an API response object, or a visual representation displayed to the user. The return of the response may also include providing additional information, such as execution details, confidence levels, or explanatory notes to help the recipient understand or verify the output. Depending on the requirements of the task and the capabilities of the system executing example method 500, the response may be transmitted over a network, saved to local or remote storage, or presented through a user interface.
[0200] In some implementations of the example methods, the machine learning model is a sequence processing model. For example, the machine learning model could be Sequence Processing Model 4. The machine learning model can be configured to receive input including one or more input portions (in a sequence) and generate output including one or more output portions (in a sequence) conditioned on the one or more input portions. The output portions can be generated autoregressively. In some implementations of example method 500, the conditioned output portion on the input portion can be achieved through various mechanisms within the machine learning model. For example, the model can use attention mechanisms, recurrent layers, or other architectural features that enable it to consider the input portion when generating the output portion. This ensures that the output is context-relevant and maintains coherence with the input. For example, in a file conversion task where the input portion specifies the target file format and conversion options, the output portion can include files that have been converted according to these specifications, ensuring that the output is conditioned on the input in a meaningful way.
[0201] In some implementations of example method 500, the one or more output portions can be generated in a manner that enables iterative or recursive processing. An initial output portion can be used as an intermediate result, which is then used as part of the input to generate subsequent output portions. For example, in a multi-step computation task, the output from the initial tool execution can be used as input for the next step in the task, with the machine learning model generating the output for each step based on the accumulated context provided by previous inputs and outputs. This approach can facilitate the execution of complex tasks requiring sequences of interdependent operations, where each output portion is conditioned on previous input and output portions.
[0202] In some implementations of Example Method 500, conditioned the output on the input may also involve an external validation or verification step. For example, after generating the output, the system can use additional tools or models to check the accuracy of the output, its compliance with task requirements, or its alignment with the expected result. This validation step can be input-conditioned because it compares the output to the task specifications provided in the input. If the output passes validation, it can be returned as part of the response; otherwise, the system can adjust the toolpath, resample the output, or take other corrective actions to ensure that the final output is properly conditioned on the input.
[0203] In some implementations of example method 500, the one or more output portions conditioned on the one or more input portions can be represented in various formats suitable for different types of tasks. For example, the output portions can be text data for natural language processing tasks, binary data for file operations, numerical data for computational tasks, or any other data type related to the specific task being performed. The machine learning model can be configured to generate these output portions in the desired format by conditioned on the input portions, which may include format specifications or other relevant data. This flexibility allows the system to adapt to a wide range of tasks and output requirements, providing customized responses conditioned on the provided inputs.
[0204] In some implementations of example method 500, generating the multiple toolpaths using a machine learning-based model involves a systematic approach, where each toolpath is constructed through a series of steps.
[0205] Figure 6 An example execution flow for such a systematic approach is illustrated. In some implementations, each step may include: at 602, a tool call generated by a machine learning model. For example, in some implementations of the example method, generating the multiple tool paths using a machine learning model includes: for each corresponding tool path, and for each corresponding step of the corresponding tool path, generating a tool call using the machine learning model. The machine learning model can be configured to analyze the task request, determine the operations to perform the task, and generate a sequence of tool calls that, when executed, will collectively achieve the desired result. Each tool call may include specific commands, parameters, and options tailored to the capabilities of the corresponding tool and the requirements of the task. For example, in a file processing task, the machine learning model may generate tool calls to enable file manipulation utilities, where each tool call specifies an operation (e.g., copy, move, delete) and necessary file paths and options (e.g., overwrite flags, recursive operations).
[0206] In some implementations of the example method, generating the multiple toolpaths using a machine learning-based model includes: for each corresponding toolpath, and for each corresponding step of the corresponding toolpath, enabling a tool call to initiate the execution of the corresponding tool in the tool execution environment (e.g., executing the tool call at 604). In some implementations of example method 500, enabling a tool call to initiate the execution of the corresponding tool in the tool execution environment may include: resolving the tool call generated by the machine learning-based model and executing the tool call within an isolated or dedicated environment where the tool is installed and configured to run. The tool execution environment may be a virtual machine, a container, a sandboxed execution context, or any secure and controlled runtime environment that supports the execution of the tool without interference from other processes or toolpaths. The enabling of the tool call may be managed by a tool control system 104, which can handle the setup, initiation, monitoring, and teardown of the tool execution environment. For example, the tool control system may ensure that necessary dependencies are in place, allocate resources for execution, and handle any errors or exceptions that occur during tool operation.
[0207] In some implementations of the example method, generating the multiple tool paths using a machine learning-based model includes, for each corresponding tool path and for each corresponding step of that tool path, obtaining a result from the tool (e.g., at 606). In some implementations of example method 500, obtaining a result from the tool may involve capturing the output, response, or any data generated by the tool as it completes its execution. The result can be direct output, such as a processed file, calculated value, or status message. The result can also be indirect output, such as a log file, error report, or performance metric. The tool control system can obtain this result from the tool execution environment and pass it back to the machine learning-based model for further processing. For example, the result could be a transformed file that is then used as input for the next tool call in the sequence, or it could be an error message that triggers a retry or alternative approach.
[0208] In some implementations of the example method, generating the multiple toolpaths using a machine learning model includes: for each corresponding toolpath, and for each corresponding step of the corresponding toolpath, evaluating the results (e.g., at 608). In some implementations of example method 500, evaluating the results obtained from the tools may include assessing the quality, relevance, or success of the tool's performance in the task context. The evaluation can be performed by the machine learning model (or different machine learning models), which can use the obtained results to determine the next steps in the toolpath or adjust the strategy used to complete the task. For example, the machine learning model can analyze the results to verify that the results meet task requirements, conform to the expected format or standard, or align with predicted results. If the results are satisfactory, the machine learning model can continue generating the next tool call in the sequence; otherwise, the machine learning model can take corrective actions, such as modifying parameters, selecting a different tool, or completely modifying the approach. The evaluation process can utilize various techniques, such as comparing with true values, applying heuristic rules, or using consistency metrics that measure consistency across multiple toolpaths or outputs.
[0209] Figure 6 The processing loops shown can be executed sequentially, in parallel, or in combination thereof. For example, the generation of each toolpath can be performed in parallel. Alternatively, toolpaths can be independent, thus allowing them to be executed independently during parallel processing. In one example, each step of a given toolpath can be generated sequentially.
[0210] Figure 7 An example execution flow for generating tool calls is shown. In some implementations of the example method, generating tool calls using a machine learning-based model includes, at 702, receiving request data describing at least one aspect of the request.
[0211] In some implementations of the example methods, generating tool calls using a machine learning-based model includes obtaining an index of available tools at 704. In some implementations of example method 500, obtaining the index of available tools may involve accessing a repository, database, or list that enumerates tools accessible to the system and usable in task execution. The index can provide comprehensive information about each tool, including its capabilities, usage syntax, available options or flags, versioning, and compatibility with the current system or environment. This index can be used as a reference for the machine learning-based model to identify which tools are suitable for the requested task and generate syntactically correct and functionally appropriate tool calls.
[0212] In some implementations of the example method, the call to the machine learning-based model generation tool includes, at 706, providing the requested data and index to the machine learning-based model. The requested data and index may, for example, be included in a text prompt sent to the machine learning-based model. This prompt may be structured (e.g., in JSON format) or unstructured.
[0213] In some implementations of the example method, using a machine learning-based model to generate a tool call involves receiving, at 708, output generated by the machine learning-based model based on the requested data and index, where the output describes the tool call. In one example, the output can be in the form of a text command, a structured object, or any other format that conveys the model's recommendation for the tool call. For example, the output could be a string specifying command-line instructions with particular flags and arguments, or it could be a JSON object mapped to an API call with defined parameters.
[0214] In some implementations of the example method, generating tool calls using a machine learning-based model includes, at 710, parsing the output to obtain the tool call. In some implementations of example method 500, parsing the output to obtain the tool call may involve extracting the executable portion of the model output and converting it into a format that can be directly executed or enabled by the tool execution system. This parsing step can handle variations in output format, resolve ambiguities, and ensure that the tool call conforms to the expected syntax and semantics of the tool being used. Parsing can be performed by processing logic that cleans, formats, and validates the tool call before it is sent for execution.
[0215] In some implementations of the example method, the machine learning model generates tool calls for each step of the operation sequence in a round-by-round manner. In some implementations of the example method, a first-round subset is used to enable the tool, and a second-round subset is used to obtain information. In some implementations of the example method, the first-round subset is resampled to obtain multiple different tool calls, and the second-round subset can be decoded using greedy sampling.
[0216] In some implementations, example method 500 includes determining a consistency metric by comparing the alignment of output states. In some implementations of example method 500, determining a consistency metric by comparing the alignment of output states can involve a quantitative or qualitative assessment of how closely results from different tool paths correspond to each other. This comparison can focus on identifying commonalities and differences across output states that can indicate the reliability and accuracy of the output relative to the task. For example, techniques such as byte-level differences or semantic similarity measures can be used to compute the consistency metric, where byte-level differences compare the binary or textual content of the output, and semantic similarity measures evaluate the meaning or intent behind the output. Vector-based similarity measures, such as cosine similarity, can also be used to compare the vector representations of output states within the embedding space. Alignment can be assessed at different granularities, from the complete output to specific features or attributes within the output. For example, if the task involves generating text summaries, the consistency metric can compare the overlap of key phrases or concepts between summaries generated by different tool paths.
[0217] In some implementations of Example Method 500, determining a consistency metric by comparing the alignment of output states may include using a voting mechanism in which outputs from multiple toolpaths are aggregated to determine a majority or relative majority consensus on certain features or results. For example, if several toolpaths generate outputs with different file attributes after a file conversion task, the consistency metric could be based on how much consensus is reached on attributes such as file size, resolution, or format in the outputs. Alignment can be determined by counting the frequency of each attribute value across the outputs and identifying the value with the highest count as the vote winner. The voting mechanism can be weighted based on a confidence score or quality metric associated with each output, giving greater influence to outputs considered more credible or accurate.
[0218] In some implementations, example method 500 includes determining the consistency metric using a machine learning-based model to evaluate the quality of each output state. In some implementations of example method 500, determining the consistency metric by comparing the alignment of output states may involve using a machine learning-based model specifically trained to evaluate the quality and consistency of outputs. For example, a separate evaluation model may be used to process the output states and determine the degree of quality or alignment based on learned patterns or features indicating successful task completion. The evaluation model may be trained on a dataset of labeled outputs where correct and consistent responses are known, allowing the evaluation model to develop an understanding of what constitutes quality or alignment in the context of various tasks. The evaluation model may output a quality score or consistency score for each output state, or a combined score for a set of output states, which can then be used to determine the most aligned and consistent (or highest quality) output to be selected as the response.
[0219] In some implementations of Example Method 500, determining a consistency metric by comparing the alignment of output states can include applying domain-specific rules or heuristics designed to capture subtle differences in a particular task or data type. For example, in tasks involving structured data such as JSON or XML, the comparison could include tree- or graph-based algorithms that assess both structural and content-based consistency of the output states, allowing for variations in the order or format of the semantic content. In tasks involving visual output, image comparison techniques such as feature embedding and comparison or perceptual hashing can be used to evaluate the visual alignment of the outputs, even if these outputs have been altered or compressed, or differ in other ways that do not affect the semantic content.
[0220] In some implementations of example method 500, the machine learning model used to evaluate the quality of each output state is the same as the machine learning model used to generate the plurality of tool paths. In other implementations of example method 500, the machine learning model used to evaluate the quality of each output state is different from the machine learning model used to generate the plurality of tool paths.
[0221] In some implementations of example method 500, a machine learning-based model for evaluating the quality of each output state is trained to process the multiple output states together to output a preferred output state, wherein the response is selected using the preferred output state.
[0222] In some implementations of Example Method 500, the consistency measure is based on the similarity of learned embeddings generated using the multiple output states. This can involve transforming the output states into a high-dimensional space, where semantically similar outputs are positioned as closer to each other. For example, machine learning models, such as neural networks or embedding models, can be used to generate vector representations—learned embeddings—for each output state. These embeddings can capture the underlying patterns, features, or semantic content of the outputs, allowing comparisons beyond surface-level textual or binary similarity. The consistency measure can then be computed by evaluating the distance or similarity between these embeddings, where closer embeddings indicate a higher degree of consistency. For example, cosine similarity or Euclidean distance metrics can be used to quantify the similarity between embeddings, where a threshold is set to determine when the outputs are considered consistent.
[0223] In some implementations of Example Method 500, the generation of the learned embeddings for the consistency metric can be performed by the same machine learning model that generates the multiple toolpaths, or by separate, dedicated embedding models. When using the same model, the embeddings can benefit from the model's comprehensive understanding of the task and the data, resulting in highly context-specific embeddings. Alternatively, separate embedding models can be employed to provide independent evaluation of the outputs, which may be useful for cross-validation or when an external perspective is required. The choice between the same model or separate models can depend on factors such as the complexity of the task, the availability of suitable pre-trained models, and the desired level of independence during the evaluation process.
[0224] In some implementations of Example Method 500, the consistency measure is determined by processing the output state using a machine learning model trained to generate outputs that indicate significant differences between inputs. This machine learning model (which may be referred to as a difference estimator model) can be specifically trained to identify and highlight differences deemed important or relevant to the current task. The training process may involve exposing the model to paired or grouped input data along with annotations or labels indicating key differences between these inputs. Through this training, the model learns to generate outputs that effectively capture and represent significant differences, such as variations in data content, structural divergences, or semantic implications. For example, the model can be trained on a dataset of text pairs with annotations highlighting differences in sentiment, topic, or factual content, enabling it to generate outputs that clearly define these differences in new text datasets.
[0225] In some implementations of Example Method 500, the machine learning model used to determine the consistency metric operates by generating a saliency map or difference score for each pair or group of output states. This saliency map can visualize or quantify the differences between outputs, providing a clear indication of regions where outputs are aligned or deviated. The difference score can be a numerical value summarizing the degree of difference, where a higher score indicates a greater difference. For example, in tasks involving image processing, a difference estimator model can generate a saliency map that highlights image regions from different tool paths that show significant variations, such as differences in object detection or segmentation boundaries. These saliency maps and difference scores can then be used to compute the consistency metric, where the system favors outputs exhibiting lower difference scores and therefore higher consistency.
[0226] In some implementations of Example Method 500, the difference evaluator model can be integrated into the tool control system to provide real-time feedback on the quality of the output status generated by the tool path. As the tool path produces outputs, the difference evaluator model can process these outputs to identify significant differences and generate corresponding saliency plots or difference scores. This feedback can inform decisions about pruning suboptimal tool paths, refining tool calls, or selecting the most consistent output for the final response.
[0227] In some implementations of Example Method 500, a difference evaluator model can be employed as part of a larger set of evaluation techniques for determining consistency metrics. Besides difference evaluation, other methods such as semantic similarity assessment, structural comparison, or heuristic rule application can be used to evaluate the output state. Consistency metrics can be determined by combining signals from these different evaluation techniques.
[0228] In some implementations of Example Method 500, the consistency metric includes one or more heuristic criteria and one or more probabilistic criteria, which are determined using the output of a machine learning model. In some examples, the heuristic criteria may include deterministic rules or patterns designed based on domain knowledge or empirical observations. These criteria can serve as an interpretable mechanism that consistently identifies output states that meet basic requirements or flags output states that exhibit obvious signs of error or anomalousness. For example, heuristic criteria may include: checks on specific file attributes in a file conversion task, validation of JSON patterns in a data transformation task, or verification of conformity to a predefined output format. Heuristic criteria can be particularly effective in identifying serious inconsistencies or deviations from expected results, thus providing a direct and computationally efficient means of assessing output state.
[0229] One or more probabilistic criteria can be generated by a machine learning model. For example, the model can use probabilistic methods, such as likelihood estimation, Bayesian inference, or probabilistic clustering, to evaluate the output state. These probabilistic criteria can capture nuances that heuristic rules might overlook, such as semantic coherence, contextual relevance, or complex patterns emerging from the data. By combining both heuristic and probabilistic criteria, consistency measures can leverage the strengths of both approaches.
[0230] In some implementations of Example Method 500, using machine learning-engineered models to determine probabilistic criteria for consistency measures can involve a variety of machine learning architectures and techniques suited to the specific task and data involved. For example, neural networks, support vector machines, decision trees, or ensemble methods can be employed to generate probabilistic criteria. The model can be pre-trained on large datasets to develop a general understanding of task-relevant features and then fine-tuned on task-specific data to refine its predictions for consistency assessment. Probabilistic criteria can be represented as confidence scores, likelihood ratios, or other statistical measures quantifying the degree of consistency across output states. These probabilistic assessments can complement heuristic criteria by providing a data-driven, adaptive, and potentially self-improving mechanism for evaluating the quality and consistency of the output as part of the consistency measure.
[0231] In some implementations, example method 500 includes selecting a response based on a consistency metric by identifying one or more features shared by a set of output states among the plurality of output states. In some implementations of example method 500, selecting a response based on a consistency metric includes identifying one or more features shared by a set of output states among the plurality of output states. This process may involve analyzing results obtained from the execution of a toolpath to detect common elements or patterns that recur across multiple outputs.
[0232] In some implementations, example method 500 includes selecting a response based on a consistency metric by selecting responses from the group. In some implementations of example method 500, selecting a response from the group based on identified shared features involves selecting one or more output states that best represent the commonalities found in the group's output states. This selection process can prioritize outputs that most prominently or accurately represent the shared features, depending on task requirements and the expected quality of the response. For example, among several output states that all include specific data visualizations, the selection may favor the output state whose visualization is most clearly presented and most closely matches the expected format. The selection may also consider additional criteria, such as the overall completeness of the output, confidence scores associated with shared features, or alignment with user preferences. The selected one or more output states from the group can then be compiled, synthesized, or simply forwarded as the final response to the task.
[0233] In some implementations of Example Method 500, one or more features include at least one latent feature generated using a machine learning model. In some implementations of Example Method 500, the one or more features include at least one latent feature generated using a machine learning model. Example latent features can be attributes, characteristics, or representations that are not explicitly present in the original data but can be inferred or derived from the original data through the processing power of the machine learning model. For example, in the context of image processing, latent features can represent abstract concepts such as the style or mood of an image, which the machine learning model can capture by analyzing patterns and relationships within pixel data that are not immediately discernible to a human observer. Similarly, in natural language processing tasks, latent features can represent the underlying sentiment or thematic content of text, which the machine learning model can identify by interpreting subtle cues and context conveyed by word choice and sentence structure.
[0234] In some implementations, example method 500 includes selecting a response based on a consistency metric by clustering representations of one or more features of the plurality of output states. This clustering process can be performed using a machine learning-based model or clustering algorithm to analyze features extracted from the output states and group these features into clusters based on similarity or correlation. Each cluster represents a set of output states that share common features, indicating that these output states may be aligned in how they solve the task. For example, in a data classification task, output states that assign similar class labels to the same data points can be grouped into the same cluster. Clustering can be based on various algorithms, such as k-means, hierarchical clustering, or density-based clustering, and can utilize different distance metrics, such as Euclidean distance or Jaccard similarity, to determine the proximity of feature representations.
[0235] In some implementations, example method 500 includes selecting a response based on a consistency metric by identifying preferred clusters. Preferred clusters can be clusters considered most likely to contain output states that best meet the task requirements. Identification can be based on criteria such as cluster size, cluster centrality in the feature space, cluster density, or quality scores associated with output states within the cluster. For example, larger clusters may indicate a higher level of consistency among outputs, while clusters with high density may indicate strong consensus on certain features. Preferred clusters can also be identified based on external validation or additional consistency checks confirming the reliability of the outputs contained in the preferred cluster. For example, a cluster may be selected as a preferred cluster for response selection if it contains all outputs that pass a set of heuristic validation checks.
[0236] In some implementations, example method 500 includes selecting a response based on a consistency metric by selecting a response from a cluster. In some implementations of example method 500, selecting a response from a cluster involves selecting an output state from a preferred cluster to use as the final response to the task. The system can select the most central or representative output state from the preferred cluster. In some implementations of example method 500, selecting a response from a cluster includes determining the centroid of the cluster and selecting a response based on that centroid.
[0237] The system can aggregate information from multiple output states within a cluster to synthesize a composite response. For example, in a task where clusters represent different textual descriptions of the same event, the system can select the most comprehensive and clearly expressed description as the response. Alternatively, the system can combine elements from several descriptions to create a more detailed and information-rich response.
[0238] In some implementations of Example Method 500, selecting a response from a cluster involves scoring the members of the cluster and selecting a response based on those scores. This process may involve assigning a numerical or categorical score to each output state within a preferred cluster based on one or more evaluation criteria. Criteria may include factors such as alignment with task requirements, the presence of desired features or attributes, the quality of the output as determined by a machine learning model or heuristic evaluation, and the confidence level associated with each output state. The scoring process may be performed by a machine learning model that generated the output states, by a separately evaluated model, or by a scoring system integrating multiple evaluation methods. For example, each output state may receive a score reflecting its semantic accuracy in a text summarization task or its precision in a data classification task. The output state with the highest score within the cluster can then be selected as the response to be returned.
[0239] In some implementations of Example Method 500, the rating of in-cluster members can be based on a weighted scoring system that considers various aspects of the output state. Each aspect can be assigned a different weight based on its importance or relevance to the task, allowing the system to prioritize certain features over others when calculating the overall score for each output state. The weighted scores can then be aggregated to produce a composite score for each output state, which can be used to rank the output states within the preferred cluster. The output state with the highest composite score—considering the weighted contributions of different aspects—can be selected as the response.
[0240] In some implementations of example method 500, the corresponding tool path includes one or more nodes, wherein the corresponding output state of the corresponding tool path is based on the completion node of the corresponding tool path. A node may, for example, correspond to tool execution (e.g., execution 304). A completion node may, for example, be a terminal node in the tool path. A node may be a completion node if the tool response is considered, when evaluated, to provide a response to request 102 (or provide data sufficient to facilitate a response to request 102).
[0241] In some implementations of example method 500, the first node of the one or more nodes is generated by using a machine learning model to generate a first tool call for performing a first step of the task. In some implementations of example method 500, the first node of the one or more nodes is generated by initiating the execution of the first tool call to obtain a first tool response. In some implementations of example method 500, initiating the execution of the first tool call to obtain a first tool response involves passing the generated tool call to a tool execution system or environment in which the corresponding tool is available.
[0242] In some implementations of Example Method 500, initiating the execution of the first tool call involves passing the first tool call to the corresponding tool in the execution environment associated with the corresponding tool. This execution environment can be a dedicated space where the tool is configured to operate, such as a virtual machine, container, cloud-based environment, or a specific runtime context within an operating system. The environment can be designed to provide the computational resources, libraries, and access permissions required for the tool to function effectively. For example, if the first tool call is a command to process a dataset using a statistical analysis tool, the execution environment could include the data science libraries and sufficient memory allocation required to process that dataset. The tool control system can facilitate the passing of the first tool call to the tool execution environment, ensuring that the tool call is properly formatted and that any required context or parameters are included. The execution environment can be isolated from other processes or tool paths to prevent interference and maintain execution integrity.
[0243] In some implementations of example method 500, the execution environment associated with the corresponding tool can be dynamically instantiated in response to a first tool call. For example, upon receiving a first tool call, the tool control system can trigger the creation of a new execution environment tailored to the requirements of that tool call. This could involve provisioning a new virtual machine or container, setting appropriate runtime settings, and deploying the tool within the environment. Dynamic instantiation of the execution environment allows for flexible and scalable execution of tool calls, as each tool call can be processed in an environment optimized for its specific requirements. Furthermore, this approach can support the parallel execution of multiple tool calls, each within its own environment, enabling concurrent processing of different steps or tasks. The dynamic environment can be managed by an orchestration layer that allocates resources, monitors the health of the environment, and handles cleanup operations when tool execution is complete.
[0244] In some implementations of Example Method 500, the machine learning model used to generate the tool path is configured to control operations within the execution environment, such as GUI interaction, command-line control, write access, and read access. The model can be integrated with the execution environment via API, command-line interface, or other interactive protocols that allow the model to exert control over tool operations.
[0245] In some implementations of example method 500, the execution environment refers to the virtual machine assigned to the task instance, which includes a machine learning-based model for generating toolpaths.
[0246] In some implementations of Example Method 500, the execution environment prevents the machine learning model used to generate tool paths from interfering with another task instance. In some implementations of Example Method 500, the execution environment prevents a corresponding tool from interfering with another tool associated with a different tool path. This isolation can be achieved through various mechanisms, such as running each task instance in a separate virtual machine, container, or process isolated at the operating system level. For example, the execution environment can leverage virtualization techniques that provide each task instance with its own set of virtualized hardware resources to ensure that the operation of one task instance does not affect the performance or stability of another task instance. Similarly, containerization can encapsulate each task instance within its own environment, including the necessary runtime, libraries, and configuration, while maintaining strict boundaries between different instances.
[0247] In some implementations of example method 500, each tool path is executed in a separate execution environment. In some implementations of example method 500, the separate execution environment is instantiated by a virtual machine. In some implementations of example method 500, the separate execution environment is instantiated by a separate working directory within a directory tree.
[0248] In some implementations, example method 500 includes caching resources for use across multiple tool paths. In some implementations, example method 500 includes caching resources for use across multiple execution environments.
[0249] In some implementations of Example Method 500, caching resources for use across multiple toolpaths involves storing frequently used data or computational routines in a cache memory or storage device to optimize the execution efficiency of the toolpaths. Cached resources may include libraries, datasets, intermediate computation results, configuration files, or any other assets frequently accessed or required by the tools associated with the toolpath. By caching these resources, the system can reduce redundant computations, minimize disk I / O operations, and reduce tool execution startup time. For example, if multiple toolpaths utilize the same data analytics library, the library can be loaded into the cache once and then shared among the toolpaths, eliminating the need to repeatedly load the library from disk or fetch it from an external source for each toolpath. In some implementations of Example Method 500, the execution environment can benefit from cached resources by avoiding redundant operations such as reloading libraries from disk, re-fetching data from a remote server, or recompiling already computed results. For example, if multiple toolpaths need to access a specific machine learning library, that library can be cached so that subsequent toolpaths can access the library from the cache without the overhead of loading the library from an external source. This can significantly reduce the startup time of each tool and improve the overall performance of the system by minimizing unnecessary I / O operations and computations.
[0250] In some implementations of example method 500, caching of resources for multiple tool paths can be managed by a resource management subsystem within the execution environment (e.g., tool execution system 108). This subsystem can monitor tool usage patterns, identify resources common across tool paths, and implement a caching strategy that balances memory usage with performance gains. The caching mechanism can be dynamic, where the subsystem adds resources to the cache when it detects high demand for resources and evicts resources that become less frequently used. Depending on the scale and requirements of the task execution framework, caching can be implemented using various technologies, such as in-memory data repositories, distributed caching systems, or dedicated caching hardware.
[0251] In some implementations of example method 500, resources cached for use across multiple tool paths can be accessed through a shared layer available to all execution environments associated with the tool path (e.g., tool execution system 108). This shared layer can serve as a centralized repository for the cached resources, providing a consistent and efficient access point for the tools. The shared layer can implement access control and synchronization mechanisms to ensure that the cached resources are used correctly and that updates to the resources are properly propagated across all tool paths.
[0252] In some implementations of Example Method 500, the caching strategy for resources used across multiple toolpaths can be instructed by predictive models or machine learning techniques that anticipate future resource needs based on historical usage patterns or task characteristics. For example, a machine learning-based model can analyze the execution history of a toolpath, identify patterns in resource usage, and predict which resources might be needed in the near future. The system can then proactively cache these predicted resources, ensuring that resources are readily available when needed by the toolpath. This predictive caching can reduce latency and improve the responsiveness of task execution because the necessary resources are already cached when the toolpath requires them.
[0253] In some implementations of Example Method 500, resource caching can be implemented using various caching mechanisms and storage solutions, such as in-memory databases, file-based caches, or distributed caching systems. For example, in-memory databases can provide fast access to cached datasets, while file-based caches are suitable for storing large files or assets that are not frequently updated. Distributed caching systems can provide scalability and resilience, enabling caching to be maintained across multiple machines or nodes in a cluster. The choice of caching mechanism can depend on factors such as the size and type of the resources to be cached, the expected access patterns, and the scalability requirements of the execution environment.
[0254] In some implementations of Example Method 500, cache management can involve complex strategies and algorithms for determining which resources to cache, when to cache them, and how long they should be retained in the cache. For example, a Least Recently Used (LRU) strategy could be employed to evict resources that haven't been accessed within a certain timeframe, freeing up space for more frequently used resources. Alternatively, predictive caching algorithms can analyze usage patterns and task scheduling to pre-cache resources that may be needed in the near future. These caching strategies and algorithms can be tuned and optimized based on real-time monitoring of the execution environment and performance metrics of toolpaths.
[0255] In some implementations of Example Method 500, resource caching may also include mechanisms to ensure the consistency and integrity of cached data across multiple execution environments. This could involve version control systems that track different versions of resources, synchronization protocols that update the cache in response to changes, and validation checks that verify the correctness of cached resources before they are used by tools. For example, if a cached dataset is updated by one execution environment, the cache can be synchronized to ensure that all other environments accessing the dataset have the latest version. These consistency mechanisms prevent problems such as data staleness, cache inconsistencies, or conflicts caused by concurrent updates, thereby ensuring that cached resources remain reliable and up-to-date across all execution environments.
[0256] In some implementations of example method 500, the second node among the one or more nodes is generated by using a machine learning-based model and generating a second tool call for performing a second step of the task based on the first tool response. In some implementations of example method 500, the second node among the one or more nodes is generated by initiating the execution of the second tool call to obtain a second tool response. For example, if the first tool response indicates a successful file transfer, the second tool call could be for a tool that performs a file format conversion on the transferred file. The machine learning-based model generates the second tool call by analyzing the first tool response, considering the overall task requirements, and selecting the next logical operation to bring the task closer to completion.
[0257] In some implementations of example method 500, the multiple tool paths include a tool path tree. In some implementations of example method 500, the tool path tree can represent a structured hierarchy or network of execution sequences, each branching from a common starting point or root based on different decision points, tool choices, or parameter configurations. The tree structure allows for the parallel exploration of multiple solution paths, each branch representing a different approach to completing the task. The branches of the tree can diverge based on the results of tool execution, resource availability, or task requirements.
[0258] In some implementations of example method 500, the second corresponding toolpath includes one or more nodes, wherein the corresponding output state of the second corresponding toolpath is based on the completion node of the second corresponding toolpath. In some implementations of example method 500, the first node of the one or more nodes is generated by using a machine learning model to generate a first tool call for performing a first step of the task. In some implementations of example method 500, the first node of the one or more nodes is generated by initiating the execution of the first tool call to obtain a first tool response.
[0259] In some implementations, example method 500 includes evaluating multiple intermediate states for each of the plurality of toolpaths, wherein a corresponding intermediate state for a given toolpath is based on an intermediate node of that toolpath prior to a completion node. In some implementations of example method 500, evaluating multiple intermediate states for each of the plurality of toolpaths involves assessing the progress and outcome of each step leading to the completion node of each toolpath. Intermediate states may correspond to the results or states of operations performed at intermediate nodes along a toolpath, such as steps occurring after the initial node but before the final completion node. Evaluation may include checking whether the intermediate state aligns with expected progress toward a task objective, verifying that the intermediate state meets intermediate criteria or milestones, and determining whether the intermediate state provides a suitable basis for subsequent steps. For example, an intermediate state may be the output of a data preprocessing tool in a data analysis task, and evaluation may involve confirming that preprocessing was successful and that the data is now ready for the next analytical step. This evaluation may be performed by a machine learning-enabled model that can use the intermediate states to guide the generation of future tool calls and effectively navigate the toolpath toward the completion node.
[0260] In some implementations, example method 500 includes pruning at least one of a plurality of toolpaths based on corresponding intermediate states. In some examples, pruning involves aborting or discarding toolpaths determined to be suboptimal, unproductive, or potentially unsuccessful based on the evaluation of intermediate states. This action can be taken when intermediate states indicate a deviation from the desired path, such as encountering errors, failing to meet quality thresholds, or producing results inconsistent with task requirements. For example, if intermediate states reveal that data transformation steps have resulted in corrupted data, the system can prune the toolpath to prevent further processing of the defective data and conserve resources. Pruning decisions can be based on heuristic rules, quality scores, or predictive models that assess the likelihood of each toolpath successfully completing the task. By selectively pruning toolpaths, the system can focus its efforts on the most promising paths to complete the task, optimizing resource utilization and increasing the chances of achieving the desired results.
[0261] In some implementations, example method 500 includes pruning the at least one tool path based on determining its redundancy. In some implementations of example method 500, pruning the at least one tool path based on determining its redundancy involves identifying and eliminating duplicate tool paths within the task execution context. Redundancy can refer to situations where multiple tool paths effectively employ the same approach, generate similar outputs, or otherwise contribute minimal added value to the task completion process. For example, if two tool paths are executing the same sequence of tool calls with little or no variation affecting the final result, one of these paths can be considered redundant and pruned to simplify the execution process. Redundancy can be determined based on a comparison of intermediate states or outputs of the tool paths, looking for a high degree of similarity or overlap indicating that these paths are actually equivalent.
[0262] In some implementations of example method 500, redundant toolpaths can be pruned dynamically in real time as the task progresses. The system can continuously monitor for signs of toolpath redundancy as new tool calls are generated and new intermediate states are evaluated.
[0263] In some implementations of example method 500, redundancy is determined based on the deterministic result of identifying the at least one tool path. In some implementations of example method 500, determining redundancy based on the deterministic result of identifying the at least one tool path involves identifying when a particular tool path is expected to produce the same result. The system can detect such deterministic behavior by comparing the input states of the same tool across multiple executions. In some implementations of example method 500, redundancy is determined based on similarity margin.
[0264] In some implementations of example method 500, the similarity margin is based on a minimum step size associated with the range of parameter values for the tool call. For example, if the tool call involves adjusting the brightness of an image, the minimum step size could be set to the smallest brightness change that produces a perceptible difference in the appearance of the image. The range of parameter values could cover the spectrum of permissible values for a given parameter, where the similarity margin is defined as the span of values that, due to the minimum step size, makes the results considered practically equivalent. For example, if the range of brightness values is from 0 to 100 and the minimum step size is 5, the similarity margin could be established such that brightness values within 5 units of each other are considered redundant.
[0265] In some implementations, example method 500 includes adjusting the at least one toolpath based on determining failure conditions associated with it. In some implementations of example method 500, adjusting the at least one toolpath based on determining failure conditions associated with it involves identifying problems or errors that occur during toolpath execution and taking corrective actions to resolve these problems. Failure conditions can be any event or outcome that deviates from the expected or desired execution flow, such as error messages returned by the tool, processing exceptions, or outputs that do not meet predefined quality criteria. The system can detect these failure conditions using monitoring mechanisms, error detection algorithms, or feedback from a machine learning-based model (e.g., validator 400) that generated the toolpath. Upon detecting a failure condition, the system can adjust the affected toolpath by modifying tool calls, changing execution parameters, rerouting the path to an alternative tool, or restarting the path from a previously successful state.
[0266] In some implementations of Example Method 500, failure conditions are determined based on received tool responses that deviate from expected criteria. In some implementations of Example Method 500, deviations can manifest in various ways, such as output that does not match the expected format, results falling outside the acceptable quality range, or data failing to meet predefined accuracy or integrity thresholds. Expected criteria can be specified as part of task requirements or derived from domain knowledge or prior experience with similar tasks. In some implementations of Example Method 500, determining failure conditions based on deviations in tool responses can involve comparing the received tool response to a set of expected or benchmark results. The comparison can be performed automatically by the tool control system, which can use validation algorithms, pattern matching techniques, or machine learning-based models to evaluate the tool response. For example, the system can use checksum comparisons to verify the integrity of document output against expected checksum values; if the values do not match—indicating a deviation—the system can mark this as a failure condition. Alternatively, the system can employ a natural language processing model to evaluate the semantic content of the text output; if the model detects significant deviations from the task's semantic criteria, such as missing concepts or incorrect information, this can be identified as a failure condition.
[0267] In some implementations of example method 500, determining failure conditions based on deviation-based tool responses (e.g., tool execution system 108) may also include using heuristic rules or expert systems encoded with domain-specific knowledge about the expected characteristics of the task and successful output. These rules or systems can provide a knowledge-based framework for evaluating tool responses, thereby identifying deviations based on established principles or best practices. For example, in a software compilation task, heuristic rules could specify that successful output must pass a series of automated tests; if the tool response indicates that the compiled software fails these tests, the system can determine that this deviation constitutes a failure condition.
[0268] In some implementations of example method 500, the failure condition is determined based on receiving an error message indicating that a tool or tool call is inoperable. In some implementations of example method 500, adjusting the at least one tool path includes pruning the at least one tool path. In some implementations of example method 500, adjusting the at least one tool path includes restarting the at least one tool path using one or more newly generated tool calls from a machine learning model. In some implementations of example method 500, restarting the at least one tool path includes generating new tool calls based on the failure condition.
[0269] In some implementations of example method 500, generating a new tool call based on the failure condition involves feeding data describing the failure condition as context input to a machine learning-based model for a query used for the new tool call.
[0270] In some implementations of Example Method 500, generating new tool calls based on failure conditions involves feeding data describing the failure conditions as contextual input to a machine learning model for queries using the new tool calls. This process may include providing the machine learning model with detailed information about the nature of the failure, such as error codes, diagnostic messages, or descriptions of how the execution of the tool path deviated from the expected results. The failure condition data serves as contextual input to inform the model about the generation of new tool calls. By incorporating this context, the model can make more informed predictions about the most effective course of action to take to resolve the failure and advance the task. For example, if the failure condition data indicates that a file conversion tool failed due to an unsupported file format, the model can use this context to generate new tool calls specifying different conversion tools that are known to support the desired format. The query may explicitly instruct the model to generate tool calls that avoid the problems identified in the failure condition data, or suggest alternative strategies to circumvent the failure.
[0271] In some implementations, example method 500 includes: evaluating at least one intermediate state using a machine learning-based model. In some implementations, example method 500 includes: evaluating at least one intermediate state using a machine learning-based model different from the machine learning-based model used to generate the tool call.
[0272] In some implementations, example method 500 includes: evaluating at least one intermediate state using a machine learning-based model to select a heuristic tool to be applied to the at least one intermediate state. In some implementations of example method 500, evaluating at least one intermediate state using a machine learning-based model to select a heuristic tool to be applied to the at least one intermediate state involves the machine learning-based model analyzing the intermediate state to determine an appropriate heuristic tool that can solve a problem or enhance the quality of the intermediate state. The intermediate state can be the output or state update from a specific operation within a tool path, and the heuristic tool can be a specialized algorithm or routine designed to perform specific checks, transformations, or optimizations on the data. For example, if the intermediate state is a partially processed dataset, the heuristic tool selected by the model could be a data cleaning algorithm that removes outliers or normalizes values.
[0273] In some implementations, example method 500 includes evaluating multiple features of at least one intermediate state. In some implementations of example method 500, evaluating multiple features of at least one intermediate state involves analyzing various aspects or dimensions of data generated during toolpath execution prior to reaching a completion node. In some implementations of example method 500, the evaluation of multiple features of at least one intermediate state may include both quantitative and qualitative assessments.
[0274] In some implementations, example method 500 includes: calculating a consistency metric for the plurality of intermediate states. In some implementations, example method 500 includes: using a request to determine one or more screening criteria for evaluating the intermediate states. In some implementations, example method 500 includes: pruning the at least one toolpath based on failures relative to the one or more screening criteria.
[0275] In some implementations, example method 500 includes: evaluating an intermediate consistency metric for one or more tool calls sampled from a machine learning model. In some implementations, example method 500 includes: selecting a representative tool path based on the consistency metric. In some implementations, example method 500 includes: pruning one or more other tool paths.
[0276] In some implementations, example method 500 includes: calculating a consistency metric for the plurality of intermediate states to identify clusters of aligned toolpaths. In some implementations, example method 500 includes: selecting a representative toolpath based on the consistency metric, wherein the representative toolpath represents a cluster of aligned toolpaths. In some implementations, example method 500 includes: pruning other toolpaths within the clusters of aligned toolpaths.
[0277] In some implementations, example method 500 includes: calculating a consistency metric for the plurality of intermediate states to identify clusters of aligned toolpaths. In some implementations, example method 500 includes: selecting a representative toolpath based on the consistency metric, wherein the representative toolpath represents a cluster of aligned toolpaths. In some implementations, example method 500 includes: pruning other toolpaths within the clusters of aligned toolpaths. In some implementations, example method 500 includes: initiating a new toolpath by sampling new tool calls from a machine learning-based model.
[0278] In some implementations of Example Method 500, the tool invocation includes a text command configured for input via a command-line interface (CLI). This text command can be a text string representing an instruction or series of instructions that, when entered into the CLI, is executed by the operating system or a specific application to perform the desired operation. The format of the text command can follow the syntax and conventions expected by the CLI, including the use of command names, flags, arguments, and options. A machine learning model can generate this text command by predicting the appropriate command structure based on task requirements and the capabilities of the tool associated with the tool invocation.
[0279] In some implementations of Example Method 500, the tool call includes a request to an Application Programming Interface (API) endpoint. In some implementations of Example Method 500, the tool call includes a request to an API endpoint over a network. In some implementations of Example Method 500, the tool call includes a request to an API endpoint. A request to an API endpoint can be a structured command conforming to the API's interface specification, such as an HTTP request with a specific method (e.g., GET, POST, PUT), a Uniform Resource Identifier (URI) identifying the API resource, and additional parameters or data payloads that potentially provide the necessary input for API operations. API endpoints can be associated with services or applications that perform task-related specific functions, such as data storage services, computing engines, or third-party tools with specialized capabilities.
[0280] In some implementations of Example Method 500, the API request constituting the tool invocation may include various elements for enabling the API endpoint. These elements may include headers specifying metadata (such as content type or authentication token), query parameters fine-tuning the API operation, and a body payload containing input data or instructions for the API service. The API request may be formulated to match the expected request format of the API, thereby ensuring that the service can correctly interpret and process the request. For example, if the task involves querying a database via the API, the API request might include query parameters defining the search criteria and a body payload specifying the fields to be retrieved.
[0281] In some implementations, example method 500 includes: using a machine learning-based model to generate a tool that includes one or more lines of programming code to perform operations related to the requested task. In some implementations, example method 500 includes: executing the tool in at least one toolpath. The programming code can be written in various languages, such as Python, Java, C++, or any other language suitable for the task and execution environment.
[0282] In some implementations of example method 500, executing a tool in at least one toolpath involves running the generated programming code within a specified execution environment that supports the execution of the code. The execution of the tool can be initiated by a tool control system, which may compile, interpret, or directly execute the code depending on the language and the nature of the tool. For example, the tool can be a script interpreted by a script engine or a compiled binary file directly executed by an operating system.
[0283] In some implementations, example method 500 includes: computing a consistency metric for multiple tool calls. In some implementations, example method 500 includes: evaluating an intermediate consistency metric for one or more tool calls sampled from a machine learning-based model. The evaluation may include assessing the similarity between tool calls, the alignment of tool calls with task requirements, and the adherence of tool calls to known best practices or success patterns. For example, if the task involves data extraction from various sources, the intermediate consistency metric may measure the degree to which the sampled tool calls employ similar extraction methods or target similar data structures, indicating a unified approach to the task. The evaluation of the intermediate consistency metric may be performed by the machine learning-based model itself or by a separate evaluation system.
[0284] In some implementations, example method 500 includes selecting one or more tool calls for execution to improve the diversity of tool paths. In some implementations of example method 500, selecting one or more tool calls for execution to improve the diversity of tool paths involves strategically selecting tool calls that introduce variation and breadth to the range of paths explored to accomplish the task.
[0285] In some implementations of Example Method 500, improving the diversity of tool paths by selecting one or more tool calls can also involve balancing the exploration of new approaches with the utilization of known successful strategies. The system can employ multi-armed slot machine algorithms, reinforcement learning techniques, or other decision-making frameworks that weigh the trade-off between trying new tool calls and focusing on tool calls that previously yielded positive results. For example, the system can allocate a portion of its computational resources to experimental tool calls using cutting-edge tools or innovative methods, while dedicating the remaining resources to established tool calls with validated track records. This balancing approach can enhance the diversity of tool paths while maintaining a consistent level of performance.
[0286] In some implementations of Example Method 500, the selection of tool calls for improving diversity can be guided by a diversity metric that quantifies the variation between tool paths. The diversity metric can be calculated based on the heterogeneity of the tools, methods, or data involved in the tool calls, where higher values indicate greater diversity. For example, the metric could consider the range of different tools being used, the diversity of data processing techniques employed, or the distribution of parameter settings across tool calls.
[0287] In some implementations, example method 500 includes clustering tool calls sampled from a machine learning model. In some implementations, example method 500 includes selecting representative tool calls from each of the top K clusters for execution. In some implementations of example method 500, selecting representative tool calls from each of the top K clusters for execution involves identifying representative tool calls representing typical pathways within each cluster that represent cluster members. The top K clusters may be clusters with the highest relevance, size, density, or quality score, indicating that the top K clusters are the most promising or aligned with the task objective. The selection of representative tool calls may be based on criteria such as their proximity to the cluster centroid, their coverage of the cluster's feature space, or their performance in initial evaluations.
[0288] In some implementations, example method 500 includes determining one or more operations to be performed using a graphical user interface (GUI) based on tool calls. In some implementations of example method 500, determining one or more operations to be performed using a GUI based on tool calls involves interpreting tool calls to identify actions that can be performed within the GUI environment. These operations may include tasks such as opening a file, selecting a menu option, entering data into a form, or triggering other GUI-based processes. These operations can be determined by parsing the GUI's image rendering (e.g., using a multimodal machine learning model).
[0289] In some implementations, example method 500 includes generating one or more navigation commands configured to interact with a graphical user interface. In some implementations of example method 500, generating one or more navigation commands configured to interact with a graphical user interface may include creating a sequence of instructions that simulates user interaction with GUI elements. The navigation commands can automate GUI actions such as clicking buttons, entering text, navigating menus, or dragging items. The commands can be generated in a format compatible with GUI automation tools or frameworks that execute commands within a GUI context. For example, a machine learning model can generate navigation commands that simulate the click sequence required to derive charts from a data analysis GUI. The generation of navigation commands allows systems to perform tasks in an automated and efficient manner using GUI-based tools, thereby expanding the range of tools and operations that can be utilized in task execution.
[0290] In some implementations, example method 500 includes initiating the execution of one or more navigation commands. In some implementations of example method 500, initiating the execution of one or more navigation commands to interact with a graphical user interface can be managed by a tool control system 104 or a tool execution environment 108, which may deploy an automation engine or virtual input device to execute commands within the GUI environment. The initiation process may involve translating the navigation commands into low-level system events recognized by the GUI, such as mouse movements, keyboard keystrokes, or touch gestures. The execution of the navigation commands can be synchronized with the GUI's response. For example, the system may wait for a menu to open before initiating a command for selecting an item from the menu. Initiating navigation commands enables the system to utilize GUI-based applications or services as part of the tool path, thereby providing a means to perform operations requiring direct interaction with the GUI.
[0291] In one aspect, this disclosure provides an example method (e.g., a method as part of or different from example method 500). In some implementations, the example method includes: transmitting a request from a client computing system to a server computing system for performing a task.
[0292] In some implementations, this example method includes: the client computing system receiving a response from the server computing system based on the execution of a task.
[0293] In some implementations, the response is obtained using any example implementation of example method 500.
[0294] In some implementations of the example method, the server computing system uses any of the example implementations of example method 500 to perform tasks.
[0295] In some implementations, this example method includes: the client computing system receiving an initial request from the user computing system. In some implementations, this example method includes: obtaining a request for performing a task based on the initial request.
[0296] Figure 8 A flowchart is depicted for a method 800 for training one or more machine learning-based models according to various aspects of this disclosure. For example, an example machine learning-based model may include machine learning model 106.
[0297] One or more portions of the example method 800 may be implemented by a computing system including one or more computing devices, such as the computing system described with reference to the other figures. Each corresponding portion of the example method 800 may be executed by any computing device (or any combination thereof) of the one or more computing devices. Furthermore, one or more portions of the example method 800 may be implemented on hardware components of the apparatus described herein, for example, for training one or more systems or models. Figure 8 For illustrative and discussion purposes, elements executed in a specific order are depicted. Those skilled in the art will understand using the disclosure provided herein that elements of any method discussed herein can be adapted, rearranged, extended, omitted, combined, or modified in various ways without departing from the scope of this disclosure. Figure 8 The elements / terms described are for illustrative purposes only and with reference to other systems and figures, and are not intended to be limiting. One or more portions of the example method 800 may be performed additionally or alternatively by other systems.
[0298] At 802, example method 800 may include obtaining training instances. The training dataset may include multiple training instances partitioned across multiple datasets (e.g., training datasets, validation datasets, or test datasets). Training instances may be labeled or unlabeled. Although referred to as "training" instances in example method 800, it should be understood that runtime inference may form training instances when the model is trained using evaluations of the model's performance on that runtime instance (e.g., online training / learning). Example data types for training instances and various tasks associated with them are described throughout this disclosure.
[0299] At 804, example method 800 may include using one or more machine learning models to process training instances to generate output. This output may be obtained directly from the one or more machine learning models, or it may be a downstream result of a processing operation chain that includes the output of the one or more machine learning models.
[0300] At 806, example method 800 may include receiving an evaluation signal associated with the output. The evaluation signal can be obtained using a loss function. Various losses can be determined, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, contrastive loss, or various other loss functions. The evaluation signal can be computed using known ground truth labels (e.g., supervised learning), predicted or estimated labels (e.g., semi-supervised or self-supervised learning), or in the absence of labels (e.g., unsupervised learning). The evaluation signal can be a reward (e.g., for reinforcement learning). The reward can be computed using a machine learning-based reward model configured to generate a reward based on the received output. The reward can be computed using feedback data describing human feedback to the output.
[0301] At 808, example method 800 may include using an evaluation signal to update a machine learning-based model. For example, in some embodiments, various training or learning techniques (such as, for example, backpropagation) may be used to learn the values of the parameters of the machine learning-based model. For example, the evaluation signal may be backpropagated from the output (or another source of the evaluation signal) through the machine learning-based model to update one or more parameters of the model (e.g., based on the gradient of the evaluation signal relative to the parameter values). For example, a system containing one or more machine learning-based models may be trained in an end-to-end manner. Gradient descent techniques may be used to iteratively update the parameters over multiple training iterations. In some implementations, performing error backpropagation may include performing truncated backpropagation through time. Example method 800 may include implementing various generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the model being trained.
[0302] In some implementations, example method 800 can be implemented to train a machine learning model from an initial state to a fully trained state (e.g., when the model exhibits a desired performance profile, such as based on accuracy, precision, recall, etc.).
[0303] In some implementations, example method 800 can be implemented for specific stages of the training process. For example, in some implementations, example method 800 can be implemented for pre-training a machine learning model. Pre-training can include, for example, large-scale training on potentially noisy data to achieve a broad performance level base across multiple tasks / data types. In some implementations, example method 800 can be implemented for fine-tuning a machine learning model. Fine-tuning can include, for example, smaller-scale training based on higher-quality (e.g., labeled, curated, etc.) data. Fine-tuning can affect all or part of the parameters of the machine learning model. For example, parts of the machine learning model can be “frozen” for certain training stages. For example, parameters associated with the embedding space can be “frozen” during fine-tuning (e.g., to preserve information learned from a broader domain than that present in the fine-tuning dataset). Example fine-tuning methods include reinforcement learning. Reinforcement learning can be based on user feedback on model performance during use.
[0304] Example of a machine learning model
[0305] Figure 9 This is a block diagram of an example processing flow for using a machine learning-engineered model 1 to process input 2 to generate output 3.
[0306] The machine learning model 1 can be or includes one or more machine learning models or model components. Example machine learning models can include neural networks (e.g., deep neural networks). Example machine learning models can include non-linear or linear models. Example machine learning models can use other architectures to replace or supplement neural networks. Example machine learning models can include decision tree-based models, support vector machines, hidden Markov models, Bayesian networks, linear regression models, k-means clustering models, etc.
[0307] Example neural networks can include feedforward neural networks, recurrent neural networks (RNNs) (including long short-term memory (LSTM) based RNNs), convolutional neural networks (CNNs), diffusion models, generative adversarial networks, or other forms of neural networks. Example neural networks can also be deep neural networks. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some example machine learning models may include multi-head self-attention models.
[0308] The machine learning-based model 1 may include one or more instances of the same model configured to operate on data from input 2. The machine learning-based model 1 may also include an ensemble of different models capable of interacting collaboratively to process data from input 2. For example, the machine learning-based model 1 may employ a hybrid expert architecture. See, for example, Zhou et al. Mixture-of-Experts with Expert Choice Routing(Expert Hybrid of Expert-Selected Routing), arXiv:2202.09368v2 (October 14, 2022).
[0309] Input 2 can typically include or otherwise represent various types of data. Input 2 can include one type or many different types of data. Output 3 can be data of the same type or a different type compared to Input 2. Output 3 can include one type or many different types of data.
[0310] Example data types for input 2 or output 3 include natural language text data, software code data (e.g., source code, object code, machine code, or any other form of computer-readable instruction or programming language), machine code data (e.g., binary code, assembly code, or other forms of machine-readable instruction that can be executed directly by a computer's central processing unit), assembly code data (e.g., a low-level programming language that uses a symbolic representation of machine code instructions to program the processing unit), genetic data or other chemical or biochemical data, image data, audio data, audiovisual data, tactile data, biometric data, medical data, financial data, statistical data, geographic data, astronomical data, historical data, and generally sensor data (e.g., digital or analog values, such as voltage or other absolute or relative level measurements from real or artificial inputs, such as from audio sensors, light sensors, displacement sensors, etc.). Data can be raw or processed and can be in any format or mode.
[0311] In multimodal input 2 or output 3, example combinations of data types include image data and audio data, image data and natural language data, natural language data and software code data, image data and biometric data, sensor data and medical data, etc. It should be understood that any combination of data types in input 2 or output 3 can exist.
[0312] Example input 2 may include one or more data types, such as the example data types noted above. Example output 3 may include one or more data types, such as the example data types noted above. The data type of input 2 may be the same as or different from the data type of output 3. It should be understood that the example data types mentioned above are provided for illustrative purposes only. The data types contemplated within the scope of this disclosure are not limited to those examples noted above.
[0313] Example of a sequence processing model using machine learning
[0314] Figure 10This is a block diagram illustrating an example implementation of a machine learning-based model configured to process information sequences. For example, an example implementation of machine learning model 1 could include a machine learning sequence processing model 4. The example system could pass input 2 to sequence processing model 4. Sequence processing model 4 could include one or more machine learning components. Sequence processing model 4 could process the data from input 2 to obtain input sequence 5. Input sequence 5 could include one or more input elements 5-1, 5-2, ..., 5- obtained from input 2. M The sequence processing model 4 can use the prediction layer 6 to process the input sequence 5 to generate the output sequence 7. The output sequence 7 may include one or more output elements 7-1, 7-2, ..., 7- generated based on the input sequence 5. N The system can generate output 3 based on output sequence 7.
[0315] Sequence processing models 4 may include one or more machine learning-based model components configured to ingest, generate, or otherwise reason about sequences of information. For example, some example sequence processing models in the text domain are referred to as “large language models” or LLMs. See, for example, the PaLM 2 technology report, Google, https: / / ai.google / static / documents / palm2techreport.pdf (nd). Other example sequence processing models can operate in other domains, for example, such as the image domain; see, for example, Dosovitskiy et al. An image is 16x16 Words: Transformers for Image Recognition at Scale [An image is worth 16x16 words: Transformers for large-scale image recognition], arXiv:2010.11929v2 (June 3, 2021); in the audio domain, see, for example, Agostinelli et al. MusicLM: Generating Music From Text [MusicLM: Generating Music from Text], arXiv:2301.11325v1 (January 26, 2023); in the field of biochemistry, see, for example, Jumper et al., Highly accurate protein structure prediction with AlphaFold, 596 Nature 583 (August 26, 2021). Sequence processing model 4 can process one or more types of data simultaneously. Sequence processing model 4 can include relatively large models (e.g., more parameters, high computational cost), relatively small models (e.g., fewer parameters, computationally lightweight), or both.
[0316] Generally, sequence processing model 4 can obtain input sequence 5 using data from input 2. For example, input sequence 5 may include a representation of the data from input 2 in a format that sequence processing model 4 can understand. One or more machine learning components of sequence processing model 4 may ingest data from input 2, parse the data into fragments compatible with the processing architecture of sequence processing model 4 (e.g., via “word segmentation”), and project these fragments into the input space associated with prediction layer 6 (e.g., via “embedding”).
[0317] Sequence processing model 4 can ingest data from input 2 and parse that data into a sequence of elements to obtain input sequence 5. For example, a portion of the input data from input 2 can be decomposed into fragments that collectively represent that portion of the input data. These fragments can provide the elements of the sequence.
[0318] In some cases, elements 5-1, 5-2, ..., 5- M Elements can represent building blocks used to capture or express meaningful information within a specific data domain. For example, elements can describe "atomic units" across one or more domains. For instance, for a text input source, these elements could correspond to groups of one or more words or sub-word components, such as a set of one or more characters.
[0319] For example, elements 5-1, 5-2, ..., 5- M This can represent the tokens obtained using a tokenizer. For example, a tokenizer can process a given part of the input source and output a series of tokens (e.g., corresponding to input elements 5-1, 5-2, ..., 5-). M These tokens represent that part of the input source. Various tokenization methods can be used. For example, a text input source can be tokenized using byte-pair encoding (BPE) techniques. See, for example, Kudo et al. SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing [SentencePiece: A simple and language-independent word segmenter and desegmenter for neural text processing] Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (System Demonstrations), pp. 66–71 (October 31–November 4, 2018), https: / / aclanthology.org / D18-2012.pdf. Tokenization of image-based input sources can be performed by extracting and serializing blocks from images.
[0320] Generally speaking, any data type can be serialized and processed into the input sequence 5. It should be understood that... Figure 10The elements depicted in the text are 5-1, 5-2, ..., 5- M It can be a word element or its embedded representation.
[0321] Prediction layer 6 can predict one or more output elements 7-1, 7-2, ..., 7- based on the input elements. N Prediction layer 6 may include one or more machine learning-based model architectures, such as one or more layers of learned parameters that manipulate and transform the inputs to adapt the input elements 5-1, 5-2, ..., 5- M We can extract higher-order meanings and the relationships between them. In this way, for example, example prediction layer 6 can predict new output elements based on the context provided by input sequence 5.
[0322] Prediction layer 6 can evaluate the associations between portions of the input sequence 5 and specific output elements. These associations can inform predictions about the likelihood that a particular output follows the input context. For example, consider the text snippet “The carpenter's toolbox was small and heavy. It was full of ___.” Example prediction layer 6 can identify that “It” refers back to “toolbox” by determining the relationships between the corresponding embeddings. Example prediction layer 6 can also associate “it” with attributes of the toolbox, such as “small” and “heavy”. Based on these associations, for example, prediction layer 6 can assign a higher probability to the word “nails” than to the word “sawdust”.
[0323] The Transformer is an example architecture that can be used in prediction layer 4. For example, see Vaswani et al. Attention is all you need. arXiv:1706.03762v7 (August 2, 2023). The Transformer is an example of a machine learning-based model architecture that uses an attention mechanism to compute associations between items within a context window. The context window can include a sequence containing an input sequence 5 and potentially one or more output elements 7-1, 7-2, ..., 7- N A Transformer block can include one or more attention layers and one or more post-attention layers (e.g., feedforward layers, such as multilayer perceptrons).
[0324] In addition to or instead of Transformer-based architectures, prediction layer 6 can include other machine learning-based model architectures. For example, recurrent neural networks (RNNs) and long short-term memory (LSTM) models, as well as convolutional neural networks (CNNs) can be used. In general, prediction layer 6 can leverage various artificial neural networks that can understand or generate sequences of information.
[0325] The output sequence 7 may include or otherwise represent the same or different data type as the input sequence 5. For example, the input sequence 5 may represent text data, and the output sequence 7 may represent text data. The input sequence 5 may represent image, audio, or audiovisual data, and the output sequence 7 may represent text data (e.g., describing image, audio, or audiovisual data). It should be understood that the prediction layer 6 and any other gap model components of the sequence processing model 4 can be configured to receive multiple data types from the input sequence 5 and output multiple data types in the output sequence 7.
[0326] Output sequence 7 can have various relationships with input sequence 5. Output sequence 7 can be a continuation of input sequence 5. Output sequence 7 can be a supplement to input sequence 5. Output sequence 7 can transform, alter, expand, or otherwise modify input sequence 5. Output sequence 7 can answer, evaluate, acknowledge, or otherwise respond to input sequence 5. Output sequence 7 can implement instructions provided via input sequence 5 (or describe instructions for implementing those instructions).
[0327] Output sequence 7 can be generated autoregressively. For example, in some applications, the output of one or more prediction layers 6 can be passed through one or more output layers (e.g., softmax layers) to obtain a probability distribution over an output vocabulary (e.g., a text or symbol vocabulary) conditioned on a set of input elements in a context window. In this way, output sequence 7 can be generated autoregressively, for example, by: sampling possible next output elements; adding that element to the context window and regenerating the probability distribution based on the updated context window; and sampling possible next output elements, etc.
[0328] Output sequence 7 can also be generated non-autoregressively. For example, multiple output elements of output sequence 7 can be predicted together without explicit order conditions. See, for example, Saharia et al., Non-Autoregressive Machine Translation with Latent Alignments, arXiv:2004.07437v3 (November 16, 2020).
[0329] Output sequence 7 may include one or more parts or elements. In the example content generation configuration, output sequence 7 may include multiple elements corresponding to multiple parts of the generated output sequence (e.g., text sentences, values of discrete waveforms, computer code, etc.). In the example classification configuration, output sequence 7 may include a single element associated with the classification output. For example, the output "vocabulary" may include the set of classes to which the input sequence will be classified. For example, a visual Transformer block may pass latent state information to a multilayer perceptron, which outputs possible class values associated with the input image.
[0330] Figure 11 This is a block diagram of an example technique for populating example input sequence 8. Input sequence 8 may include various functional elements forming part of the model infrastructure, such as element 8-0 obtained from task indicator 9, which signals to any model processing input sequence 8 that a specific task is being performed (e.g., helping to adapt the model's performance to that specific task). Input sequence 8 may include various data elements from different data modalities. For example, input modality 10-1 may include one data modality. Data-to-sequence model 11-1 may process the data from input modality 10-1 to project the data into a format compatible with input sequence 8 (e.g., determining one or more vectors of dimensions based on the dimensions of input sequence 8) to obtain elements 8-1, 8-2, and 8-3. Another input modality 10-2 may include a different data modality. Data-to-sequence model 11-2 may project the data from input modality 10-2 into a format compatible with input sequence 8 to obtain elements 8-4, 8-5, and 8-6. Another input modality 10-3 may include yet another different data modality. The data to sequence model 11-3 can project the data from the input mode 10-3 into a format compatible with the input sequence 8 to obtain elements 8-7, 8-8, and 8-9.
[0331] Input sequence 8 may be the same as or different from input sequence 5. Input sequence 8 may be a multimodal input sequence containing elements representing data from different modalities using a common dimension representation. For example, the embedding space may have P The input sequence 8 can be configured to contain dimensions. P Multiple elements in one dimension. In this way, for example, the example implementation can facilitate information extraction and reasoning across different data modalities by projecting data onto elements in the same embedding space to compare, combine or otherwise compute between them.
[0332] For example, elements 8-0, ..., 8-9 can indicate specific locations within a multidimensional embedding space. Some elements can map to a discrete set of locations in the embedding space. For example, elements corresponding to discrete members in a predefined vocabulary can map to discrete locations in the embedding space associated with those lexics. Other elements can be continuously distributed across the embedding space. For example, some data types can be decomposed into continuously defined parts (e.g., image patches), which can be described using continuously distributed locations within the embedding space.
[0333] In some implementations, the expressive power of the embedding space may not be limited to meanings associated with any particular set of lexical units or other building blocks. For example, a continuous embedding space can encode a series of higher-order information. Individual fragments of information (e.g., lexical units) can be mapped to specific points in the space: for example, lexical units of the word "dog" can be projected to embedding values that point to specific locations in the embedding space associated with dog-related information. Similarly, image patches of a dog on grass can be projected into the embedding space. In some implementations, the projection of the dog image can resemble the projection of the word "dog" and also resemble the projection of the word "grass," but potentially differ from both. In some implementations, the projection of the image patch may not be perfectly aligned with any single projection of a single word. In some implementations, the projection of the image patch can be aligned with a combination of the projections of the words "dog" and "grass." In this way, for example, a higher-order embedding space can encode information that is independent of the data modality expressing the information.
[0334] Task indicator 9 may include a model or model component configured to identify the ongoing task and inject the input value represented by element 8-0 into input sequence 8, which signals which task is being performed. For example, the input value may be provided as a data type associated with an input modality and projected along with that modality (e.g., the input value may be a text task label embedded along with other text data in the input; the input value may be a pixel-based representation of the task embedded along with other image data in the input; etc.). The input value may be provided as a data type different from or at least independent of other inputs. For example, the input value represented by element 8-0 may be learned within a continuous embedding space.
[0335] Input modes 10⁻¹, 10⁻², and 10⁻³ can be associated with a variety of different data types (e.g., as described above with respect to input 2 and output 3).
[0336] Data-to-sequence models 11-1, 11-2, and 11-3 can be the same as or different from each other. Data-to-sequence models 11-1, 11-2, and 11-3 can be adapted to each corresponding input modality 10-1, 10-2, and 10-3. For example, a text data-to-sequence model can subdivide a portion of the input text and project these subdivisions onto elements in input sequence 8 (e.g., elements 8-1, 8-2, 8-3, etc.). An image data-to-sequence model can subdivide the input image and project these subdivisions onto elements in input sequence 8 (e.g., elements 8-4, 8-5, 8-6, etc.). An arbitrary data type data-to-sequence model can subdivide the input of that arbitrary data type and project these subdivisions onto elements in input sequence 8 (e.g., elements 8-7, 8-8, 8-9, etc.).
[0337] Data-to-sequence models 11-1, 11-2, and 11-3 can form part of the machine learning-based sequence processing model 4. Data-to-sequence models 11-1, 11-2, and 11-3 can be jointly trained with the machine learning-based sequence processing model 4 or trained independently of it. Data-to-sequence models 11-1, 11-2, and 11-3 can be trained end-to-end with the machine learning-based sequence processing model 4.
[0338] Example of a machine learning-based model development platform
[0339] Figure 12 This is a block diagram of an example model development platform 12, which facilitates the creation, adaptation, and refinement of example machine learning models (e.g., machine learning model 1, sequence processing model 4, etc.). The model development platform 12 can provide several different toolkits that developer systems can use to develop new or adapted machine learning models.
[0340] The model development platform 12 can provide one or more model libraries 13 containing building blocks for new models. Model libraries 13 can include one or more pre-trained base models 13-1, which can provide a backbone of processing power across a variety of tasks. Model libraries 13 can include one or more pre-trained expert models 13-2, which can focus on performance in a specific domain of expertise. Model libraries 13 can include various model primitives 13-3, which can provide low-level architectures or components (optionally pre-trained) that can be assembled in various arrangements as needed.
[0341] The model development platform 12 can receive selections of various model components 14. The model development platform 12 can transfer the selected model components 14 to the workbench 15, which combines the selected model components 14 into the development model 16.
[0342] Workbench 15 can facilitate further refinement and adaptation of the development model 16 by utilizing multiple different toolkits integrated with the model development platform 12. For example, workbench 15 can facilitate the use of model alignment toolkit 17 to align the development model 16 with expected performance profiles for various tasks.
[0343] The model alignment toolkit 17 can provide a variety of tools for enabling the development model 16 to generate outputs aligned with desired behavioral characteristics. Alignment can include increasing the accuracy, precision, recall, etc., of the model output. Alignment can include enforcing output styles, patterns, or other preferred characteristics of the model output. Alignment can be general or domain-specific. For example, the pre-trained base model 13-1 can start from an initial performance level across multiple domains. Alignment of the pre-trained base model 13-1 can include improving performance in a specific information or task domain (e.g., even at the expense of performance in another information or task domain).
[0344] The model alignment toolkit 17 can integrate one or more datasets 17-1 used to align development models 16. Selected datasets 17-1 may include labeled or unlabeled training data. Datasets 17-1 can be obtained from public domain datasets. Datasets 17-1 can also be obtained from private datasets associated with one or more developer systems used to align custom machine learning models tailored for private use cases.
[0345] The pre-training pipeline 17-2 may include a machine learning-based model training workflow configured to update the development model 16 on a large-scale, potentially noisy dataset. For example, pre-training may utilize unsupervised learning techniques (e.g., denoising, etc.) to process a large number of training instances to update model parameters from an initial state and achieve the desired baseline performance. The pre-training pipeline 17-2 may utilize the unlabeled dataset from dataset 17-1 to perform pre-training. Workbench 15 may implement the pre-training pipeline 17-2 to pre-train the development model 16.
[0346] The fine-tuning pipeline 17-3 may include a machine learning-based model training workflow configured to refine the model parameters of the development model 16 using higher-quality data. The fine-tuning pipeline 17-3 can update the development model 16 through supervised training using the labeled dataset from dataset 17-1. The fine-tuning pipeline 17-3 can also update the development model 16 through reinforcement learning using reward signals from user feedback. Workbench 15 enables the fine-tuning pipeline 17-3 to fine-tune the development model 16.
[0347] Hint library 17-4 may include a set of inputs configured to induce behavior aligned with desired performance criteria. Hint library 17-4 may include few-shot hints (e.g., inputs that provide examples of the desired model output to be added before the desired runtime query), thought chain hints (e.g., inputs that provide step-by-step reasoning within a paradigm to facilitate comprehensive reasoning by the model), and so on.
[0348] Sample hints can be obtained from the available repositories in Hint Library 17-4. Sample hints can be facilitated by one or more developer systems using Workbench 15.
[0349] In some implementations, pre-trained or fine-tuned models can achieve satisfactory performance even when there are no paradigms in the input. For example, zero-shot hints can include inputs lacking paradigms. Zero-shot hints can be within the domain of the training dataset or outside the training domain.
[0350] Hint library 17-4 may include one or more hint engineering tools. Hint engineering tools can provide a workflow for retrieving or learning optimized hint values. Hint engineering tools can facilitate the direct learning of hint values (e.g., input element values) based on one or more training iterations. Workbench 15 can implement the hint engineering tools in development model 16.
[0351] Hint library 17-4 may include a pipeline for hint generation. For example, input can be generated using development model 16 itself or other machine learning-based models. In this way, for example, a first model can process information about the task and output input for a second model to process in order to perform the steps of the task. The second model may be the same as or different from the first model. Workbench 15 can implement the hint generation pipeline within development model 16.
[0352] Hint library 17-4 may include a pipeline for context injection. For example, if additional context is provided for performing a specific task, the performance of development model 16 on that task can be improved. Hint library 17-4 may include software components configured to identify desired context, retrieve context from external sources (e.g., databases, sensors, etc.), and add the context to input hints. Workbench 15 can implement the context injection pipeline in development model 16.
[0353] Although the various training examples described herein regarding model development platform 12 involve "pre-training" and "fine-tuning," it should be understood that the model alignment toolkit 17 generally supports a wide variety of training techniques adapted to train a wide range of machine learning models. Example training techniques may correspond to the example training methods 800 described above.
[0354] The model development platform 12 may include a model plug-in toolkit 18. The model plug-in toolkit 18 may include a variety of tools configured to enhance the functionality of the machine learning model by integrating it with other systems, devices, and software components. For example, the machine learning model can use tools to improve performance quality where appropriate. For instance, deterministic tasks can be offloaded to dedicated tools instead of performing tasks probabilistically when the risk of error increases. For example, the machine learning model can identify the tools to be invoked to obtain a solution and pass the system of equations to the appropriate tool instead of predicting the solution to the system of equations autoregressively. This tool may be a conventional equation solver that operates deterministically to solve the system of equations. The tool's output can be returned in response to the original query. In this way, tool use can allow some example models to focus on the strengths of the machine learning model—e.g., understanding the intent in unstructured requests for a task—while enhancing model performance by offloading certain tasks to more focused tools to mechanically apply deterministic algorithms to well-defined problems.
[0355] The model plugin toolkit 18 may include a validation tool 18-1. The validation tool 18-1 may include tools capable of parsing and validating the output of a machine learning model. The validation tool 18-1 may include engineered heuristics that establish certain thresholds applied to the model output. For example, the validation tool 18-1 may base the output of the machine learning model on a structured data source (e.g., to mitigate "illusion").
[0356] The model plugin toolkit 18 may include a toolkit 18-2 for implementing one or more tools, which may include scripts or other executable code that can be executed with the development model 16. The toolkit 18-2 may include one or more inputs configured to enable the machine learning-based model to implement the tools (e.g., few-shot hints that induce the model to output tool calls with correct syntax). For example, the toolkit 18-2 may include fine-tuned training data for training the model to use the tools.
[0357] The model plugin toolkit 18 may include interfaces for calling external application programming interfaces (APIs) 18-3. For example, as a supplement or alternative to directly implementing tool calls or tool code using development model 16, development model 16 may be aligned with output instructions that initiate API calls to send or obtain data via external systems.
[0358] The model plugin toolkit 18 can be integrated with the hint library 17-4 to create a catalog of available tools for use with the development model 16. For example, the model can receive a catalog of available tools in its input, and the model can generate output that selects a tool from the available tools and initiates a tool call for using that tool.
[0359] Model development platform 12 may include a suite of computational optimization tools 19 for optimizing the computational performance of development model 16. For example, tools for model compression 19-1 can reduce the size of development model 16 while maintaining the desired performance level. For example, model compression 19-1 may include quantization workflows, weight pruning, and sparsification techniques. Tools for hardware acceleration 19-2 can facilitate the configuration of model storage and execution formats for optimal operation on different hardware resources. For example, hardware acceleration 19-2 may include tools for optimally sharding the model for distributed processing across multiple processing units to increase bandwidth, reduce uniform memory requirements, etc. Tools for refinement 19-3 can provide tools for training a lighter model based on the knowledge encoded in development model 16. For example, development model 16 may be a large, high-performance machine learning model optimized using model development platform 12. To obtain a lightweight model for operation in resource-constrained environments, the smaller model may be a “student model” that learns from and imitates development model 16 as the “teacher model.” In this way, for example, the investment in learning and developing the parameters and configuration of model 16 can be efficiently transferred to a smaller model for more efficient inference.
[0360] Workbench 15 may implement one or more of the toolkits implemented in model development platform 12, or may not implement any toolkits. Workbench 15 may output output model 20 based on development model 16. Output model 20 may be a deployment version of development model 16. Output model 20 may be a development or training checkpoint of development model 16. Output model 20 may be a distilled version, compressed version, or otherwise optimized version of development model 16.
[0361] Figure 13 This is a block diagram of an example training process for training a machine learning development model 16. One or more parts of the example training process can be implemented by a computing system (such as the computing system described, for example, with reference to other figures) that includes one or more computing devices. Each corresponding part of the example training process can be performed by any one (or any combination of) of the one or more computing devices. Furthermore, one or more parts of the example training process can be implemented on the hardware components of the apparatus described herein, for example, to train one or more systems or models. Figure 13 For illustrative and discussion purposes, elements executed in a specific order are depicted. Those skilled in the art will understand using the disclosure provided herein that elements of any method discussed herein can be adapted, rearranged, extended, omitted, combined, or modified in various ways without departing from the scope of this disclosure. Figure 13 The elements / terms described are for illustrative purposes only and with reference to other systems and figures, and are not intended to be limiting. Alternatively, one or more parts of the example training process may be performed by other systems.
[0362] Initially, development model 16 can be kept in its initial state as initialization model 21. Development model 16 can be initialized using weight values. The initial weight values can be random or based on an initialization pattern. The initial weight values can be based on previous pre-training for the same or different models.
[0363] The initialization model 21 can undergo pre-training in the pre-training phase 22. The pre-training phase 22 can be implemented using one or more pre-training pipelines 17-2 on data from dataset 17-1. For example, if the initialization model 21 has already been pre-trained (e.g., the development model 16 contains, is, or is based on a pre-trained base model or expert model), pre-training can be omitted.
[0364] The pre-trained model 23 can then be a new version of the development model 16, which can remain as the development model 16 or be a new development model. If the development model 16 has already been pre-trained, the pre-trained model 23 can be in its initial state. The pre-trained model 23 can undergo fine-tuning in the fine-tuning phase 24. The fine-tuning phase 24 can be implemented using one or more fine-tuning pipelines 17-3 on data from dataset 17-1. For example, fine-tuning can be omitted if the pre-trained model has satisfactory performance, if the model has already been fine-tuned, or if other tuning methods are preferred.
[0365] The fine-tuned model 29 can then be a new version of the development model 16, which can remain as the development model 16 or be a newly developed model. If the development model 16 has already been fine-tuned, the fine-tuned model 29 can be in its initial state. The fine-tuned model 29 can undergo refinement 26 using user feedback. For example, refinement 26 using user feedback can optionally include reinforcement learning based on human feedback from human users of the fine-tuned model 25. Since reinforcement learning can take the form of fine-tuning, it should be understood that the fine-tuning phase 24 can include a phase for refinement 26 using user feedback. Refinement 26 using user feedback can produce a refined model 27. The refined model 27 can be output to the downstream system 28 for deployment or further development.
[0366] In some implementations, computational optimization operations can be applied before, during, or after each stage. For example, initializing model 21 may undergo computational optimization 29-1 (e.g., using computational optimization toolkit 19) before pre-training stage 22. Pre-trained model 23 may undergo computational optimization 29-2 (e.g., using computational optimization toolkit 19) before fine-tuning stage 24. Fine-tuned model 25 may undergo computational optimization 29-3 (e.g., using computational optimization toolkit 19) before refining 26 using user feedback. Refined model 27 may undergo computational optimization 29-4 (e.g., using computational optimization toolkit 19) before outputting to downstream system 28. Computational optimizations 29-1, ..., 29-4 may all be the same, all be different, or include at least some different optimization techniques.
[0367] Example of a machine learning-based model inference system
[0368] Figure 14This is a block diagram of an inference system used to operate one or more machine learning-processed models 1 to perform inference (e.g., for training, for deployment, etc.). Model host 31 can receive machine learning-processed models 1. Model host 31 can host one or more model instances 31-1, which can be one or more instances of one or more models. Model host 31 can use available computing resources 31-2 associated with model host 31 to host model instances 31-1.
[0369] Model host 31 can perform inference on behalf of one or more clients 32. Client 32 can transmit input request 33 to model host 31. Using input request 33, model host 31 can obtain input 2 to feed into machine learning model 1. Machine learning model 1 can process input 2 to generate output 3. Using output 3, model host 31 can return output payload 34 in response to input request 33 from client 32. Output payload 34 can include or be based on output 3.
[0370] Model host 31 can utilize various other resources and tools to enhance the inference task. For example, model host 31 can communicate with tool interface 35 to facilitate the use of tools by model instance 31-1. Tool interface 35 may include local or remote APIs. Tool interface 35 may include integrated scripts or other software functionality. Model host 31 can use online learning interface 36 to facilitate continuous improvement of the machine learning-based model 1. For example, online learning interface 36 can be used in reinforcement learning loops to retrieve user feedback on the inference served by model host 31. Model host 31 can access runtime data source 37 to enhance input 2 using additional contextual information. For example, runtime data source 37 may include knowledge graph 37-1 that facilitates structured information retrieval for information associated with input request 33 (e.g., search engine service). Runtime data source 37 may include public or private, external or local database 37-2 that can store information associated with input request 33 for enhancing input 2. The runtime data source 37 may include account data 37-3, which can be retrieved in association with the user account corresponding to the client 32 to customize the behavior of the model host 31 accordingly.
[0371] The model host 31 may be implemented by one or more computing devices or systems. The client 2 may be implemented by one or more computing devices or systems, which may include computing devices or systems shared with the model host 31.
[0372] For example, model host 31 can operate on a server system that provides machine learning services (e.g., via a local area network or wide area network) to client devices operating client 32. The client device can be an end-user device used by an individual. The client device can also be a server system that operates client 32 to provide various functionalities as services to downstream end-user devices.
[0373] In some implementations, model host 31 may operate on the same device or system as client 32. Model host 31 may be a machine learning service that runs on the device to provide machine learning functionality to one or more applications operating on the client device, which may include the application implementing client 32. Model host 31 and client 32 may be part of the same application. For example, model host 31 may be a subroutine or method implemented as part of the application, and client 32 may be another subroutine or method that uses model host 31 to perform inference functionality within the application. It should be understood that model host 31 and client 32 may have various different configurations.
[0374] Model instance 31-1 may include one or more machine learning models that can be used to perform inference. Model instance 31-1 may include weights or other model components stored in persistent storage, temporarily cached, or loaded into memory. Model instance 31-1 may include multiple instances of the same model (e.g., for parallel execution of more requests on the same model). Model instance 31-1 may include instances of different models. Model instance 31-1 may include cached intermediate states of active or inactive models, which are used to accelerate inference for those models. For example, an inference session with a particular model can generate a significant amount of computational results that can be reused for future inference runs (e.g., using a KV cache for a Transformer-based model). These computational results can be stored in association with the inference session, allowing for more efficient execution when the session resumes.
[0375] Computing resource 31-2 may include one or more processors (central processing unit, graphics processing unit, tensor processing unit, machine learning accelerator, etc.) connected to one or more memory devices. Computing resource 31-2 may include a dynamic pool of available resources shared with other processes. Computing resource 31-2 may include a memory device large enough to fit an entire model instance into a single memory instance. Computing resource 31-2 may also shard model instances across multiple memory devices (e.g., using data parallelization or tensor parallelization). Doing so can increase parallelization or execute large models using multiple memory devices that, individually, may not be able to fit the entire model into memory.
[0376] Input request 33 may include data for input 2. Model host 31 can process input request 33 to obtain input 2. Input 2 can be obtained directly from input request 33 or can be obtained using input request 33. Input request 33 can be submitted to model host 31 via API.
[0377] Model host 31 can perform inference on multiple batches of input requests 33 in parallel. For example, model instance 31-1 can be configured with an input structure having a batch dimension. Individual inputs 2 can be distributed across batch dimensions (e.g., rows of an array). Individual inputs 2 can include completely different contexts. Individual inputs 2 can be multiple inference steps for the same task. Individual inputs 2 can be interleaved in the input structure, such that any given inference loop can operate on different parts of the corresponding inputs 2. In this way, for example, model host 31 can perform inference on batches in parallel, such that output 3 can also contain a batch dimension and return the inference results of the batched inputs 2 in parallel. In this way, for example, multiple batches of input requests 33 can be processed in parallel to achieve higher throughput of output payload 34.
[0378] The output payload 34 may include or be based on the output 3 from the machine learning model 1. The model host 31 may process the output 3 to obtain the output payload 34. This may include linking multiple rounds of inference (e.g., iteratively, recursively, across the same model or different models) to obtain the final output of the task to be returned in the output payload 34. The output payload 34 may be transferred to the client 32 via an API.
[0379] Online learning interface 36 can facilitate reinforcement learning of machine learning-based Model 1. Online learning interface 36 can facilitate reinforcement learning with human feedback (RLHF). Online learning interface 36 can facilitate federated learning of machine learning-based Model 1.
[0380] Model host 31 can execute machine learning-enabled model 1 to perform inference for various tasks using various types of data. For example, various inputs 2 and outputs 3 can be used for various different tasks. In some implementations, input 2 may be or otherwise represent image data. Machine learning-enabled model 1 can process image data to generate outputs. As an example, machine learning-enabled model 1 can process image data to generate image recognition outputs (e.g., image data recognition, latent embedding of image data, encoded representation of image data, hashing of image data, etc.). As another example, machine learning-enabled model 1 can process image data to generate image segmentation outputs. As another example, machine learning-enabled model 1 can process image data to generate image classification outputs. As another example, machine learning-enabled model 1 can process image data to generate image data modification outputs (e.g., image data alterations, etc.). As another example, machine learning-enabled model 1 can process image data to generate encoded image data outputs (e.g., encoded and / or compressed representations of image data, etc.). As another example, machine learning-enabled model 1 can process image data to generate expanded image data outputs. As another example, Model 1, which is processed by machine learning, can process image data to generate predictive output.
[0381] In some implementations, the task is a computer vision task. In some cases, input 2 includes pixel data from one or more images, and the task is an image processing task. For example, the image processing task could be image classification, where the output is a set of scores, each score corresponding to a different object class and representing the probability that the one or more images depict an object belonging to that object class. The image processing task could be object detection, where the image processing output identifies one or more regions in the one or more images, and for each region, identifies the probability that the region depicts an object of interest. As another example, the image processing task could be image segmentation, where the image processing output defines a corresponding probability for each class in a predetermined set of classes for each pixel in the one or more images. For example, the class set could be foreground and background. As another example, the class set could be object classes. As another example, the image processing task could be depth estimation, where the image processing output defines a corresponding depth value for each pixel in the one or more images. As another example, the image processing task could be motion estimation, where the network input includes multiple images, and the image processing output defines the motion of the scene depicted at each pixel in one of the input images between the images in the network input.
[0382] In some implementations, input 2 can be or otherwise represent natural language data. The machine learning-enabled model 1 can process the natural language data to generate output. As an example, the machine learning-enabled model 1 can process natural language data to generate a language-encoded output. As another example, the machine learning-enabled model 1 can process natural language data to generate a latent text embedding output. As another example, the machine learning-enabled model 1 can process natural language data to generate a translation output. As another example, the machine learning-enabled model 1 can process natural language data to generate a classification output. As another example, the machine learning-enabled model 1 can process natural language data to generate a text segmentation output. As another example, the machine learning-enabled model 1 can process natural language data to generate a semantic intent output. As another example, the machine learning-enabled model 1 can process natural language data to generate an expanded text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language). As another example, the machine learning-enabled model 1 can process natural language data to generate a predicted output (e.g., one or more predicted subsequent portions of the natural language content).
[0383] In some implementations, input 2 can be or otherwise represent speech data (e.g., data describing spoken natural language, such as audio data, text data, etc.). The machine learning model 1 can process the speech data to generate output. As an example, the machine learning model 1 can process speech data to generate speech recognition output. As another example, the machine learning model 1 can process speech data to generate speech translation output. As another example, the machine learning model 1 can process speech data to generate latent embedding output. As another example, the machine learning model 1 can process speech data to generate encoded speech output (e.g., encoded and / or compressed representations of speech data, etc.). As another example, the machine learning model 1 can process speech data to generate amplified speech output (e.g., speech data of higher quality than the input speech data, etc.). As another example, the machine learning model 1 can process speech data to generate text representation output (e.g., a text representation of the input speech data, etc.). As another example, the machine learning model 1 can process speech data to generate predictive output.
[0384] In some implementations, input 2 can be or otherwise represent latent encoded data (e.g., a latent space representation of the input). The machine learning-processed model 1 can process the latent encoded data to generate an output. As an example, the machine learning-processed model 1 can process the latent encoded data to generate a recognition output. As another example, the machine learning-processed model 1 can process the latent encoded data to generate a reconstruction output. As another example, the machine learning-processed model 1 can process the latent encoded data to generate a search output. As another example, the machine learning-processed model 1 can process the latent encoded data to generate a re-clustering output. As yet another example, the machine learning-processed model 1 can process the latent encoded data to generate a prediction output.
[0385] In some implementations, input 2 may be or otherwise represent statistical data. Statistical data may be, represent, or otherwise include data calculated and / or computed from another data source. The machine learning model 1 can process the statistical data to generate output. As an example, the machine learning model 1 can process the statistical data to generate a recognition output. As another example, the machine learning model 1 can process the statistical data to generate a prediction output. As another example, the machine learning model 1 can process the statistical data to generate a classification output. As another example, the machine learning model 1 can process the statistical data to generate a segmentation output. As another example, the machine learning model 1 can process the statistical data to generate a visualization output. As another example, the machine learning model 1 can process the statistical data to generate a diagnostic output.
[0386] In some implementations, input 2 can be or otherwise represent sensor data. The machine learning-based model 1 can process the sensor data to generate output. As an example, the machine learning-based model 1 can process sensor data to generate a recognition output. As another example, the machine learning-based model 1 can process sensor data to generate a prediction output. As another example, the machine learning-based model 1 can process sensor data to generate a classification output. As another example, the machine learning-based model 1 can process sensor data to generate a segmentation output. As another example, the machine learning-based model 1 can process sensor data to generate a visualization output. As another example, the machine learning-based model 1 can process sensor data to generate a diagnostic output. As another example, the machine learning-based model 1 can process sensor data to generate a detection output.
[0387] In some implementations, the machine learning-based Model 1 can be configured to perform a task that includes encoding input data to achieve reliable and / or efficient transmission or storage (and / or corresponding decoding). For example, the task could be an audio compression task. The input could include audio data, and the output could include compressed audio data. In another example, the input includes visual data (e.g., one or more images or videos), the output includes compressed visual data, and the task is a visual data compression task. In another example, the task could include generating embeddings for input data (e.g., input audio or visual data). In some cases, the input includes audio data representing spoken utterances, and the task is a speech recognition task. The output could include text output mapped to spoken utterances. In some cases, the task includes encrypting or decrypting the input data. In some cases, the task includes microprocessor performance tasks such as branch prediction or memory address translation.
[0388] In some implementations, the task is a generative task, and the machine learning-based model 1 can be configured to output content generated from input 2. For example, input 2 can be, or otherwise represent, data of one or more modalities that encode the context used to generate additional content.
[0389] In some implementations, the task can be a text completion task. A machine learning-based model 1 can be configured to process input 2, representing text data, and generate output 3, which represents additional text data to complete the text sequence of input 2. For example, the machine learning-based model 1 can be configured to generate output 3 to complete a sentence, paragraph, or section of text following a portion of the text represented by input 2.
[0390] In some implementations, the task can be an instruction-following task. A machine learning model 1 can be configured to process input 2 representing instructions for performing a function and generate output 3 that advances towards a goal satisfying the function of the instructions (e.g., at least one step of a multi-step process for performing the function). Output 3 can represent data of the same or different modality as input 2. For example, input 2 can represent text data (e.g., natural language instructions for a task to be performed), and machine learning model 1 can process input 2 to generate output 3 representing text data in response to the instructions (e.g., a natural language response, a programming language response, a machine language response, etc.). Input 2 can represent image data (e.g., image-based instructions for a task to be performed, optionally accompanied by text instructions), and machine learning model 1 can process input 2 to generate output 3 representing text data in response to the instructions (e.g., a natural language response, a programming language response, a machine language response, etc.). One or more outputs 3 can be generated iteratively or recursively to sequentially process and complete steps tending towards completing the requested functionality. For example, the initial output can be executed by an external system or processed by a machine learning model 1 to complete the initial steps of the function. Multiple steps can be executed, in which the final output is obtained in response to the initial instructions.
[0391] In some implementations, the task can be a question-answering task. A machine learning model 1 can be configured to process input 2 representing a question to be answered and generate output 3 that advances towards the goal of returning an answer to the question (e.g., at least one step in a multi-step process for performing the function). Output 3 can represent data of the same or different modality as input 2. For example, input 2 can represent text data (e.g., natural language instructions for a task to be performed), and machine learning model 1 can process input 2 to generate output 3 representing text data in response to the question (e.g., a natural language response, a programming language response, a machine language response, etc.). Input 2 can represent image data (e.g., image-based instructions for a task to be performed, optionally accompanied by text instructions), and machine learning model 1 can process input 2 to generate output 3 representing text data in response to the question (e.g., a natural language response, a programming language response, a machine language response, etc.). One or more outputs 3 can be generated iteratively or recursively to process and complete the steps toward answering the question sequentially. For example, the initial output can be executed by an external system or processed by a machine learning model 1 to complete the initial steps to obtain an answer to the question (e.g., querying a database, performing calculations, executing scripts, etc.). Multiple steps can be performed, in which the final output in response to the question is obtained.
[0392] In some implementations, the task can be an image generation task. A machine learning model 1 can be configured to process input 2, which represents context regarding a desired portion of the image content. Context can include text data, image data, audio data, etc. The machine learning model 1 can be configured to generate output 3, which represents image data depicting the image in context. For example, the machine learning model 1 can be configured to generate pixel data of an image. The values of the channels associated with pixels in the pixel data can be selected based on context (e.g., based on probabilities determined according to the context).
[0393] In some implementations, the task can be an audio generation task. A machine learning model 1 can be configured to process input 2, which represents context regarding a desired portion of the audio content. Context can include text data, image data, audio data, etc. The machine learning model 1 can be configured to generate output 3, which represents context-dependent audio data. For example, the machine learning model 1 can be configured to generate waveform data in the form of an image (e.g., a spectrogram). The values of channels associated with pixels in the image can be selected based on the context. The machine learning model 1 can also be configured to generate waveform data in the form of a sequence of discrete samples of a continuous waveform. The values of the sequence can be selected based on the context (e.g., based on probabilities determined according to the context).
[0394] In some implementations, the task can be a data generation task. The machine learning-based model 1 can be configured to process input 2, which represents context regarding a desired portion of the data (e.g., data from various data domains, such as sensor data, image data, multimodal data, statistical data, etc.). For example, the desired data can be synthetic data used to train other machine learning models. The context can include any data type. The machine learning-based model 1 can be configured to generate output 3, which represents data aligned with the desired data. For example, the machine learning-based model 1 can be configured to generate data values to populate a dataset. The values of data objects can be selected based on context (e.g., based on probabilities determined according to the context).
[0395] Example computing systems and devices
[0396] Figure 15This is a block diagram of an example networked computing system capable of implementing aspects of the exemplary implementations of this disclosure. The system may include multiple computing devices and systems communicatively coupled via network 49. Example computing device 50 is described as an example of a computing device capable of implementing any aspect of this disclosure (e.g., implementing model host 31, client 32, or both). Example server computing system 60 is described as an example of a server computing system capable of implementing any aspect of this disclosure (e.g., implementing model host 31, client 32, or both). Computing device 50 and server computing system 60 may interact collaboratively (e.g., via network 49) to implement any aspect of this disclosure (e.g., implementing model host 31, client 32, or both). Model development platform system 70 is an example system that can host or provide a model development platform 12 for developing machine learning models. Third-party system 80 is an example system that any of the computing device 50, server computing system 60, or model development platform system 70 can interact with when performing various aspects of this disclosure (e.g., using third-party tools, accessing third-party databases or other resources, etc.).
[0397] Network 49 can be any type of communication network, such as a local area network (e.g., intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. Generally, communication via network 49 can use a wide variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), or protection schemes (e.g., VPN, Secure HTTP, SSL) via any type of wired or wireless connection. Network 49 can also be implemented via a system bus. For example, Figure 15 One or more devices or systems may be located in the same place as, contained in, or otherwise integrated into one or more other devices or systems.
[0398] Computing device 50 can be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop computer), a mobile computing device (e.g., a smartphone or tablet), a game console or controller, a wearable computing device, an embedded computing device, a server computing device, a virtual machine operating on a host device, or any other type of computing device. Computing device 50 can be a client computing device. Computing device 50 can be an end-user computing device. Computing device 50 can be a computing device that provides services to an end user (who may use another computing device to interact with computing device 50).
[0399] Computing device 50 may include one or more processors 51 and memory 52. Processor 51 may be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be a single processor or multiple processors operatively connected. Memory 52 may include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 52 may store data 53 and instructions 54, which can be executed by processor 51 to cause computing device 50 to perform operations. These operations may implement any one or more of the functions described herein. These operations may implement the example methods and techniques described herein.
[0400] The computing device 50 may also include one or more input components for receiving user input. For example, the user input component may be a touch-sensitive component (e.g., a touch-sensitive display or touchpad) that is sensitive to the touch of a user input object (e.g., a finger or stylus). The touch-sensitive component can be used to implement a virtual keyboard. Other example user input components include a microphone, camera, LiDAR, physical keyboard or other buttons, or other components through which the user can provide input.
[0401] The computing device 50 may store or include one or more machine learning models 55. The machine learning models 55 may include one or more machine learning models 1, such as sequence processing model 4. The machine learning models 55 may include one or more model instances 31-1. The machine learning models 55 may be received from a server computing system 60, a model development platform system 70, a third-party system 80 (e.g., an application distribution platform), or developed locally on the computing device 50. The machine learning models 55 may be loaded into memory 52 and used by the processor 51 or otherwise implemented. The computing device 50 may implement multiple parallel instances of the machine learning models 55.
[0402] Server computing system 60 may include one or more processors 61 and memory 62. Processor 61 may be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be a single processor or multiple processors operatively connected. Memory 62 may include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 62 may store data 63 and instructions 64, which can be executed by processor 61 to cause server computing system 60 to perform operations. These operations may implement any one or more of the functions described herein. These operations may implement the exemplary methods and techniques described herein.
[0403] In some implementations, the server computing system 60 includes one or more server computing devices or is otherwise implemented by one or more server computing devices. In instances where the server computing system 60 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.
[0404] Server computing system 60 may store or otherwise include one or more machine learning models 65. Machine learning models 65 may be the same as or different from machine learning models 55. Machine learning models 65 may include one or more machine learning models 1, such as sequence processing model 4. Machine learning models 65 may include one or more model instances 31-1. Machine learning models 65 may be received from computing device 50, model development platform system 70, third-party system 80, or developed locally on server computing system 60. Machine learning models 65 may be loaded into memory 62 and used by processor 61 or otherwise implemented. Server computing system 60 may implement multiple parallel instances of machine learning models 65.
[0405] In the example configuration, the machine learning model 65 may be included in or otherwise stored and implemented by the server computing system 60 to establish a client-server relationship with the computing device 50 for service model inference. For example, the server computing system 60 may implement the model host 31 on behalf of client 32 on the computing device 50. For example, the machine learning model 65 may be implemented by the server computing system 60 as part of a web service (e.g., a remote machine learning model hosting service, such as an online interface for performing machine learning model operations on the server computing system 60 over a network). For example, the server computing system 60 may communicate with the computing device 50 via a local intranet or internet connection. For example, the computing device 50 may be a workstation or endpoint communicating with the server computing system 60, where the implementation of the machine learning model 65 is managed by the server computing system 60 to remotely perform inference (e.g., for runtime or training operations), and where output is returned (e.g., projected, streamed, etc.) to the computing device 50. The machine learning model 65 can work collaboratively or interoperably with the machine learning model 55 on the computing device 50 to perform various tasks.
[0406] The model development platform system 70 may include one or more processors 71 and memory 72. Processor 71 may be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be a single processor or multiple processors operatively connected. Memory 72 may include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 72 may store data 73 and instructions 74, which can be executed by processor 71 to cause the model development platform system 70 to perform operations. These operations may implement any one or more of the functions described herein. These operations may implement the example methods and techniques described herein. Example operations include the functionality described herein with respect to model development platform 12. This functionality and other functionalities may be implemented by developer tools 75.
[0407] The third-party system 80 may include one or more processors 81 and memory 82. Processor 81 may be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be a single processor or multiple processors operatively connected. Memory 82 may include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 82 may store data 83 and instructions 84, which may be executed by processor 81 to cause the third-party system 80 to perform operations. These operations may implement any one or more of the functions described herein. These operations may implement the example methods and techniques described herein. Example operations include the functionality described herein with respect to tools and other external resources (e.g., third-party resource 85) invoked when training machine learning models 1, 4, 16, 20, 55, 65, etc., or performing inference using machine learning models.
[0408] Figure 15 An example arrangement of a computing system that can be used to implement the present disclosure is shown. Other computing system configurations may also be used. For example, in some implementations, one or both of computing system 50 or server computing system 60 may implement all or part of the operation of model development platform system 70. For example, computing system 50 or server computing system 60 may implement developer tool 75 (or extensions thereof) to develop, update / train, or refine machine learning models 1, 4, 16, 20, 55, 65, etc., using one or more techniques described herein with respect to model alignment toolkit 17. In this way, for example, computing system 50 or server computing system 60 may develop, update / train, or refine machine learning models based on local datasets (e.g., for model personalization / customization, as permitted by user data preference selection).
[0409] Figure 16 This is a block diagram of an example computing device 98 implemented according to an example embodiment of the present disclosure. The computing device 98 may be a user computing device or a server computing device (e.g., computing device 50, server computing system 60, etc.). The computing device 98 may implement model host 31. For example, the computing device 98 may include multiple applications (e.g., applications 1 to N). Each application may contain its own machine learning library and machine learning-enabled models. For example, each application may include machine learning-enabled models. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, etc. Figure 16As shown, each application can communicate with multiple other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, or additional components. In some implementations, each application can communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is application-specific.
[0410] Figure 17 This is a block diagram of an example computing device 99 implemented according to an example embodiment of the present disclosure. Computing device 99 may be the same as or different from computing device 98. Computing device 99 may be a user computing device or a server computing device (e.g., computing device 50, server computing system 60, etc.). Computing device 98 may implement model host 31. For example, computing device 99 may include multiple applications (e.g., applications 1 to N). Each application may communicate with a central intelligence layer. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, etc. In some implementations, each application may use an API (e.g., a common API across all applications) to communicate with the central intelligence layer (and the model stored therein).
[0411] The central intelligence layer can include multiple machine learning-engineered models. For example, such as... Figure 17 As shown, a corresponding machine learning model can be provided for each application, and this corresponding machine learning model is managed by a central intelligence layer. In other implementations, two or more applications can share a single machine learning model. For example, in some implementations, the central intelligence layer can provide a single model for all applications. In some implementations, the central intelligence layer is included within or otherwise implemented by the operating system of the computing device 99.
[0412] The central intelligence layer can communicate with the central device data layer. The central device data layer can be a centralized data storage repository for computing device 99. For example... Figure 17 As shown, the central device data layer can communicate with multiple other components of the computing device (e.g., one or more sensors, context managers, device state components, or additional components). In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).
[0413] Additional Disclosure
[0414] The technologies discussed in this paper involve servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and divisions of tasks and functionalities among and within components. For example, the processes discussed in this paper can be implemented using a single device or component, or multiple devices or components working collaboratively. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.
[0415] While the subject matter has been described in detail with respect to various specific exemplary embodiments, each example is provided by way of explanation rather than limitation. Those skilled in the art, upon gaining an understanding of the foregoing, will readily generate changes, variations, and equivalents to such embodiments. Therefore, this disclosure does not exclude such modifications, alterations, or additions to the subject matter that will be readily understood by those of ordinary skill in the art. For example, features shown or described as part of an embodiment may be used with another embodiment to produce yet another embodiment. Therefore, this disclosure is intended to cover such changes, alterations, and equivalents.
[0416] Various aspects of this disclosure have been described with reference to their illustrative embodiments. Any and all features of the appended claims can be combined or rearranged in any possible manner, including combinations of claims not expressly listed together, because the illustrative claims dependencies listed herein should not be construed as limiting the scope of possible combinations of features disclosed herein. Therefore, the scope of this disclosure is illustrative rather than limiting, and this disclosure does not exclude such modifications, alterations, and / or additions to the subject matter that will be readily understood by one of ordinary skill in the art. Furthermore, terms are described herein using a list of illustrative elements connected by conjunctions such as “and,” “or,” “however,” etc. It should be understood that such conjunctions are provided for illustrative purposes only. For example, a sequence of terms and other items connected by a specific conjunction such as “or” may refer to “and / or,” “at least one of,” “any combination,” etc., of the illustrative elements listed therein. Terms such as “based on” should be understood as “at least partially based on.”
[0417] The term "may" should be understood as referring to the possibility of a feature in various implementations, rather than a capability that must exist in every implementation. For example, the phrase "X can execute Y" should be understood as indicating that in various implementations, X may be configured to execute Y, rather than indicating that X must always be able to execute Y in every instance. It should be understood that in various implementations, X may not be able to execute Y and is still within the scope of this disclosure.
[0418] The term "may" should be understood as referring to the possibility of a feature in various implementations, rather than specifying a capability that must exist in every implementation. For example, the phrase "X can perform Y" should be understood as indicating that in various implementations, X may be configured to perform Y, rather than indicating that X must always be able to perform Y in every instance. It should be understood that in various implementations, X may not be able to perform Y and is still within the scope of this disclosure.
Claims
1. A computer-implemented method for validating the output of a machine learning-based task processing system, comprising: Receive requests to perform tasks; Multiple tool paths for performing the task are generated using a machine learning model, wherein each tool path includes one or more tool calls generated by the machine learning model, and each tool call is configured to cause the corresponding tool to perform the steps of the task; The response to the request is selected based on a consistency metric determined by evaluating multiple output states from the plurality of tool paths, respectively; and Return the response.
2. The computer-implemented method according to claim 1, wherein, The machine learning model is a sequence processing model configured to receive input comprising one or more input portions and generate output comprising one or more output portions conditioned on the one or more input portions.
3. The computer-implemented method according to any of the preceding claims, wherein, Generating the multiple tool paths using the machine learning-based model includes: For each corresponding tool path: For each corresponding step of the corresponding tool path: Invoke using the machine learning-based model generation tool; Enable the tool call to initiate the execution of the corresponding tool in the tool execution environment; Obtain results from the tool; and Evaluate the results.
4. The computer-implemented method according to any of the preceding claims, wherein, Generating the tool call using the machine learning-based model includes: Receive request data describing at least one aspect of the request; Obtain an index of available tools; The requested data and the index are provided to the machine learning-processed model; Receive the output generated by the machine learning model based on the requested data and the index, wherein the output describes the tool invocation; and Parse the output to obtain the tool call.
5. The computer-implemented method according to any of the preceding claims, wherein: The machine learning model generates tool calls for each step of the operation sequence in a round-by-round manner, wherein a first-round subset is used to activate the tool, and a second-round subset is used to obtain information; and The first round subset is resampled to obtain multiple different tool calls, and the second round subset is able to be decoded using greedy sampling.
6. The computer-implemented method according to any one of the preceding claims, comprising: The consistency metric is determined by comparing the alignment of the output states.
7. The computer-implemented method according to any one of the preceding claims, comprising: The consistency metric is determined using a machine learning-based model to evaluate the quality of each output state, wherein: The machine learning model used to evaluate the quality of each output state is the same as the machine learning model used to generate the plurality of tool paths; or The machine learning model used to evaluate the quality of each output state is different from the machine learning model used to generate the multiple tool paths.
8. The computer-implemented method according to any of the preceding claims, wherein, The machine learning model used to evaluate the quality of each output state is trained to process the multiple output states together to output a preferred output state, wherein the response is selected using the preferred output state.
9. The computer-implemented method according to any of the preceding claims, wherein, The consistency metric is based on the similarity of the learned embeddings generated using the multiple output states.
10. The computer-implemented method according to any of the preceding claims, wherein, The consistency metric is determined by processing the output state using a machine learning model trained to generate outputs that indicate significant differences between the inputs.
11. The computer-implemented method according to any of the preceding claims, wherein, The consistency metric includes one or more heuristic criteria and one or more probabilistic criteria, which are determined using the output of a machine learning model.
12. The computer-implemented method according to any one of the preceding claims, comprising: The response is selected based on the consistency metric in the following manner: Identify one or more features shared by a set of output states among the plurality of output states; as well as Select the response from the group.
13. The computer-implemented method according to any of the preceding claims, wherein, The one or more features include at least one latent feature generated using a machine learning model.
14. The computer-implemented method according to any one of the preceding claims, comprising: The response is selected based on the consistency metric in the following manner: Cluster the representations of one or more features of the multiple output states; Identify preferred clusters; and The response is selected from the cluster.
15. The computer-implemented method according to any of the preceding claims, wherein, Selecting the response from the cluster includes scoring the members of the cluster and selecting the response based on the scores.
16. The computer-implemented method according to any of the preceding claims, wherein, Selecting the response from the cluster includes determining the centroid of the cluster and selecting the response based on the centroid.
17. The computer-implemented method according to any of the preceding claims, wherein: The corresponding tool path includes one or more nodes, wherein the corresponding output status of the corresponding tool path is based on the completion node of the corresponding tool path; and The first node of the one or more nodes is generated in the following manner: The machine learning model is used to generate a first tool invocation for performing the first step of the task; and Initiate the execution of the first tool call to obtain the first tool response.
18. The computer-implemented method according to any of the preceding claims, wherein, Initiating the execution of the first tool call includes passing the first tool call to the corresponding tool in the execution environment associated with the corresponding tool.
19. The computer-implemented method according to any of the preceding claims, wherein, The machine learning model used to generate the tool path is configured to control operations in the execution environment.
20. The computer-implemented method according to any of the preceding claims, wherein, The execution environment refers to the virtual machine assigned to the task instance, the task instance including the machine learning-processed model used to generate the tool path.
21. The computer-implemented method according to any of the preceding claims, wherein, The execution environment prevents the machine learning model used to generate the tool path from interfering with another task instance.
22. The computer-implemented method according to any of the preceding claims, wherein, The execution environment prevents the corresponding tool from interfering with another tool that is associated with another tool path.
23. The computer-implemented method according to any of the preceding claims, wherein, Each tool path is executed in a separate execution environment.
24. The computer-implemented method according to any of the preceding claims, wherein, The separate execution environment is instantiated by a virtual machine.
25. The computer-implemented method according to any of the preceding claims, wherein, The separate execution environment is instantiated from a separate working directory within the directory tree.
26. The computer-implemented method according to any one of the preceding claims, comprising: Cache resources for use across multiple tool paths.
27. The computer-implemented method according to any one of the preceding claims, comprising: Cache resources for use across multiple execution environments.
28. The computer-implemented method according to any of the preceding claims, wherein: The second node among the one or more nodes is generated in the following manner: Using the machine learning-based model and based on the first tool response, a second tool invocation is generated to perform the second step of the task; and Initiate the execution of the second tool call to obtain the second tool response.
29. The computer-implemented method according to any of the preceding claims, wherein: The plurality of tool paths includes a tool path tree; The second corresponding toolpath includes one or more nodes, wherein the corresponding output status of the second corresponding toolpath is based on the completion node of the second corresponding toolpath; and The first node of the one or more nodes is generated in the following manner: The machine learning model is used to generate a first tool invocation for performing the first step of the task; and Initiate the execution of the first tool call to obtain the first tool response.
30. The computer-implemented method according to any one of the preceding claims, comprising: Multiple intermediate states are evaluated for each of the multiple tool paths, wherein the corresponding intermediate state of each tool path is based on the intermediate nodes of that tool path before the completion node; and Prune at least one of the plurality of tool paths based on the corresponding intermediate state.
31. The computer-implemented method according to any one of the preceding claims, comprising: The at least one tool path is pruned based on the redundancy of at least one tool path.
32. The computer-implemented method according to any of the preceding claims, wherein, The redundancy is determined based on the deterministic results of identifying the at least one tool path.
33. The computer-implemented method according to any of the preceding claims, wherein, The redundancy is determined based on the similarity margin.
34. The computer-implemented method according to any of the preceding claims, wherein, The similarity margin is based on a minimum step size associated with the range of parameter values invoked by the tool.
35. The computer-implemented method according to any one of the preceding claims, comprising: The at least one tool path is adjusted based on the failure conditions identified associated with it.
36. The computer-implemented method according to any of the preceding claims, wherein, The failure status is determined based on the received tool response that deviates from the expected standard.
37. The computer-implemented method according to any of the preceding claims, wherein, The failure status is determined based on the received error message indicating that the tool or tool call is inoperable.
38. The computer-implemented method according to any of the preceding claims, wherein, Adjusting the at least one tool path includes: Prune the at least one tool path.
39. The computer-implemented method according to any of the preceding claims, wherein, Adjusting the at least one tool path includes: Restart the at least one tool path using one or more tool calls newly generated by the machine learning model.
40. The computer-implemented method according to any of the preceding claims, wherein, Restarting at least one tool path includes: A new tool call is generated based on the failure status.
41. The computer-implemented method according to any of the preceding claims, wherein, Generating the new tool call based on the aforementioned failure condition includes: The data describing the failure conditions is input into the machine learning-based model as context for queries used to invoke the new tool.
42. The computer-implemented method according to any one of the preceding claims, comprising: At least one intermediate state is evaluated using the machine learning-equipped model.
43. The computer-implemented method according to any one of the preceding claims, comprising: At least one intermediate state is evaluated using a machine learning model that is different from the machine learning model used to generate the tool call.
44. The computer-implemented method according to any one of the preceding claims, comprising: The machine learning-equipped model is used to evaluate at least one intermediate state to select a heuristic tool to be applied to the at least one intermediate state.
45. The computer-implemented method according to any one of the preceding claims, comprising: Evaluate multiple features of at least one intermediate state.
46. The computer-implemented method according to any one of the preceding claims, comprising: Calculate a consistency metric for the multiple intermediate states; Select a representative tool path based on the consistency metric; as well as Prune one or more other tool paths.
47. The computer-implemented method according to any one of the preceding claims, comprising: A consistency metric is calculated for the multiple intermediate states to identify clusters along the alignment tool path; Representative tool paths are selected based on the consistency metric, wherein the representative tool paths represent clusters of the aligned tool paths; as well as Prune other tool paths in the cluster of aligned tool paths.
48. The computer-implemented method according to any one of the preceding claims, comprising: A consistency metric is calculated for the multiple intermediate states to identify clusters along the alignment tool path; Representative tool paths are selected based on the consistency metric, wherein the representative tool paths represent clusters of the aligned tool paths; Prune other toolpaths in the cluster of aligned toolpaths; and The new tool path is initiated by sampling new tool calls from the machine learning-based model.
49. The computer-implemented method according to any of the preceding claims, wherein, The tool invocation includes text commands configured for input via a command-line interface.
50. The computer-implemented method according to any of the preceding claims, wherein, The tool invocation includes requests to application programming interface endpoints.
51. The computer-implemented method according to any of the preceding claims, wherein, The tool invocation includes requests to application programming interface endpoints over a network.
52. The computer-implemented method according to any one of the preceding claims, comprising: The machine learning-based model generation tool includes one or more lines of programming code that perform operations related to the requested task. as well as The tool is executed in at least one tool path.
53. The computer-implemented method according to any one of the preceding claims, comprising: The machine learning-based code generation model is used to generate tools that include one or more lines of programming code that perform operations related to the requested task. as well as The tool is executed in at least one tool path.
54. The computer-implemented method according to any one of the preceding claims, comprising: Use the request to determine one or more screening criteria for evaluating the intermediate state; as well as The at least one tool path is pruned based on failures relative to the one or more filtering criteria.
55. The computer-implemented method according to any one of the preceding claims, comprising: Evaluate intermediate consistency metrics for one or more tool calls sampled from the machine learning model.
56. The computer-implemented method according to any one of the preceding claims, comprising: Calculate consistency metrics for multiple tool calls.
57. The computer-implemented method according to any one of the preceding claims, comprising: Select one or more tool calls for execution to improve the diversity of tool paths.
58. The computer-implemented method according to any one of the preceding claims, comprising: Cluster the tool calls sampled from the machine learning model; as well as Select a representative tool from each of the first K clusters for execution.
59. The computer-implemented method according to any one of the preceding claims, comprising: Based on the tool call, determine one or more operations to be performed using the graphical user interface; Generate one or more navigation commands configured to interact with the graphical user interface; as well as Initiate the execution of one or more navigation commands.
60. A computer-implemented method, comprising: The client computing system transmits a request to the server computing system to execute a task; as well as The client computing system receives a response from the server computing system based on the execution of the task; The response is obtained using the method according to any one of the preceding claims.
61. The computer-implemented method according to claim 60, wherein: The server computing system performs the task using the method according to any one of the preceding claims.
62. The computer-implemented method according to claim 60 or 61, comprising: The client computing system receives the initial request from the user computing system; as well as The request for performing the task is obtained based on the initial request.
63. One or more non-transitory computer-readable media storing instructions executable by one or more processors to cause a computing system to perform operations, including the methods according to any one or more of the preceding claims.
64. A computing system, comprising: One or more processors; as well as One or more non-transitory computer-readable media storing instructions executable by one or more processors to cause a computing system to perform operations, including the methods described in any one or more of the preceding claims.
