Proxy workflow programming with execution branch search

Abstract

The invention relates to proxy workflow programming with execution branch search. One embodiment of the present invention sets forth a technique for executing a proxy workflow. The technique includes, upon determining that a first branch point is reached during execution of the proxy workflow, executing the proxy workflow based on one or more calls received through an interface associated with the first branch point, a first plurality of branches are executed from the first branch point and a state of the proxy workflow at the first branch point to one or more subsequent branch points within the proxy workflow. The technique further includes executing a second plurality of branches from at least a portion of the one or more subsequent branch points to one or more additional points within the proxy workflow based on a first plurality of evaluation scores associated with executing the first plurality of branches.

Description

[0001] Cross-reference to related applications This application claims priority to U.S. Patent Application No. 19 / 346,151, filed September 30, 2025, entitled “AGENTIC WORKFLOW PROGRAMMING WITH SEARCH OVER EXECUTION BRANCHES”; U.S. Patent Application No. 63 / 702,079, filed October 1, 2024, entitled “META-PROGRAMMING FOR AGENT WORKFLOWS”; and U.S. Provisional Application No. 63 / 751,757, filed January 30, 2025, entitled “SEARCH OVERNONDETERMINISTIC EXECUTION PATHS IN AGENTIC WORKFLOWS”. The subject matter of these related applications is incorporated herein by reference in its entirety. Technical Field

[0002] Embodiments of this disclosure generally relate to artificial intelligence and agent design, and more specifically, to agent workflow programming that utilizes execution branch search. Background Technology

[0003] Artificial intelligence (AI) agents include software systems that use large language models (LLMs) and / or other machine learning techniques to complete complex, multi-step workflows on behalf of users. For example, AI agents may be able to process various types of data, such as (but not limited to) text, images, audio, video, and / or code; converse with users and / or collaborate with other agents or software systems; and / or reason, plan, learn over time, recall, and / or make decisions.

[0004] However, the probabilistic nature of the LLMs and / or other components used in agent workflows involving AI agents can lead to unreliable and / or unpredictable outputs from these workflows. Specifically, the generation of LLMs may vary significantly between calls involving the same input, and / or produce incorrect and / or suboptimal results that propagate into subsequent workflow steps.

[0005] Recently, inference-time search strategies involving the selection and / or improvement of LLM outputs have been developed to enhance the performance and reliability of agent workflows. These strategies include: best-of-N sampling, where multiple LLM outputs are generated and the best response is selected based on a reward pattern and / or a scoring function; self-consistency, where multiple LLM outputs are sampled and aggregated to determine the most consistent output; refinement, where subsequent LLM outputs are modified and / or improved using feedback associated with an initial set of LLM outputs; and / or tree-search techniques that search along an inference chain containing a sequence of LLM inference steps.

[0006] However, traditional implementations of inference-time search strategies are often embedded and entangled with the core logic of the proxy workflow. More specifically, each inference-time strategy can be implemented within the corresponding proxy workflow by applying, compiling, and running different sets of modifications to the source code. This entangled code obscures the proxy workflow logic and lacks modularity, resulting in significant time and resource overhead when configuring, customizing, designing, and / or exploring different inference-time strategies for a given proxy workflow, and / or propagating the same inference-time strategy to multiple proxy workflows.

[0007] Furthermore, these modifications become more difficult to read, extend, debug, and / or maintain due to the addition of optimizations and / or the use of complex search strategies involving explicitly defined proxy workflow search spaces. Therefore, complex inference-time strategies tend to be limited to simple proxy workflows, such as those that repeatedly sample actions from an LLM, while complex proxy workflows tend to be limited to simple inference-time strategies, such as refining and / or resampling individual LLM calls.

[0008] As mentioned above, there is a need in the art for more effective techniques to improve the execution of proxy workflows. Summary of the Invention

[0009] One embodiment of the present invention describes a technique for executing a proxy workflow. The technique includes: upon determining that a first branch point has been reached during the execution of the proxy workflow, executing a first plurality of branches from the first branch point and the state of the proxy workflow at the first branch point to one or more subsequent branch points within the proxy workflow, based on one or more calls received through an interface associated with the first branch point. The technique further includes: executing a second plurality of branches from at least a portion of one or more subsequent branch points to one or more additional points within the proxy workflow, based on a first plurality of evaluation scores associated with the execution of the first plurality of branches.

[0010] One technical advantage of the disclosed technique compared to existing technologies is that it separates (i) the source code, executable code, and / or other logical representation of the proxy workflow from (ii) the corresponding representation of the inference-time search strategy, which can be used to improve the reliability and / or performance of the proxy workflow. Therefore, compared to traditional inference-time search strategy implementations (which apply, compile, and run a different set of source code modifications for each proxy workflow), the disclosed technique reduces the time and resource overhead associated with implementing, modifying, customizing, and / or transferring the inference-time search strategy. Furthermore, the increased speed and efficiency of exploring and / or optimizing the inference-time search strategy can improve the reliability and / or performance of the proxy workflow compared to proxy workflows executed using traditional implementations of the inference-time search strategy. Moreover, since the disclosed technique allows the use of a basic set of statements and calls and / or eliminates the need to explicitly define the search space of the proxy workflow to define the inference-time search strategy, it allows proxy workflows of arbitrary complexity and inference-time search strategies to work together. These technical advantages provide one or more technical improvements over existing methods. Attached Figure Description

[0011] To gain a detailed understanding of the features of the above embodiments, the inventive concept briefly outlined above can be described in more detail with reference to the embodiments, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings only show typical embodiments of the inventive concept and should not be construed as limiting the scope in any way; other equally effective embodiments exist.

[0012] Figure 1 A computing device configured to implement one or more aspects of the various embodiments is shown.

[0013] Figure 2 According to the various embodiments Figure 1 A more detailed illustration of the generation engine and execution engine.

[0014] Figure 3 According to the various embodiments Figure 1 The generation and execution engines compile and execute the example agent workflow.

[0015] Figure 4 This is a flowchart of method steps for generating and executing agent workflows according to various embodiments. Detailed Implementation

[0016] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the various embodiments. However, those skilled in the art will understand that the inventive concept can be practiced even without one or more of these specific details.

[0017] System Overview Figure 1 A computing device 100 configured to implement one or more aspects of various embodiments is shown. In one embodiment, the computing device 100 includes a desktop computer, laptop computer, smartphone, personal digital assistant (PDA), tablet computer, or any other type of computing device configured to receive input, process data, and optionally display images, and is suitable for practicing one or more embodiments. The computing device 100 is configured to run a generation engine 122 and an execution engine 124 residing in memory 116.

[0018] It should be noted that the computing device described herein is exemplary, and any other technically feasible configuration falls within the scope of this disclosure. For example, multiple instances of the generation engine 122 and execution engine 124 may execute on a set of nodes in a distributed and / or cloud computing system to implement the functionality of the computing device 100. In another example, the generation engine 122 and / or execution engine 124 may execute on various hardware, device types, or environments to adapt the generation engine 122 and / or execution engine 124 to different use cases or applications. In a third example, the generation engine 122 and execution engine 124 may execute on different computing devices and / or different sets of computing devices.

[0019] In one embodiment, computing device 100 includes, but is not limited to: interconnect (bus) 112 connecting one or more processors 102, input / output (I / O) device interface 104 coupled to one or more input / output (I / O) devices 108, memory 116, storage device 114, and network interface 106. Processor 102 can be any suitable processor implemented as a central processing unit (CPU), graphics processing unit (GPU), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), artificial intelligence (AI) accelerator, any other type of processing unit, or combination of different processing units, such as a CPU configured to work in conjunction with a GPU. Typically, processor 102 can be any technically feasible hardware unit capable of processing data and / or executing software applications. Furthermore, in the context of this disclosure, the computing element shown in computing device 100 can correspond to a physical computing system (e.g., a system in a data center) or can be a virtual computing instance performed in a computing cloud.

[0020] I / O device 108 includes devices capable of providing input, such as a keyboard, mouse, touchscreen, microphone, etc., and devices capable of providing output, such as a display device or speaker. Furthermore, I / O device 108 may also include devices capable of receiving input and providing output, such as a touchscreen, Universal Serial Bus (USB) port, etc. I / O device 108 can be configured to receive various types of input from end users of computing device 100 (e.g., designers), and can also provide various types of output to end users of computing device 100, such as displayed digital images or digital video or text. In some embodiments, one or more I / O devices 108 are configured to couple computing device 100 to network 110.

[0021] Network 110 is any technically feasible communication network that allows data exchange between computing device 100 and external entities or devices (such as web servers or other networked computing devices). For example, network 110 may include a wide area network (WAN), a local area network (LAN), a wireless (WiFi) network, and / or the Internet.

[0022] Storage device 114 includes non-volatile storage for applications and data, and may include fixed or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD, or other magnetic, optical, or solid-state storage devices. Generation engine 122 and execution engine 124 may be stored in storage device 114 and loaded into memory 116 during execution.

[0023] Memory 116 includes random access memory (RAM) modules, flash memory cells, or any other type of memory cell or combination thereof. Processor 102, I / O device interface 104, and network interface 106 are configured to read data from memory 116 and write data to memory 116. Memory 116 includes various software programs executable by processor 102 and application data associated with said software programs, including generation engine 122 and execution engine 124.

[0024] In one or more embodiments, the generation engine 122 and the execution engine 124 include functionality for programming the proxy workflow using execution branch search, wherein the proxy workflow program is compiled and run in a manner that separates the logic of the proxy workflow from search strategies used to improve the performance and / or reliability of the proxy workflow. The proxy workflow may involve multi-step processes, including invoking large language models (LLMs) and / or other machine learning components that generate output probabilistically, tool usage, inference changes, code generation and / or execution, and / or other operations that may result in variable and / or unreliable output.

[0025] More specifically, the generation engine 122 and the execution engine 124 provide a probabilistic angelic nondeterminism (PAN) framework, which allows the source code of the agent workflow to decouple statements and / or calls related to the inference-time search strategy from the underlying logic of the agent workflow. These statements and / or calls can specify branching points, at which the execution of the agent workflow can branch into multiple execution threads.

[0026] During the execution of the proxy workflow, each branch branching off from the corresponding branch point is executed until a subsequent branch point and / or another termination point specified in the source code is reached. The proxy workflow state at the termination point of each branch and / or the evaluation score associated with that branch is stored, and a search for execution paths consisting of the sequence of branches spanning the proxy workflow is performed based on the corresponding state and / or evaluation score. Search parameters and / or strategies used to perform the search can be additionally specified via calls on interfaces provided by the framework. This process is repeated recursively for each path until the end of the program is reached. The final state and / or output of the proxy workflow can then be determined and / or output based on the target associated with the "final evaluation score" and / or other attributes at the end of each path. The generation engine 122 and execution engine 124 will be described in further detail below.

[0027] Proxy workflow programming using execution branch search Figure 2 According to various embodiments Figure 1 A more detailed illustration of the generation engine 122 and execution engine 124 is provided below. As described above, the generation engine 122 and execution engine 124 contain functionality for performing agent workflow programming that utilizes execution branch search. Each of these components will be described in further detail below.

[0028] The generation engine 122 generates a search space 220 associated with the proxy workflow 206 by compiling a corresponding program 200 that contains source code and / or other representations of the proxy workflow 206. For example... Figure 2 As shown, program 200 includes a set of logic code units 202(1)-202(M) (each unit is referred to as logic code unit 202 in this document) and a set of search code units 204(1)-204(N) (each unit is referred to as search code unit 204 in this document).

[0029] Logic code unit 202 includes a representation of the logic used for proxy workflow 206. For example, each logic code unit 202 may include statements, expressions, conditions, instructions, functions, methods, lines of code and / or another source code "unit", pseudocode and / or another way of defining the behavior of proxy workflow 206.

[0030] Search code unit 204 includes a representation of the inference-time search strategy associated with agent workflow 206. For example, each search code unit 204 may include statements, expressions, conditions, instructions, functions, methods, lines of code and / or another source code "unit", pseudocode and / or another way of defining a strategy for executing, searching, sampling, selecting and / or otherwise using the probabilistic outputs generated by the LLM and other components in agent workflow 206.

[0031] In one or more embodiments, search code unit 204 designates one or more branch points 210(1)-210(Z) of agent workflow 206 (each branch point is referred to herein as branch point 210). Each branch point 210 may correspond to a location in agent workflow 206 that is “unreliable” or “unpredictable.” For example, a given branch point 210 may be located before an LLM call, a sample from a distribution, and / or other parts of agent workflow 206 that involve generating probabilistic, randomized, and / or uncertain outputs. Each branch point 210 may be added to program 200 by a user (e.g., the developer of program 200); added to program 200 by another program (e.g., an LLM, agent, and / or machine learning model that predicts the location of probabilistic outputs in logic code unit 202); and / or added to program 200 based on rules, heuristics, and / or patterns associated with logic code unit 202.

[0032] In some embodiments, generation engine 122 transforms branch point 210 and / or other points in program 200 into nodes 222(1), 222(2)-222(A), 222(A+1)-222(B), 222(C)-222(D) (each node is referred to herein as node 222 individually). For example, generation engine 122 may create a root node 222(1) representing the start of program 200, one or more leaf nodes 222 representing the end of program 200 (e.g., represented by a return statement and / or other indicator indicating the completion of execution of agent workflow 206) and one or more nodes 222 between the root node 222(1) and the leaf nodes 222, the leaf nodes 222 representing one or more branch points 210 specified in program 200.

[0033] The generation engine 122 also connects the node pairs 222 in the search space 220 with edges 224(1)-224(A-1), 224(A)-224(B-1), and 224(E)-224(F) (each edge is referred to as edge 224 in this document), which represent transitions between corresponding points in the proxy workflow 206. For example, each edge 224 may represent a probabilistic transition from state 216(1)-216(Z) (each state is referred to as state 216 in this document) located at the starting point represented by the first node 222 to another state 216 located at the ending point represented by the second node 222, where the edge 224 originates at the first node 222 and terminates at the second node 222. Each state may include a set of variables that map to a set of values ​​at the corresponding point in the procedure 200. Therefore, the search space 220 can correspond to the Markov chain in the space of possible point pairs and corresponding states 216 in program 200.

[0034] Furthermore, edges 224 from parent node 222 in search space 220 to a set of child nodes 222 can indicate that the execution branches of program 200 are multiple execution branches located between a first point and corresponding state 216 represented by parent node 222 and one or more subsequent points and corresponding states 216 represented by child nodes 222. Each execution branch can correspond to a different transition, which is represented by the starting state 216 at the first point and the ending state 216 at the subsequent points.

[0035] In one or more embodiments, generation engine 122 compiles program 200 into search space 220 for searching by converting the execution portion into a continuation-passing style (CPS), where the control flow is explicitly defined in the form op(input, rest), where the operation op is applied to the input and the result is passed to the function rest, which represents the remainder of program 200. Within program 200, branch points 210 and / or control flow structures containing branch points 210 (e.g., loops, conditional statements, jump statements, etc.) can be converted into CPS.

[0036] The example program 200, which has a search code unit 204 with a specified set of branch points 210, includes the following: @encompass.compile def statement_sequence(args):x1 = do_task1(args)branchpoint()x2 = do_task2(x1)branchpoint()x3 = do_task3(x2)return x3 This example program 200 includes the `@encompass.compile` function decorator, which instructs the `statement_sequence` function to be compiled into the corresponding search space 220. Example program 200 also includes a linear control flow with logical code units 202 (e.g., `x1 = do_task1(args)`, `x2 = do_task2(x1)`, `x3 = do_task3(x2)`, etc.), which are interleaved with `branchpoint()` statements specifying the location of corresponding branch points 210 in program 200.

[0037] Example program 200 can be compiled into the corresponding search space 220, and its representation is as follows: def statement_sequence(frame, branchpoint_callback, return_callback): frame["x1"] = do_task1(frame["args"]) def rest(frame, branchpoint_callback, return_callback): frame["x2"] = do_task2(frame["x1"]) def rest(frame, branchpoint_callback, return_callback): frame["x3"] = do_task3(frame["x2"]) return_callback(frame["x3"]) branchpoint_callback(frame, rest)branchpoint_callback(frame, rest) In the above representation, each branchpoint() in program 200 is converted into a branchpoint_callback(frame, rest) and a return_callback(rv). The former defines the operation to be performed when execution reaches the corresponding branch point 210, and the latter defines the operation to be performed when execution at branch point 210 returns. The compiled search space 220 takes as input the initial state 216 frame containing the input arguments for program 200, and the branchpoint_callback and return_callback functions that define the behavior of branch point 210 and the corresponding return statement.

[0038] After program 200 is compiled into search space 220, execution engine 124 executes proxy workflow 206 in such a manner that it searches within search space 220 on possible paths 232 from root node 222 (1) to a set of leaf nodes 222. More specifically, execution engine 124 generates and executes a separate set of one or more branches 208 (1)-208 (Y) between consecutive branch points 210 and / or nodes 222 connected by edges 224 in search space 220, representing other points in program 200 (e.g., the start of program 200, the end of program 200, etc.). Each branch is referred to herein as branch 208.

[0039] Each branch 208 corresponds to a different execution thread that begins at a state 216 at a first branch point 210 (or another point in program 200) and ends at a second branch point 210 (or another point in program 200) and the corresponding state 216. Each branch 208 may also be associated with a corresponding score 214(1)-214(Y) (each score is referred to herein as score 214), which is calculated based on the starting state 216, the ending state 216, and / or other values ​​associated with the branch 208. As described in further detail below, the score 214 associated with branch 208 and / or the search strategy 212 may be used to guide the search of the search space 220 and generate the final output 234 of the proxy workflow 206.

[0040] When branch 208 within the agent workflow 206 is executed, the execution engine 124 populates the search tree 226 corresponding to the search space 220 with the score 214, state 216, and / or other data associated with branch 208. In the search tree 226, nodes represent specific branch points 210 and / or other points in the program 200 and their corresponding states 216. A directed edge between a first node and a second node in the search tree 226 represents an executed branch 208 that transitions from the starting point and state 216 represented by the first node to the ending point and state 216 represented by the second node.

[0041] More specifically, execution engine 124 can initialize search tree 226 with a root node corresponding to the root node 222(1) of search space 220, which represents the starting point and corresponding starting state 216 of proxy workflow 206. For each node in search tree 226, execution engine 124 can generate and execute one or more branches 208 from the starting point and state 216 represented by that node to the ending point and state 216 within proxy workflow 206. After executing a given branch 208, execution engine 124 can add directed edges to search tree 226 from that node to the child nodes representing the ending point and state 216. Execution engine 124 can also store in each node and / or one or more data elements and / or data structures associated with each node the corresponding point in program 200, the state at that point, and / or the numerical score 214 associated with the corresponding branch 208 terminating at that node.

[0042] After the execution of the proxy workflow 206 is completed (e.g., after the search tree 226 has been populated with one or more paths 232 from the start to the end of program 200), the execution engine 124 determines the final output 234 of the proxy workflow 206 based on the state 216, the score 214, and / or other attributes associated with the points on the branches 208 and / or paths 232. For example, the execution engine 124 may choose the final output 234 as the output of the final branch 208 that terminates at the end of program 200 and is associated with the highest score 214, the output that appears most frequently in a set of final branches 208 that terminate at the end of program 200, and / or another state 216 and / or value that is otherwise optimized for one or more objectives associated with the proxy workflow 206.

[0043] Figure 3 Various embodiments are shown. Figure 1 The generation engine 122 and execution engine 124 in the example compile and execute the sample agent workflow 206. For example... Figure 3As shown, the proxy workflow 206 is defined by a corresponding program 200, which contains the `@encompass.compile` function decorator and a function named `solve`. This function contains three `branchpoint()` statements that place the corresponding branch point 210 before the following three steps: writing the equation for solving the problem passed in as a string, solving the equation by iterative transformation, and generating a solution for the problem.

[0044] The program 200 is compiled into an object representing a search space 220 on which various search strategies can be superimposed. A set of paths 232 can be generated using a given search strategy, exploring various branch sets 208 between consecutive points in the proxy workflow 206 (e.g., the start point of program 200, the three branch points 210, and the end point of program 200) and their corresponding states 216. The final output 234 of the proxy workflow 206 can then be selected from the given paths based on the score 214, the state 216, and / or other attributes associated with the branches 208 and / or paths 232 generated by the search strategy.

[0045] Back Figure 2 In some embodiments, execution engine 124 executes proxy workflow 206 based on a given search strategy 212 associated with search space 220. Search strategy 212 may include search strategies (e.g., depth-first search, breadth-first search, best-first search, bundle search, Monte Carlo tree search (MCTS), etc.) to be applied to branches 208 of proxy workflow 206. Search strategy 212 may also include, or alternatively include, a set of search parameters associated with the search strategy, such as (but not limited to) branch factor, bundle width, optimization objective (e.g., based on state 216 and / or score 214 associated with a branch), and / or other attributes that control and / or influence the selection 218 of one or more branch points 210 and corresponding states 216 to which a given set of branches 208 terminates as the starting point for the next set of branches 208 to be executed in proxy workflow 206.

[0046] The execution engine 124 additionally or alternatively executes the proxy workflow 206 based on a set of calls 230(1)-230(X) (each call is referred to individually as call 230 herein) received through an interface 228 associated with the search space 220 (e.g., after the program 200 has been compiled into the search space 220). Each call 230 may be used to execute and / or control the search strategy used to execute the proxy workflow 206 during inference.

[0047] In one or more embodiments, search code unit 204 includes statements and / or primitives associated with a reasoning-time search strategy involving branch point 210. For example, search code unit 204 associated with Python program 200 may include the following core primitives: • branchpoint(**branchpoint_params): This statement marks the location of a given branch point 210 in procedure 200. A set of branchpoint_params can be used with this statement to provide search parameters related to an external search strategy, such as (but not limited to) branch factors, bundle widths, and / or the target to be optimized.

[0048] • record_score(score): This statement records a numerical score 214 that can be used to guide the search process in various search strategies, such as heuristics in best-first search and value functions in MCTS. The final score 214 (e.g., the last score 214 recorded before execution returns) can correspond to and / or be used to compute the final evaluation score 214 as the goal of a given search strategy.

[0049] When a branchpoint statement is combined with correct verification from a record_score statement (see below), the execution of proxy workflow 206 may produce the effect that the corresponding program, which had already executed without a branchpoint, now tends to produce more desirable outputs by following the random operations of the original branchpoint, and unreliable operations (e.g., LLM calls) become more reliable. This effect, a form of proxy uncertainty, is achieved by searching for nondeterministic branches 208 during the execution of program 200. More specifically, when the execution of program 200 reaches a given branchpoint 210, execution engine 124 branches program 200 into multiple copies of itself and searches for branches 208 of program 200 using external search strategy 212.

[0050] After a given function (or another part of program 200) is decorated with search code unit 204, a search can be performed on the nondeterministic execution branch 208 and / or path 232 via search call 230 on search space interface 228: my_func(...).search(policy, **search_config) In the above call, policy is a string that specifies the search strategy to be used in the search, and search_config contains the search parameters to be used in the search.

[0051] Examples of usage of the above core primitives include the following: @encompass.compile def branchpoint_example(...): branchpoint() ... # Perform operation record_score(...) # Sample 10 times and output the result with the highest score branchpoint_example(...).search("dfs", default_branching=10) In the example above, a branchpoint statement was added at the top of the function body, thus achieving optimal sampling in N.

[0052] The search code unit 204 associated with Python program 200 may also, or alternatively, contain the `branchpoint_choose(choices:Iterable,**branchpoint_params)` primitive, a variant of `branchpoint`, where branch 208 evaluates to elements in `Iterable`, thus leading to regular proxy uncertainty. An example use of this primitive includes a function that guesses the path from the starting node to the target in the graph: @encompass.compile def graph_search(graph, start_node, goal): """ In a graph represented as an adjacency list, guess the path from the start node to the goal.

[0053] """ cur_node = start_node path = [cur_node] cost_so_far while cur_node != goal: next_node = branchpoint_choose(graph[cur_node], identity=cur_node) path = path + [cur_node] cost_so_far += get_edge_cost(cur_node, next_node) total_estimated_cost cost_so_far+estimate_cost_to_go(next_node, goal) record_score(-total_estimated_cost) cur_node = next_node return path # Perform best-priority search -> Use A* search for the shortest path graph_search(my_graph, my_start_node, my_goal).search("best_first",top_k_popped=1, default_branching=None) In the example above, searching for nondeterministic execution branches is equivalent to searching the graph.

[0054] The search code unit 204 associated with the Python program 200 may also, or alternatively, contain the `early_stop_search()` primitive to stop the external search process early (e.g., because the correct and / or optimal final output 234 has been found). An example use of this primitive includes a function that stops the search after a success condition has been verified: @encompass.compile def early_stop_search_example(...): ... # Perform some operations before ... branchpoint() # Request LLM to generate answer answer = llm.generate(...) # Check the answer success = check_answer(answer) If successful: early_stop_search() return answer The search code unit 204 associated with the Python program 200 may also, or alternatively, contain the `delete_branch(err=None)` primitive, which is used to delete the current branch 208 of program execution. For example, this primitive can be used to stop the search based on the current branch 208 after an LLM call generates irreversible bad output. @encompass.compile def delete_branch_example(...): ... # Pre-execution operations branchpoint() # Request LLM to perform operation response = llm.generate(...) sanity_check_passed = sanity_check_llm_response(response) if not sanity_check_passed: delete_branch() ... # Post-execution operations The search code unit 204 associated with Python program 200 may also, or alternatively, contain the primitive `optional_return(return_value)`, which signals to the external search process that although the execution of program 200 has not yet been completed, the output `return_value` has been produced and should be considered as a possible final output 234 of program 200. @encompass.compile def optional_return_example(...): answer = llm.generate_answer(...) optional_return(answer) refined_answer = llm.refine_answer(answer, . . . ) return refined_answer The search code unit 204 associated with the Python program 200 may also, or alternatively, contain the `protect(expr, exception)` primitive, which prevents an expression that would otherwise throw an exception from throwing one. This is achieved by resampling from the most recent branch point 210 until the evaluation of the expression no longer throws an exception. Example use of this primitive might include parsing the output of an LLM, which can be unreliable and / or lead to errors: @encompass.compile def parse_llm_output_example(...): ... # Pre-execution operations branchpoint() # Request LLM to generate Python code response = llm.generate(...) # Extract Python code python_code = protect(response.split("'"'python\n",1)[1] .split("6c'",1)[0], IndexError) # Parsing Python code python_ast = protect(ast.parse(python_code), SyntaxError) ... # Post-execution operations The search code unit 204 associated with the Python program 200 may also, or alternatively, contain the primitive record_costs(cost_name1=cost_amount1, cost_name2=cost_amount2,...), which can be used to track costs and / or other types of overhead or resource consumption associated with LLM usage. @encompass.compile def record_costs_example(...): response, cost = llm.generate(...) record_costs(llm_cost=cost, llm_num_calls=1) return response The recorded costs can be aggregated and accessed through the dictionary func.aggregate_costs (described in detail below), where func is the compiled function.

[0055] The search code unit 204 associated with the Python program 200 may also, or alternatively, contain the primitive `record_score(multiple_evaluator, eval_target, label=eval_label)`, which is an overloading of `record_score`, enabling the calculation of a score 214 based on comparisons of states 216 across multiple execution branches 208 of program 200. One use case for this primitive involves self-consistent majority voting, where the evaluation of a result is performed relative to all results.

[0056] The search code unit 204 associated with Python program 200 may also, or alternatively, contain the `searchover(func(*args, **kwargs))` primitive for calling the main branch `func` in another branch: @encompass.compile def helper_function(...): ... # Perform an operation @encompass.compile def searchover_example(...): ... # Before performing an operation # Check the answer helper_result = searchover(helper_function(...)) ... # Post-execution operations This is similar to calling an asynchronous function from within an asynchronous function using the await func(...) syntax.

[0057] In some embodiments, the interface 228 associated with the search space 220 includes the following call 230: • `func(*args, **kwargs).start()`: This will begin execution of the function with the given arguments until the first branch point is reached (e.g., `branchpoint()` or `branchpoint_choose()`), which can be located within a nested `searchover()` function call. The corresponding state 216 at this point is wrapped in a `Checkpoint` object, which can be used to step through the execution function and / or generate a pair of states 216 and branch point 210.

[0058] •func(*args, **kwargs).search(search_policy, **search_params) -> Any: This will search the compiled search space 220 using the specified search policy 212 and return the corresponding final output 234, which may contain the return value from a branch 208 (from returning_value or optional_return(return_value)), which terminates at the end point of program 200 with the highest record score 214.

[0059] • func.aggregate_costs: This is a dictionary containing the total cost from all record_cost statements. The key cost_name maps to the sum of the costs of all records with that name: record_cost(cost_name=...).

[0060] • `func.branchpoint_step_counts`: This is a dictionary that maps the name of a branch point to the number of times `step()` (described in detail below) is called on that branch point's checkpoint. This count includes all calls to `func` since the last call to `zero_branchpoint_counts()` (see below). The dictionary will contain the step counts for branch point 210 with the name parameter (e.g., `branchpoint(name=...)` or `branchpoint_choose(choices,name=...)`).

[0061] •func.zero_branchpoint_counts(): This is a dictionary used to zero out the total step count for each named branch point 210.

[0062] In some embodiments, the given state 216 at the corresponding branch point 210 is stored in a Checkpoint object, which supports the following interface 228 calls 230: • `Checkpoint.step(max_protection=None, score_db_flush_queue=True) -> Checkpoint`: This continues execution of program 200 from the stored state 216 at the given branch point 210 until the next branch point 210 is reached, returning a new Checkpoint object. Any expression protected by `protect(expr, exception)` will trigger resampling whenever an exception occurs, up to a maximum of `max_protection` times. If `score_db_flush_queue` is set to `False`, pending evaluations recorded via the multivariate evaluation version of `record_score` will not be processed. Multiple calls to `step()` at the same Checkpoint are mostly independent: while variable assignments are independent, variable references are shared, so mutations to objects created before the current checkpoint will be seen by all execution branches 208 derived from that checkpoint. If the given branch point 210 is branchpoint_choose(choices:Iterable) instead of a regular branchpoint() statement, then multiple step() calls will iterate through the options (choices), and the resulting branch 208 will evaluate the elements in the choices.

[0063] • `Checkpoint.step_sampler(max_samples=None, max_protection=None, score_db_flush_queue=True) -> Generator[Checkpoint, None, None]`: This repeatedly calls `step()` and generates the resulting Checkpoint objects. This operation is performed at most `max_samples` times, and / or until the list of `choices` in `branchpoint_choose` is exhausted. Additionally, `max_protection` specifies the total number of resamples allowed for the evaluation of protected expressions.

[0064] •Checkpoint.parallel_step_sampler(max_samples=None, chunk_size=None, max_protection=None, max_workers=None, score_db_flush_queue=True) ->Generator[Checkpoint, None, None]: This is a multi-threaded version of Checkpoint.step_sampler(), where max_workers specifies the maximum number of threads to use, and if chunk_size is given, parallel sampling is performed in batches of that size.

[0065] • Checkpoint.status: Status: The status of the checkpoint object. Status can be one of Status.RUNNING, Status.DONE_STEPPING, Status.RETURNED, and Status.KILLED.

[0066] •Checkpoint.has_return_value: bool: Determines whether return return_value (if the checkpoint is at a return statement) or optional_return(return_value) (if the checkpoint is at the given branch point 210) has a return value.

[0067] •Checkpoint.return_value: Any: The return value of this function (if it exists) (i.e., if the checkpoint is at a return statement, or at a branch point after an optional_return statement and without intermediate branch points 210).

[0068] •Checkpoint.early_stopped_search: bool: Whether the early_stopped_search() statement was called on any branch 208 of the execution of program 200.

[0069] •Checkpoint.score: The most recent record score obtained through record_score, 214.

[0070] •Checkpoint.branchpoint_params: dict: This is a dictionary containing the parameters for a given branch point 210 specified by branchpoint(**branchpoint_params) branchpoint_choose(choices, **branchpoint_params)).

[0071] In one or more embodiments, converting a function (or another part of program 200) to CPS includes an initial syntax check to verify that the search code unit 204 is used correctly; then an abstract syntax tree (AST) preprocessing is performed to transform variable access into explicit access to a dictionary storing the corresponding state 216 (e.g., x → frame["x"]); primitives and / or other search code units 204 are marked; and nested calls to the search code unit 204 as a function are extended to directly assign the function to temporary variables (e.g., ...). Then, by calling the callback function for branch point 210 using the current state 216 frame and the continuation frame, along with additional arguments to convey further information, the AST of branch point 210 is transformed into CPS style. Similarly, the return statement is transformed into a call to the return callback function.

[0072] Subsequently, to create checkpoint interface 228, the branch point callback registers the current state 216 frame and the rest function containing the remainder of program 200's execution, and returns a checkpoint object containing that state and the rest function. A simplified implementation of checkpoint interface 228 (supporting only branch point 210 as a primitive) might include the following: class Checkpoint: ... # Omit constructor def step(self): new_checkpoint = Checkpoint() def branchpoint_callback(frame, rest): new_checkpoint._frame = frame new_checkpoint._rest = rest def return_callback(rv): new_checkpoint._rv = rv self._rest( self._frame.copy(), branchpoint_callback, return_callback, ) return new_checkpoint In the above implementation of checkpoint interface 228, the frame object is copied to prevent different branches 208 from mutating the same frame object. Most other primitives provide auxiliary information as additional arguments in the callback function. The delete_branch() function requires that the return callback be split into two callback functions: return_callback is called when the given function containing search code unit 204 returns to another function containing search code unit 204; finish_callback is called when the top-level function containing search code unit 204 returns, and / or when delete_branch() is called in the top-level function or in a call to other functions containing search code unit 204.

[0073] As described above, the search code unit 204 can be placed in program 200 to achieve best-in-N sampling at a global level (e.g., during the execution of agent workflow 206). This can be extended to local best-in-N, where the multi-step agent workflow 206 applies best-in-N sampling to each step. This can be achieved by adding a different branch point 210 before each step and performing a bundle search with a bundle width of 1: @encompass.compile def agent_forward(...): branchpoint() ... # Step 1 record_score(evaluate_step1(...)) branchpoint() ... # Step 2 record_score(evaluate_step2(... )) branchpoint() ... # Step k record_score(evaluate_stepk(...)) return stepk_result N = ... # The "N" in "N's best" result = agent_forward(...).search("beam", beam_width=1, default_branching=N) Since the optimal sampling in the global N corresponds to a beam search with a beam width of N and a branch factor of 1, while the optimal sampling in the local N corresponds to a beam search with a beam width of 1 and a branch factor of N, the general beam search can be viewed as interpolation between global resampling and local resampling. This interpolation method effectively restricts the search space 220 and performs local verification without sacrificing global diversity. Increasing the branch factor ensures that each step is completed correctly, avoiding the accumulation of errors; while increasing the beam width increases diversity and / or improves reliability, thereby reducing errors that may occur in previous steps.

[0074] Search code unit 204 can also be used to implement refinement, that is, to sample using additional feedback from past sampling attempts. This can be achieved by generating multiple samples by adding a given branch point 210 and sharing the memory of past attempts among sampling branches 208: @encompass.compile def agent_forward(...): ... # Previous code # Detailed steps feedbacks = [] branchpoint(branching=n_refine +1) result = do_step(..., feedbacks) score, feedback = get_score_and_feedback(result) feedbacks.append(feedback) record_score(score) ... # Code after result = agent_forward(...).search("beam", beam_width=1) More specifically, different branches 208 share the same reference to the feedbacks variable, so all branches 208 can see the additional feedback.

[0075] Another branch point 210 can be added immediately before the feedbacks variable defined above to create multiple parallel refinement loops, thereby interpolating between fresh sampling and refinement and increasing the diversity of the proxy workflow that focuses too much on past feedback. Furthermore, backtracking with memory can be performed by creating a mutable data structure before a given branch point 210, i.e., using the memory of previous attempts to perform backtracking to the previous step (e.g., branch point 210), whereby this data structure acts as a shared memory across all subsequent branches 208.

[0076] Furthermore, search code unit 204 can be used to achieve self-consistency, i.e., sampling N times and selecting the output that appears most frequently (e.g., majority vote), by performing the best sample among N using an evaluation function that evaluates multiple results simultaneously. This can be achieved using the overloaded record_score() primitive: def majority_vote(results): counts = defaultdict(int) For result in results: counts[result] += 1 return [ counts[result] for result in results ] @encompass.compile def agent_forward(...): branchpoint() record_score(majority_vote, result, label=None) return result result = agent_forward(...).search(...) Typically, when it's difficult to evaluate a single result individually, it can be helpful to allow the evaluation function to evaluate multiple results at once. Another example involves evaluating multiple LLM-generated code samples against multiple LLM-generated unit test cases based on the number of unit tests passed and the consistency between the code samples that passed the tests.

[0077] Figure 4 This is a flowchart of method steps for generating and executing an agent workflow according to various embodiments. Although combined... Figures 1 to 2The system described herein is a method step, but those skilled in the art should understand that any system configured to execute the method steps in any order is within the scope of this disclosure.

[0078] As shown in the figure, in step 402, the generation engine 122 compiles the proxy workflow program into a search space associated with a set of points specified in the program. For example, the generation engine 122 can convert primitives, statements, and / or other search code units in the program's source code into CPS.

[0079] In step 404, execution engine 124 executes branches of the proxy workflow from one point to subsequent points. For example, execution engine 124 may first execute a branch that starts at the beginning of the program and ends at the first branch point in the program. Execution engine 124 may also initialize a search tree associated with the search space, which has a root node representing the beginning of the program and the state associated with the beginning of the program.

[0080] In step 406, execution engine 124 stores the nodes representing subsequent points, the status of the proxy workflow at the subsequent points, and / or the evaluation scores associated with the execution of the proxy workflow up to the subsequent points in a search tree associated with the search space. For example, execution engine 124 may add child nodes to the search tree that represent the end point and status of an executed branch, as well as any scores recorded during the execution of that branch.

[0081] In step 408, execution engine 124 determines whether to perform an additional branch between the point and subsequent points. This determination may be based on a branching factor that specifies the number of branches to explore from a given starting point and / or other search parameters that control the exploration of the search space.

[0082] If execution engine 124 determines that an additional branch should be executed, execution engine 124 repeats step 404 to execute another branch from the same starting point and state to the subsequent point. Execution engine 124 also repeats step 408 to determine whether to execute the additional branch.

[0083] When execution engine 124 determines that no additional branch needs to be executed from the current point, execution engine 124 executes step 410 to determine whether to continue executing the proxy workflow. For example, when the search space contains unexplored nodes, the end of the program has not been reached, and / or other termination conditions have not been met (e.g., the maximum number of branches and / or paths explored, the time limit has been reached, etc.), execution engine 124 may determine that the execution of the proxy workflow will continue.

[0084] If execution engine 124 determines that the execution of the proxy workflow will continue, execution engine 124 executes step 412, advancing the proxy workflow to one or more subsequent points. For example, execution engine 124 may select one or more subsequent points from search tree 214 that have the highest score 214, and / or select one or more subsequent points from search tree 214 based on the search strategy associated with the proxy workflow and / or other criteria specified in the search parameters. Execution engine 124 may then recursively repeat steps 404, 406, 408, and 410 to execute additional branch sets from each selected point to one or more additional subsequent points in the program, determine whether to continue executing the proxy workflow, and / or advance the proxy workflow to some or all of the additional subsequent points.

[0085] If execution engine 124 determines in step 410 to stop the execution of the proxy workflow, then execution engine 124 executes step 414 to return the final output of the proxy workflow based on the evaluation score associated with the path formed by the branches between points in the program. For example, execution engine 124 may select the final output from the branch outputs that terminate at the end of the program based on the relevant score, status, and / or other optimization criteria or objectives specified in the search strategy and / or search parameters.

[0086] In summary, the disclosed technical implementation utilizes proxy workflow programming that performs branch search, wherein the program for the proxy workflow is compiled and run in a manner that separates the logic for the proxy workflow from the search strategy used to improve the performance and / or reliability of the proxy workflow. The proxy workflow may involve a multi-step process that includes invoking a large language model (LLM) and / or other machine learning components that generate output probabilistically, tool usage, inference changes, code generation and / or execution, and / or other operations that may result in variable and / or unreliable output.

[0087] More specifically, the disclosed technology provides a probabilistic proxy nondeterminism (PAN) framework that allows the source code used for proxy workflows to decouple statements and / or calls related to the inference-time search strategy from the underlying logic of the proxy workflow. These statements and / or calls can specify branching points where the execution of the proxy workflow can branch into multiple execution "threads".

[0088] During the execution of the proxy workflow, each branch that branches off from its corresponding point is executed until a subsequent branch point and / or another termination point specified in the source code is reached. The state and / or evaluation score associated with each branch at the termination point of the proxy workflow are stored, and a search for an execution path consisting of the sequence of branches across the proxy workflow is performed based on the corresponding state and / or evaluation score. The search parameters and / or strategies used to perform the search can also be specified via calls on an interface provided by the framework. This process is repeated recursively for each path until the end of the program is reached. The final state and / or output of the proxy workflow can then be determined and / or output based on the target associated with the "final evaluation score" and / or other attributes at the end of each path.

[0089] Compared to existing technologies, one technical advantage of the disclosed technology lies in separating (i) the source code, executable code, and / or other logical representations used for the proxy workflow from (ii) the corresponding representation of the inference-time search strategy that can be used to improve the reliability and / or performance of the proxy workflow. Therefore, compared to conventional inference-time search strategy implementations that apply, compile, and run different sets of modifications to the source code for each proxy workflow, the disclosed technology reduces the time and resource overhead associated with implementing, modifying, customizing, and / or transferring the inference-time search strategy. Furthermore, the improved speed and efficiency of exploring and / or optimizing the inference-time search strategy can improve the reliability and / or performance of the proxy workflow compared to proxy workflows executed using conventional implementations of inference-time search strategies. Moreover, since the disclosed technology allows the inference-time search strategy to be defined using a set of basic statements and calls and / or without explicitly defining the search space of the proxy workflow, the disclosed technology allows proxy workflows and inference-time search strategies of arbitrary complexity to be used in conjunction with each other. These technical advantages provide one or more technical improvements over existing methods.

[0090] 1. In some embodiments, a computer-implemented method for executing a proxy workflow includes: after determining that a first branch point has been reached during the execution of the proxy workflow, executing a first plurality of branches from the first branch point and the state of the proxy workflow at the first branch point to one or more subsequent branch points within the proxy workflow based on one or more calls received through an interface associated with the first branch point; and executing a second plurality of branches from at least a portion of the one or more subsequent branch points to one or more additional points within the proxy workflow based on a first plurality of evaluation scores associated with the execution of the first plurality of branches.

[0091] 2. The computer-implemented method according to Clause 1 further includes: determining the final output of the agent workflow based on a second plurality of evaluation scores associated with the execution of the second plurality of branches.

[0092] 3. A computer-implemented method according to any one of Clauses 1-2, wherein determining the final output comprises: determining that the point to which the branch included in the second plurality of branches terminates corresponds to an endpoint in the proxy workflow; and determining the final output based on the subsequent state of the proxy workflow at that point.

[0093] 4. The computer-implemented method according to any one of clauses 1-3, further comprising: compiling the program of the agent workflow into a search space containing the first branch point, the one or more subsequent branch points, and the one or more additional points, wherein the first plurality of branches and the second plurality of branches are executed based on a search of the search space.

[0094] 5. A computer-implemented method according to any one of clauses 1-4, wherein the program for the agent workflow specifies: a first set of locations for the first branch point, the one or more subsequent branch points, and the one or more additional points; and a second set of locations recording the first plurality of evaluation scores.

[0095] 6. The computer-implemented method according to any one of clauses 1-5, wherein a search of the search space is performed based on a search strategy associated with the agent workflow.

[0096] 7. A computer-implemented method according to any one of clauses 1-6, wherein the one or more calls are associated with at least one of: searching the first plurality of branches and the second plurality of branches; a set of costs associated with the first plurality of branches and the second plurality of branches; and the number of branches contained in the first plurality of branches.

[0097] 8. The computer-implemented method according to any one of clauses 1-7, further comprising: storing in a search tree associated with the agent workflow: a first node representing the first branch point and a first state of the agent workflow at the first branch point; and a plurality of nodes as child nodes of the first node, wherein each of the plurality of nodes represents (i) a branch point included in the one or more subsequent branch points, and (ii) the state of the agent workflow after a corresponding execution branch included in the first plurality of branches terminates at the branch point.

[0098] 9. The computer-implemented method according to any one of clauses 1-8, wherein the first plurality of branches are performed based on at least one of a branch factor associated with the first branch point or a bundle width associated with the first branch point.

[0099] 10. The computer-implemented method according to any one of clauses 1-9, wherein the state of the proxy workflow includes at least one of the following: a set of variables, a set of values ​​of the set of variables, or the final output of the proxy workflow.

[0100] 11. In some embodiments, one or more non-transitory computer-readable media store instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: after determining that a first branch point has been reached during the execution of a proxy workflow, based on one or more calls received through an interface associated with the first branch point, execute a first plurality of branches from the first branch point and the state of the proxy workflow at the first branch point to one or more subsequent branch points within the proxy workflow; and based on a first plurality of evaluation scores associated with the execution of the first plurality of branches, execute a second plurality of branches from at least a portion of the one or more subsequent branch points to one or more additional points within the proxy workflow.

[0101] 12. One or more non-transitory computer-readable media as described in Clause 11, wherein the instructions further cause the one or more processors to perform the step of determining the final output of the agent workflow based on a second plurality of evaluation scores associated with the execution of the second plurality of branches.

[0102] 13. One or more non-transitory computer-readable media according to any one of clauses 11-12, wherein determining the final output comprises: determining that a second branch point to which an execution branch included in the second plurality of branches terminates (i) corresponds to an endpoint in the proxy workflow, and (ii) is associated with the highest score included in the second plurality of evaluation scores; and determining the final output based on a third state of the proxy workflow at the second branch point.

[0103] 14. One or more non-transitory computer-readable media according to any one of clauses 11-13, wherein executing the first plurality of branches comprises: generating a plurality of execution threads corresponding to the first plurality of branches based on the state of the agent workflow at the first branch point; and executing each of the plurality of execution threads until a corresponding branch point included in the one or more subsequent branch points is reached.

[0104] 15. One or more non-transitory computer-readable media according to any one of clauses 11-14, wherein executing the first plurality of branches comprises: performing a search on a search space containing the first branch point and the one or more subsequent branch points based on one or more search parameters associated with the first branch point.

[0105] 16. One or more non-transitory computer-readable media according to any one of clauses 11-15, wherein the one or more search parameters include at least one of a branching factor or a bundle width.

[0106] 17. One or more non-transitory computer-readable media according to any one of clauses 11-16, wherein each execution branch included in the first plurality of branches includes a call to a language model.

[0107] 18. One or more non-transitory computer-readable media according to any one of clauses 11-17, wherein the one or more calls are associated with at least one of: executing one or more branches included in the first plurality of branches; determining the state of the agent workflow at the first branch point; or searching the first plurality of branches.

[0108] 19. One or more non-transitory computer-readable media according to any one of clauses 11-18, wherein the first plurality of evaluation scores are calculated based on at least one of a target associated with the agent workflow, a reinforcement learning reward, or the final output of the agent workflow.

[0109] 20. In some embodiments, a system includes: one or more memories storing instructions, and one or more processors coupled to the memories, the processors being configured, when executing the instructions, to perform the following steps: after determining that a first branch point has been reached during the execution of a proxy workflow, executing a first plurality of branches from the first branch point and the state of the proxy workflow at the first branch point to one or more subsequent branch points within the proxy workflow based on one or more calls received through an interface associated with the first branch point; and executing a second plurality of branches from at least a portion of the one or more subsequent branch points to one or more additional points within the proxy workflow based on a first plurality of evaluation scores associated with the execution of the first plurality of branches.

[0110] Any and all combinations of any claim element set forth in any claim and / or any element described in this application, in any manner, fall within the intended scope of this invention and protection.

[0111] The various embodiments described are for illustrative purposes and are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

[0112] Various aspects of the embodiments of this invention can be embodied as a system, method, or computer program product. Therefore, aspects of this disclosure can take the form of a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, collectively referred to herein as a "module," "system," or "computer." Furthermore, any hardware and / or software technology, process, function, component, engine, module, or system described in this disclosure can be implemented as a circuit or a set of circuits. Additionally, aspects of this disclosure can take the form of a computer program product contained in one or more computer-readable media that contain computer-readable program code.

[0113] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example (but not limited to), an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (not an exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that can contain or store a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0114] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to create a machine. When these instructions are executed by a processor of a computer or other programmable data processing apparatus, the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams can be implemented. Such processors can be (but are not limited to) general-purpose processors, special-purpose processors, application-specific processors, or field-programmable gate arrays.

[0115] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, code segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions shown in the blocks may differ from the order shown in the figures. For example, two consecutively shown blocks may actually execute substantially simultaneously, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware system or a combination of dedicated hardware and computer instructions that performs the specified function or action.

[0116] While the foregoing describes embodiments of this disclosure, other and further embodiments of this disclosure may be devised without departing from its basic scope, the scope of which is defined by the appended claims.

Claims

1. A computer-implemented method for executing a proxy workflow, the method comprising: After determining that a first branch point has been reached during the execution of the proxy workflow, based on one or more calls received through the interface associated with the first branch point, a first plurality of branches are executed from the first branch point and the state of the proxy workflow at the first branch point to one or more subsequent branch points within the proxy workflow. as well as Based on the first plurality of evaluation scores associated with the execution of the first plurality of branches, execute a second plurality of branches from at least a portion of the one or more subsequent branch points to one or more additional points within the agent workflow.

2. The computer-implemented method according to claim 1 further includes: The final output of the agent workflow is determined based on the second plurality of evaluation scores associated with the execution of the second plurality of branches.

3. The computer-implemented method according to claim 2, wherein, Determining the final output includes: Determine that the points where the branches contained in the second plurality of branches terminate correspond to endpoints in the proxy workflow; and The final output is determined based on the subsequent state of the agent workflow at that point.

4. The computer-implemented method according to claim 1 further includes: The program of the agent workflow is compiled into a search space containing the first branch point, the one or more subsequent branch points, and the one or more additional points, wherein the first plurality of branches and the second plurality of branches are executed based on the search of the search space.

5. The computer-implemented method according to claim 4, wherein, The procedure used for the agent workflow specifies: The first set of locations for the first branch point, the one or more subsequent branch points, and the one or more additional points; and The second set of positions is recorded with the first plurality of evaluation scores.

6. The computer-implemented method according to claim 4, wherein, The search of the search space is performed based on the search strategy associated with the agent workflow.

7. The computer-implemented method according to claim 1, wherein, The one or more calls are associated with at least one of the following: Search the first plurality of branches and the second plurality of branches; A set of costs associated with the first plurality of branches and the second plurality of branches; and The number of branches contained in the first plurality of branches.

8. The computer-implemented method according to claim 1, further comprising: Stored in the search tree associated with the agent workflow: The first node represents the first branch point and the first state of the proxy workflow at the first branch point; as well as Multiple nodes that are child nodes of the first node, wherein each of the multiple nodes represents (i) a branch point included in the one or more subsequent branch points, and (ii) the state of the agent workflow after the corresponding execution branch included in the first multiple branches has terminated at the branch point.

9. The computer-implemented method according to claim 1, wherein, The first plurality of branches are performed based on at least one of a branching factor associated with the first branch point or a bundle width associated with the first branch point.

10. The computer-implemented method according to claim 1, wherein, The state of the proxy workflow includes at least one of the following: a set of variables, a set of values ​​for the set of variables, or the final output of the proxy workflow.

11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: After determining that a first branch point has been reached during the execution of the proxy workflow, based on one or more calls received through the interface associated with the first branch point, a first plurality of branches are executed from the first branch point and the state of the proxy workflow at the first branch point to one or more subsequent branch points within the proxy workflow; and Based on the first plurality of evaluation scores associated with the execution of the first plurality of branches, execute a second plurality of branches from at least a portion of the one or more subsequent branch points to one or more additional points within the agent workflow.

12. One or more non-transitory computer-readable media according to claim 11, wherein, The instructions also cause the one or more processors to perform the following steps: determining the final output of the agent workflow based on a second plurality of evaluation scores associated with the execution of the second plurality of branches.

13. One or more non-transitory computer-readable media according to claim 12, wherein, Determining the final output includes: The second branch point (i) to which the execution branch contained in the second plurality of branches terminates corresponds to an endpoint in the agent workflow, and (ii) is associated with the highest score contained in the second plurality of evaluation scores; and The final output is determined based on the third state of the agent workflow at the second branch point.

14. One or more non-transitory computer-readable media according to claim 11, wherein, Executing the first plurality of branches includes: Based on the state of the proxy workflow at the first branch point, multiple execution threads corresponding to the first plurality of branches are generated; and Each of the plurality of execution threads is executed until the corresponding branch point contained in one or more subsequent branch points is reached.

15. One or more non-transitory computer-readable media according to claim 11, wherein, Executing the first plurality of branches includes: performing a search on a search space containing the first branch point and the one or more subsequent branch points based on one or more search parameters associated with the first branch point.

16. One or more non-transitory computer-readable media according to claim 15, wherein, The one or more search parameters include at least one of the branching factor or the bundle width.

17. One or more non-transitory computer-readable media according to claim 11, wherein, Each execution branch contained in the first plurality of branches includes a call to the language model.

18. One or more non-transitory computer-readable media according to claim 11, wherein, The one or more calls are associated with at least one of the following: Execute one or more branches contained in the first plurality of branches; Determine the state of the proxy workflow at the first branch point; or Search the first plurality of branches.

19. One or more non-transitory computer-readable media according to claim 11, wherein, The first plurality of evaluation scores are calculated based on at least one of the objectives associated with the agent workflow, reinforcement learning rewards, or the final output of the agent workflow.

20. A system comprising: One or more memories, wherein the one or more memories store instructions, and One or more processors, said one or more processors coupled to said one or more memories, said processors being configured to perform the following steps when executing said instructions: After determining that the first branch point has been reached during the execution of the proxy workflow, based on one or more calls received through the interface associated with the first branch point, the first plurality of branches are executed from the first branch point and the state of the proxy workflow at the first branch point to one or more subsequent branch points within the proxy workflow. as well as Based on the first plurality of evaluation scores associated with the execution of the first plurality of branches, execute a second plurality of branches from at least a portion of the one or more subsequent branch points to one or more additional points within the agent workflow.

Images