Multi agent task planning

Abstract

A computer-implemented, machine learning method for guided programming of autonomous agents using a meta agent includes generating a first task that includes a start tag, script agents involved in the first task, and a timer associated with the first task or the script agents. A prompt is generated for each script agent of the script agents associated with the first task that includes the timer, wherein each script agent is configured to generate a script that corresponds to the prompt and includes an end tag for executing a second task associated with the prompt. An action is determined based on the script agents returning a response to the prompt or upon expiration of the timer. The method has applications including, but not limited to, use cases in medical Al / healthcare, cybersecurity, city planning, safer cities, industry 4.0 and material acquisition for optimization of actions or to support decision-making.

Description

MULTI AGENT TASK PLANNINGCROSS-REFERENCE TO PRIOR APPLICATION

[0001] Priority is claimed to U.S. Provisional Application Serial No. 63 / 547,403 filed on November 6, 2023, the entire contents of which is hereby incorporated by reference herein. FIELD

[0002] The present invention relates to Artificial Intelligence (Al) and machine learning (ML), and in particular to a method, system, data structure, computer program product and computer-readable medium for task planning using multiple Al agents, such as multiple Large Language Model (LLM) agents.BACKGROUND

[0003] LLM agents are capable of planning simple tasks, such as how to bake a cake (see FIG. 1). In these tasks, there is only one agent that has to do the different steps. But often solving a task involves more than one agent. For example, for a knee operation, a patient and a doctor, at a minimum, need to work together. This brings with it a series of new complications and technical challenges for the Al system, such as: what needs to be done by whom by when so that the two agents can proceed to the next step together.SUMMARY

[0004] In an embodiment, the present invention provides a computer-implemented, machine learning method for guided programming of autonomous agents using a meta agent. A first task is generated that includes a start tag, script agents involved in the first task, and a timer associated with the first task or the script agents. A prompt is generated for each script agent of the script agents associated with the first task that includes the timer, wherein each script agent is configured to generate a script that corresponds to the prompt and includes an end tag for executing a second task associated with the prompt. An action is determined based on the script agents returning a response to the prompt or upon expiration of the timer. The method has applications including, but not limited to, use cases in medical Al / healthcare, cybersecurity, city planning, safer cities, industry 4.0 and material acquisition for optimization of actions or to support decision-making.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Embodiments of the present invention will be described in even greater detail below based on the exemplary figures. The present invention is not limited to the exemplary embodiments. All features described and / or illustrated herein can be used alone or combined in different combinations in embodiments of the present invention. The features and advantages ofvarious embodiments of the present invention will become apparent by reading the following detailed description with reference to the attached drawings which illustrate the following:

[0006] FIG. 1 illustrates instructions on how to bake a cake;

[0007] FIG. 2 illustrates an example of a peer-to-peer approach for organizing work of multiple agents;

[0008] FIG. 3 illustrates an example of a planner agent and executor agents approach for organizing work of multiple agents;

[0009] FIG. 4 schematically illustrates a method and system using a meta agent to enable parallel execution of script agents to coordinate their operation for a common goal in accordance with an embodiment of the present invention using a simple example;

[0010] FIG. 5 illustrates a potential infinite loop;

[0011] FIG. 6 illustrates the infinite loop being avoided in accordance with an embodiment of the present invention;

[0012] FIG. 7 illustrates the parallel execution of two agent with a common goal;

[0013] FIG. 8 is a block diagram of an exemplary processing system, which can be configured to perform any and all operations disclosed herein.DETAILED DESCRIPTION

[0014] Embodiments of the present invention provide solutions to the technical challenges of coordinating multiple agents to perform a machine learning task by introducing a meta planning agent.

[0015] In a first aspect, the present invention provides a computer-implemented, machine learning method for guided programming of autonomous agents using a meta agent. A first task is generated that includes a start tag, script agents involved in the first task, and a timer associated with the first task or the script agents. A prompt is generated for each script agent of the script agents associated with the first task that includes the timer, wherein each script agent is configured to generate a script that corresponds to the prompt and includes an end tag for executing a second task associated with the prompt. An action is determined based on the script agents returning a response to the prompt or upon expiration of the timer.

[0016] In a second aspect, the present invention provides the method according to the first aspect, wherein the action includes updating the prompt with all interactions occurring between the start tag and the end tag received in the response.

[0017] In a third aspect, the present invention provides the method according to the first aspect or the second aspect, further comprising: determining to restart the first task based on information provided by a large language model (LLM) that uses the updated prompt, therestarted first task including a new start tag and different input; and providing the restarted first task to the script agents.

[0018] In a fourth aspect, the present invention provides the method according to any of the first to third aspects, further comprising: determining to restart the first task based on the information provided by the LLM that uses the updated prompt, the restarted first task including a continue tag that asks about a failure from the script agents; and providing the restarted first task to the script agents.

[0019] In a fifth aspect, the present invention provides the method according to any of the first to fourth aspects, further comprising: receiving responses from the script agents; and tagging the first task with a continue tag or an end tag based on the responses from the script agents that are provided to a large language model (LLM).

[0020] In a sixth aspect, the present invention provides the method according to any of the first to fifth aspects, wherein the first task is tagged with an end tag, the method further comprising generating a next task that includes the start tag, the script agents involved in the next task, and the timer associated with the next task or the script agents.

[0021] In a seventh aspect, the present invention provides the method according to any of the first to sixth aspects, wherein the next task includes a symbol for an end of the next task, the method further comprising returning a plan associated with the first task and the next task.

[0022] In an eighth aspect, the present invention provides the method according to any of the first to seventh aspects, wherein the timer includes a counter of interactions between the script agents or between the meta agent and the script agents.

[0023] In a ninth aspect, the present invention provides the method according to any of the first to eighth aspects, wherein the prompt includes at least the start tag and an end tag associated with the first task.

[0024] In a tenth aspect, the present invention provides the method according to any of the first to ninth aspects, further comprising: instantiating the meta agent; collecting a list of agent types associated with the script agents; and specifying a number of the script agents and a type of the script agents, wherein generating the first task is based on the meta agent, the list of agent types, the number of script agents, and the type of the script agents.

[0025] In an eleventh aspect, the present invention provides the method according to any of the first to tenth aspects, wherein instantiating the meta agent includes obtaining information about the first task and context for the first task.

[0026] In a twelfth aspect, the present invention provides the method according to any of the first to eleventh aspects, wherein the first task corresponds to determining a treatment plan, the script agents include a doctor agent and a patient agent, the prompt includes medical history of apatient and information about treatment practices, and wherein an end task for the first task includes the treatment plan with individual steps outlined for the patient and a doctor.

[0027] In a thirteenth aspect, the present invention provides the method according to any of the first to twelfth aspects, wherein the first task corresponds to determining a cybersecurity operational plan, the script agents include a security operations center (SOC) agent and a cyber threat intelligence (CTI) agent, the prompt includes at least a current status of an information technology (IT) infrastructure, best practices of CTI teams, and CTI intelligence, and wherein an end task for the first task includes the cybersecurity operational plan with individual steps outlined for the SOC and CTI and a timeline for the cybersecurity operational plan.

[0028] In a fourteenth aspect, the present invention provides a computer system for guided programming of autonomous agents comprising one or more processors, which, alone or in combination, are configured to perform a machine learning method for guided programming of autonomous agents using a meta agent according to any of the first to thirteenth aspects.

[0029] In a fifteenth aspect, the present invention provides a tangible, non-transitory computer-readable medium for guided programming of autonomous agents using a meta agent which, upon being executed by one or more hardware processors, provide for execution of a machine learning method according to any of the first to thirteenth aspects.

[0030] According to existing technology, LLM agents can plan and act autonomously.Currently, an LLM agent can only plan a task with one executor. FIG. 1 depicts a workflow 100 with instructions on baking a cake as executed by a single executor. For example, FIG. 1 depicts a simple task, such as baking a cake, where only a single LLM agent has to do the different steps. However, solving a task often involves more than one agent. Embodiments of the present invention enhance the computational functionality of the Al systems having the LLM agents generally such that multiple LLM agents can plan a more complex task than either could manage alone. Embodiments of the present invention can be practically applied to a number of use cases in medical Al / healthcare, medical planning / diagnosis, drug development, resource planning, smartcity applications, etc., such as for patient treatment planning via citizen procedures in a digital government and maintenance work in factories.

[0031] According to existing technology, there are two different approaches to organizing the work of multiple agents.

[0032] The first approach is a peer-to-peer approach, which is the most common approach (see Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein, “Generative Agents: Interactive Simulacra of Human Behavior,” arXiv:2304.03442 (2023), hereinafter “Park et al.”, which is hereby incorporated by reference herein). In this case, two or more script agents 200 and 202 collaborate together if they share thesame goal - heal knee (as depicted in FIG. 2). However, this approach has several drawbacks. First, collaboration happens only on a spontaneous basis; there is no external mechanism that guarantees that the script agents 200 and 202 will collaborate to achieve the goal. Second, script agents 200 and 202 could easily become stuck in a loop. FIG. 2 illustrates an example of such a situation. Two script agents 200 and 202 share a common goal, but without common planning, the script agents 200 and 202 can be blocked into infinite loops 204 or incur a deadlock situation 206 where one of the agents waits forever for an input coming from the other agent. In this example, the script agent 202 representing the doctor and the script agent 200 representing the patient must conduct a health check before a knee operation. In the first iteration, the doctor advises the patient to lose weight to become fit for the operation, but the patient cannot do so. When they return to the doctor, the doctor suggests once again that the patient lose weight, initiating an endless loop 204. Finally, when a coordination mechanism is absent, script agents 200 and 202 may become indefinitely blocked 206, waiting for another agent to provide them with the necessary information or resources. For example, in FIG. 2, the script agent 202 representing the doctor waits for the script agent 200 representing the patient to go to the hospital, thereby becoming indefinitely blocked in a deadlock situation 206.

[0033] The second approach is a Planner agent and Executor agents approach (AutoGPT, MRKL (see, Transformers agents), in which a single agent 300 takes on the responsibility of both planning and executing the tasks, as depicted in FIG. 3. Initially, the planner agent 300 devises a sequence of steps required to achieve the goal. Subsequently, it assigns tasks 302 or subtasks 304 to specialized agents or tools 306 in a sequential manner. In particular, complex tasks 302 are split into smaller subtasks 304 executed by specialized agents 306. The outputs produced by these specialized agents or tools 306 are utilized by the Planner agent 300 to progress toward the goal. However, this approach is not suitable for action plans involving more than one agent taking an active part in the script. Specialized agents 306 are exclusively aware of their designated subtask 304 and related subgoals, and they play a passive role in achieving the final objective. Furthermore, the specialized agents 306 lack direct interaction with one another and never share a common goal. In this approach, given that the Planner agent 300 is the sole entity aware of the final goal, it is required to act as the task “expert”. This considerably limits the applicability of this approach for replicating or executing complex scripts where agents need to collaborate to achieve a common goal. Moreover, this approach does not support the parallel execution of agents with shared subgoals.Definitions:

[0034] Script: A directed acyclic graph.

[0035] Large Language Model (LLM): A sequential neural network (e.g. transformer) with text as input and output. LLM is used herein as the commonly agreed upon term, but no restriction is made on the model size itself (i.e., it could also be small and therefore only an ‘LM’ but still within the definition of an LLM as used herein).

[0036] Script agent: An LLM agent capable of generating scripts as described in Keisuke Sakaguchi, Chandra Bhagavatula, Ronan Le Bras, Niket Tandon, Peter Clark, and Yejin Choi, “proScript: Partially Ordered Scripts Generation,” In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2138-2149, Punta Cana, Dominican Republic, Association for Computational Linguistics (2021) (including the example of FIG. 1), hereinafter “Sakaguchi et al.”; and Quan Yuan, Mehran Kazemi, Xin Xu, Isaac Noble, Vaiva Imbrasaite, and Deepak Ramachandran, “TaskLAMA: Probing the Complex Task Understanding of Language Models,” arXiv:2308. 15299 (2023), hereinafter ‘Yuan et al.”, each of which is hereby incorporated by reference herein. The script agent represents an actor (see definition below).

[0037] Meta agent: An LLM agent that communicates between two or more script agents.

[0038] Actor: An entity such as a human person or a company that exists in real life and is represented by a script agent.

[0039] Task: A joint task that needs to be solved by several actors. In embodiments of the present invention, each actor is represented by a script agent. Embodiments of the present invention plan how to solve this task via an overseeing meta agent.

[0040] Prompt: The task instruction for the LLM.

[0041] Context: Information given to the LLM in addition to the prompt. This information can be used as an input to the LLM.

[0042] BIO tags: Stands for Begin, Inside, and Outside. It is used for sequence labelling, for example:Alex B-PER is O going O to O Los B-LOC Angeles I-LOC in OCalifornia B-LOC see also<<https: / / en.wikipedia.org / wiki / Inside%E2%80%93outside %E2%80%93beginning_(tagging)»

[0043] SCE: Embodiments of the present invention provide a new notation and new tags that are adapted and used in a different manner relative to BIO tags. These new tags are termed Start, Continue, End. For example a series with action “Plan operation” might be: “Plan operation - S”, “Plan operation - C”, “Plan operation - E”. Unlike BIO tags, there will always be at least an S and E of each action.

[0044] Timer: Timer or counter associated with a specific task and actor. For sake of simplicity, the present disclosure refers to timers, but any timer could be implemented as a counter of interaction between script agents or meta agent and script agent.

[0045] Embodiments of the present invention provide an approach for organizing the work of multiple agents that collaborate together toward a common goal. The approach introduces and involves the use of a meta agent that acts as a coordinator between the different script agents. The meta agent is aware of the intermediate steps / tasks that need to be performed to achieve the final objective, assigns its tasks to script agents, and handles the communication among agents that collaborate together on a specific task.

[0046] Unlike the Planner agent and Executor agents architecture, the meta agent is not an “expert” on the final objective, but rather acts as a simple proxy between script agents collaborating on a common task. When two or more agents work on the same task, there is a risk that they will loop infinitely into the same interaction or be blocked indefinitely while waiting for the output of one agent. Embodiments of the present invention introduce and utilize the SCE notation to enable the meta agent to detect and eventually resolve these situations. Moreover, the SCE notation makes use of text tags that can be easily interpreted by an LLM, enabling the implementation of an LLM-based meta agent.

[0047] FIG. 4 illustrates an example according to an embodiment of the present invention in which two script agents collaborate to achieve a common goal. In this scenario, one of the script agents 400 represents a patient with a knee injury, while the other script agent 402 represents a doctor specialized in knee operations. Their shared objective is to heal the knee. The meta agent 404 is aware of the various phases that must be executed to achieve the goal and initiates the script agents 400 and 402 when necessary. The meta agent 404 enables parallel execution of the script agents 400 and 402, it also prevents infinite loops 406 and deadlocks 408 - as discussed above with reference to FIG. 2. Preventing the infinite loops 406 is described below with reference to FIG. 6 and preventing deadlocks is described below with reference to FIG. 7.

[0048] Embodiments of the present invention provide that the meta agent is configured to handle two critical situations, in particular, infinite loops and deadlocks / providing for parallel execution.

[0049] As discussed above, two script agents may enter an infinite loop. FIG. 5 illustrates an example of such a situation (a potential infinite loop without SCE tags). The doctor must assess the patient’s health before proceeding with the operation. Therefore, the doctor script agent will be initiated with the following prompt: “Evaluate the patient's health, patient weight 100kg,” and it will instruct the patient to lose weight. If the patient is unable to complete this task, the doctor will once again instruct them to lose weight, initiating an infinite loop. It's important to note that the meta agent will monitor this conversation, and even if it detects the loop, it may restart the task with the same initial state (starting prompt), resulting in the same loop again. Indeed, since the meta agent does not possess domain knowledge (is not an “expert” in the final task), it is unable to determine whether the problem is caused by the patient or the doctor.

[0050] Thus, in the example of FIG. 5, the doctor has to evaluate patient health, and the patient acts depending on what the doctor says. Even if a loop is detected, the task will restart with the same state. Thus, it is a technical challenge for the LLM to detect the problem without domain knowledge. FIG. 5 represents a serialized interaction between two script agents without using SCE tags but using a meta agent.

[0051] Embodiments of the present invention provide that, by using SCE notation, the meta agent can detect the situation and avoid entering an infinite loop. FIG. 6 shows how the infinite loop is avoided using the SCE tags. After the loop is detected, the meta agent can restart the task with a different state. It begins by initiating the task with the proposed SCE notation, where the “health evaluation” task is marked with the S (Start) tag and associated metadata containing information related to timers and involved script agents. In this specific example, two script agents are involved (representing the two actors - doctor and patient), and they are explicitly associated with two timers, labeled A and B. The meta agent proceeds by instantiating the first timer (A), issuing a C (Continue) task, providing the doctor agent with their respective prompt, and awaiting the doctor agent's task output. In embodiments instantiating the first timer (A) can include defining the iterations or time unit associated with the first timer (A). The doctor agent performs their task and responds with a message tagged with the E (End) tag. The meta agent updates the prompt with all interactions occurring between the S tag and the most recent E tag (1st iteration in FIG. 6). The meta agent's LLM is invoked with this updated prompt, after which it issues a C tag and conveys the doctor agent's instructions to the patient agent. The iteration between script agents repeats only until the timer associated with the patient expires. For example, the timer B may expire after a number of iterations or after an amount of elapsed time. At that point, the meta agent detects the loop and can restart the task with a different initial state. This is because, by passing all the events that occur between the S tag and the E tag as a prompt, the meta agent has all the necessary information to reset the "health evaluation" task with adifferent input - depicted in FIG. 6 as “Task: {health evaluation, patient field to lose weight}.” Based on the information, then the doctor can, for example, decide to move ahead with the operation despite the increased risk and the infinite loop is prevented 600.

[0052] Embodiments of the present invention also provide that two or more script agents can work in parallel to achieve a common goal. In the example, this applies to a doctor and a patient who must both be prepared for the operation. When multiple script agents should act in parallel, the S task is associated with only one timer. Because the timer is shared among script agents performing the same task, the meta agent can issue a C operation that involves all the script agents associated with the task (including both the doctor agent and the patient agent in the example). The meta agent then awaits their respective E tags. When the agents return with their output, the task is considered complete, and the meta agent can proceed to schedule the next task (as shown in box (a) of FIG. 7). In the event that the timer expires (as shown in boxes (b) and (c) of FIG. 7), the meta agent will utilize all the interactions that occurred between the S tag and the most recent E tag as a prompt to reschedule the task with different input. If the available information is insufficient to reschedule the task with different input, the meta agent will contact the script agent to inquire why they were unable to complete the task (as depicted in box (c) of FIG. 7).

[0053] Thus, FIG. 7 shows the parallel execution of two agents with a common goal. Box (a) is an example where both doctor and patient handle their tasks as they should and the end goal is immediately reached. Box (b) is an example where the patient failed their task and the meta agent needs to intervene: it restarts the task with the history in the prompt and, optionally, an additional tag, such as “patient failed to prepare for operation”. Box (c) shows an example where the meta agent directly mediates by asking why the doctor agent and patient agent failed to complete their task in time.

[0054] Embodiments of the present invention thus provide for general improvements to computers in Al and ML systems to enable coordination of Al agents for a common task, and to overcome the technical problems of infinite loops and waiting on another agent or deadlocks. Moreover, embodiments of the present invention can be practically applied to use cases to effect further improvements in a number of technical fields including, but not limited to, medical (e.g., digital medicine, personalized healthcare, Al -assisted drug or vaccine development (AIDD), diagnosis, patient planning etc.), planning operations or procedures, cyberthreat intelligence, city planning, digital governance, maintenance, public safety (e.g., disaster / emergency planning) and smart cities (e.g., automated traffic or vehicle control, smart districts, smart buildings, smart industrial plants, smart agriculture, energy management, etc.). Embodiments of the presentinvention can be applied to effect improvements in any use case involving the planning and coordination of multiple Al agents for a common objective.

[0055] In an embodiment, the present invention can be applied for patient treatment planning. A use case is that a patient has a health issue and a treatment plan needs to be devised. This treatment plan should reflect (1) the doctor’s best practice, (2) the patient’s individual situation and (3) other information that is relevant, but potentially unknown to the doctor. As input, the agents are given medical history of the patient, information about general treatment practices and the overall goal. There are at least three agents: a doctor agent, a patient agent and the meta agent. There could be additional agents, such as more doctors or physiotherapists. Application of the method according to an embodiment of the present invention plans out the common joint steps, resolves issues and conflicts between the schedules of the different actors, which include at least the doctor and the patient, but could also include more actors, such as a physiotherapist. The output includes a treatment plan where the patient and the doctor have their individual steps outlined.

[0056] In an embodiment, the present invention can be applied for cyberthreat intelligence (CTI). A use case is that cybersecurity operations, such as threat hunting, incident response, and forensic analysis, require strategic coordination between multiple actors, like Security Operations Center (SOC) members and CTI analysts. An embodiment of the present invention provides the planning of such tasks. The input includes operational manuals for the SOC, prompts with the current status of the IT infrastructure, best practices of CTI teams, prompts containing the relevant context and CTI intelligence. There are at least three agents: a SOC agents, a CTI agent, and the meta agent. Application of the method according to an embodiment of the present invention outlines the joint tasks of all actors involved in the cybersecurity operations and manages conflicts and disruptions between different roles. Finally, it will ensure the correct and timely delivery of information that has to be shared between groups, guaranteeing that vital data is shared promptly and without errors, ultimately enhancing the overall security posture of the organization or system it is applied to. The output includes a comprehensive cybersecurity operational plan where all stakeholders are aware of their roles and the established timeline. This plan can be promptly adapted based on the current situation or task.

[0057] In an embodiment, the present invention can be applied for planning operations. A use case is that multiple actors are required to solve a task. For input, a lead planner collects all required information as relevant context and formulates a prompt. Application of the method according to an embodiment of the present invention plans out the common joint steps of all actors participating in the work and resolves issues and conflicts between different steps. Thereare at least three agents: two executing agents and the meta agent. The output includes a task plan where all actors know what they should do and when.

[0058] In an embodiment, the present invention can be applied for city planning. A use case is that city works, such as maintenance, new developments, event setups, etc., require planning between multiple actors, such as different construction firms. An embodiment of the present invention plans the tasks for such work. For input, the city planning lead provides all relevant information, e.g. documentation, other relevant context. Application of the method according to an embodiment of the present invention plans out the common joint steps of all actors participating in the work and resolves issues and conflicts between different steps. There are at least three agents: two executing agents and the meta agent. The output includes a city work plan where all actors know what they should do and when.

[0059] In an embodiment, the present invention can be applied for digital government / public administration. A use case is that, in a digital government, the public administration needs to plan their tasks with two or more actors, e.g. the government worker and then citizens. For input, the government worker provides all context and the relevant prompt. Application of the method according to an embodiment of the present invention plans out the common joint steps of all actors participating in the work and resolves issues and conflicts between different steps. There are at least three agents: a citizen agent, a digital government agent, and the meta agent. The output includes a public administration plan where all actors know what they should do and when.

[0060] In an embodiment, the present invention can be applied for maintenance (e.g., of factories or buildings). A use case is that factories have complex plants and it requires specialized knowledge to understand what precautions and steps need to be taken, e.g. to replace a certain part. This specialized knowledge might not be available (e.g. aching population, legacy systems). For input, the maintenance lead provides all relevant information, e.g. documentation, notes from pervious workers, etc. There are at least three agents: two maintenance agents and the meta agent. Application of the method according to an embodiment of the present invention plans out the common joint steps of all people participating in the maintenance work and resolves issues and conflicts between different steps. The output includes a maintenance plan where all actors know what they should do and when.

[0061] In an embodiment, the present invention can be applied for manufacturing or production. A use case is that factory operations, such as production, material acquisition, and product delivery, require coordinated planning among multiple actors, such as different production lines and suppliers. For input, the factory production manager provides all relevant information, e.g. production schedules, supplier details, delivery times, and other relevantcontext. Application of the method according to an embodiment of the present invention lays out the collaborative steps of all actors participating in the operation and resolves issues and conflicts between different. There are at least three agents: two production agents and the meta agent. The output includes a factory operation plan where all actors know their specific roles and responsibilities at any given time.

[0062] In an embodiment, the present invention can be applied for disaster or emergency response planning. A use case is that, in an emergency situation, several actors have to coordinate who helps whom where and when. For input, the lead provides all context and the prompt. Application of the method according to an embodiment of the present invention plans out the common joint steps of all actors participating in the work and resolves issues and conflicts between different steps. There are at least three agents: two emergency response agents and the meta agents. The output includes an emergency response plan where all actors know what they should do and when.

[0063] In an embodiment, the present invention provides a method for multi agent planning, comprising the steps of:1) Providing input by: a) Instantiating the meta agent. A meta agent may be instantiated with a list of supported agents, a number of the supported agents, and a type associated with each of the supported agents. In embodiments the meta agent may be instantiated with information about the task (e.g. different phases and goal) and information about the context. b) Collecting a list of agent types (e.g. doctor, patient). The list of agent types depends on the task at hand, as in the previous example a doctor agent and a patient agent may be expected to be collected for a task corresponding to an operation or medical procedure. c) Specifying the number of script agents and their type. The meta agent uses the number of script agents and the type of script agents to orchestrate the task and / or assign subtasks.2) The meta agent produces the first action with an S tag, all script agents that need to be called upon and timeout, finally it repeats the first action but omits tag, script agents and timeout. An example of the first action with an S tag is depicted in FIG. 7 (e.g., “S: {Task: {prepare for operation}, timer : {A}, actor: {doctor, patient} } .)” A task is a joint task that needs to be solved by several actors (represented by script agents), and a task can be composed of multiple actions. In embodiments an action can be predefined or dynamically decided by the meta agent. The S tag is associated with metadata: involved script agents, timer associated with script agents or with the S action (in this case the timer is the same for all the agents).3) The meta agent waits for each of the script agents: Each script agent specified by the meta agent in step 1) is called upon with a prompt (specifying which task / action they have to perform) by the meta agent and generates its own script for this step. As the script agents are agents, given a task they can autonomously proceed to the solution of the task. At the end of a script, agents specify an end symbol (e.g., the “E” tag in FIG. 7). Together with a script agent identifier, all information is sent to the meta agent. In embodiments the script agent identifier is a simple identifier (ID) used by the meta agent to understand which script agent is communicating or providing data / information to it. In FIG. 7 the “doctor” and “patient” represent examples of a script agent identifier.4) The meta agent is locked until one of the following two conditions are met: a) All called upon script agents return their script. b) The timeout is hit. If a timeout is hit, it can be resolves in the following way:The meta agent’s prompt is updated with all interactions occurring between the S tag and the most recent E tag.The meta agent decides (prompt as input to LLM) if there is enough information to restart S action with different input, if yes, the new S action is initiated, if no, a C action explicitly asking about the failure to the script agents is initiated. The meta agent determines if there is enough information by using the LLM and logs of the interactions with the SCE tags to understand why a timer has expired and why a script agent(s) have failed. For example, referring to FIG. 6, an LLM may be prompted with the definition of SCE tags, involved script agents, their type, the final goal, and the log of the conversation (e.g., information between the start of the task and the last iteration), and a question such as “tell me why the task failed?”). The LLM can determine the reason for the failure and restart the task accordingly. In situations where the LLM is unable to determine a reason for the failure the meta agent will query the script agents to understand the reasons for the failure.5) The meta agent continues its generation (e.g., the script agents continue to do their actions and the meta agent continues to orchestrate communication between the script agents and completing the task). Based on the output from the script agents, the meta agent first tags the action. It can be either C (for continue) or E (for End). a) Continue: The meta agent produces all script agents and timeouts and forward the action to the script agent without the tag. b) End: The meta agent produced a next action with an S tag, all script agents that need to be called upon and an overall timeout, finally it forwards the first action, but omits the tag, script agents and timeout.6) End point: If the next chosen action is the symbol for the end of the task, then the meta agent terminates and the plan is returned. In embodiments the meta agent uses an LLM to determine if the goal is reached (E) or not (C).

[0064] Embodiments of the present invention provide for the following improvements and technical advantages over existing technology:1) Introducing a new computational entity to the multi agent Al system, in particular a meta agent that can handle the communication between two or more script agents. To enable this, the following technical solutions to technical problems are provided: a) Task planning serialization for input to a language-based, sequential Al system: The meta agent needs to follow a tree structure without loops, otherwise script agent communication could get caught in endless loops. To ensure no loops are created, embodiments of the present invention provide to be able to split up the meta agent tasks using the defined SCE tags. Without these tags, there would be loops, which would make it impossible to directly use an LLM as a meta agent. Only by using the SCE tags and defining a sequential structure for the tasks, can LLM agents communicate via the meta agent. b) The meta agent adjusts its strategy (e.g., task prompt) for the script agents to mediate and prevent deadlocks: The structure in which the meta agent outputs the meta timeline and communicates with the other script agents. If the timeout and a timeout handling strategy wasn’t given, then a task planning could get stuck indefinitely. The timeout is used to detect parallel or sequential tasks in SCE tags and based on the information provided by this, the meta agent can adjust its strategy (e.g., the task prompt) before re-engaging with the script agents.2) Providing for increased scalability and flexibility of Al systems for improving and enabling further real world applications. Existing agents cannot communicate and agree on a joint timeline. Sakaguchi et al. and Yuan et al. can create task scripts for one agent only. Park et al. has multiple agents, but there are no task scripts. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang, “AutoGen: Enabling Next- Gen LLM Applications via Multi-Agent Conversation,” arXiv:2308.08155 (2023), which is hereby incorporated by reference herein can have a “manager” agent (see FIG. 3), but this agent only does these three tasks: “dynamically selecting a speaker, collecting responses from the selected speaker, and broadcasting the message” (page 9). None of these steps addresses or solves the technical issues identified and solved by embodiments of the present invention. Being able to coordinate multiple Al agents as provided by embodiments of the present invention is an important technological improvement to move LLM agents towards more real world applications, and provides an improved multi agent Al system as a base for further applications.

[0065] Alexandras Nikon, Jana Tumova, and Dimos V. Dimarogonas, “Cooperative Task Planning of Multi -Agent Systems Under Timed Temporal Specifications,” arXiv: 1509.09137 (2015), which is hereby incorporated by reference herein, describe a planner for a multi agent system as it exists in traditional robotics. In traditional robotics, the agents are usually preprogrammed or might use an Al module that merely classifies over a set of pre-specified actions, which actions should be taken next. In contrast, embodiments of the present invention are based on generative Al agents, in particular LLM agents, which can produce completely new, previously unseen actions. This is a significant technological development that will allow Al agents to act autonomously and react in situations that where never seen before. With this enhanced computer functionality to generative LLM agents, there are different technical challenges that are overcome by embodiments of the present invention as discussed above, for the first time enabling a meta agent that can coordinate between other LLM script agents.

[0066] Referring to FIG. 8, a processing system 800 can include one or more processors 802, memory 804, one or more input / output devices 806, one or more sensors 808, one or more user interfaces 810, and one or more actuators 812. Processing system 800 can be representative of each computing system disclosed herein.

[0067] Processors 802 can include one or more distinct processors, each having one or more cores. Each of the distinct processors can have the same or different structure. Processors 802 can include one or more central processing units (CPUs), one or more graphics processing units (GPUs), circuitry (e.g., application specific integrated circuits (ASICs)), digital signal processors (DSPs), and the like. Processors 802 can be mounted to a common substrate or to multiple different substrates.

[0068] Processors 802 are configured to perform a certain function, method, or operation (e.g., are configured to provide for performance of a function, method, or operation) at least when one of the one or more of the distinct processors is capable of performing operations embodying the function, method, or operation. Processors 802 can perform operations embodying the function, method, or operation by, for example, executing code (e.g., interpreting scripts) stored on memory 804 and / or trafficking data through one or more ASICs. Processors 802, and thus processing system 800, can be configured to perform, automatically, any and all functions, methods, and operations disclosed herein. Therefore, processing system 800 can be configured to implement any of (e.g., all of) the protocols, devices, mechanisms, systems, and methods described herein.

[0069] For example, when the present disclosure states that a method or device performs task “X” (or that task “X” is performed), such a statement should be understood to disclose that processing system 800 can be configured to perform task “X”. Processing system 800 isconfigured to perform a function, method, or operation at least when processors 802 are configured to do the same.

[0070] Memory 804 can include volatile memory, non-volatile memory, and any other medium capable of storing data. Each of the volatile memory, non-volatile memory, and any other type of memory can include multiple different memory devices, located at multiple distinct locations and each having a different structure. Memory 804 can include remotely hosted (e.g., cloud) storage.

[0071] Examples of memory 804 include a non-transitory computer-readable media such as RAM, ROM, flash memory, EEPROM, any kind of optical storage disk such as a DVD, a Blu- Ray® disc, magnetic storage, holographic storage, a HDD, a SSD, any medium that can be used to store program code in the form of instructions or data structures, and the like. Any and all of the methods, functions, and operations described herein can be fully embodied in the form of tangible and / or non-transitory machine-readable code (e.g., interpretable scripts) saved in memory 804.

[0072] Input-output devices 806 can include any component for trafficking data such as ports, antennas (i.e., transceivers), printed conductive paths, and the like. Input-output devices 806 can enable wired communication via USB®, DisplayPort®, HDMI®, Ethernet, and the like. Input-output devices 806 can enable electronic, optical, magnetic, and holographic, communication with suitable memory 806. Input-output devices 806 can enable wireless communication via WiFi®, Bluetooth®, cellular (e.g., LTE®, CDMA®, GSM®, WiMax®, NFC®), GPS, and the like. Input-output devices 806 can include wired and / or wireless communication pathways.

[0073] Sensors 808 can capture physical measurements of environment and report the same to processors 802. User interface 810 can include displays, physical buttons, speakers, microphones, keyboards, and the like. Actuators 812 can enable processors 802 to control mechanical forces.

[0074] Processing system 800 can be distributed. For example, some components of processing system 800 can reside in a remote hosted network service (e.g., a cloud computing environment) while other components of processing system 800 can reside in a local computing system. Processing system 800 can have a modular design where certain modules include a plurality of the features / functions shown in FIG. 8. For example, I / O modules can include volatile memory and one or more processors. As another example, individual processor modules can include read-only-memory and / or local caches.

[0075] While subject matter of the present disclosure has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to beconsidered illustrative or exemplary and not restrictive. Any statement made herein characterizing the invention is also to be considered illustrative or exemplary and not restrictive as the invention is defined by the claims. It will be understood that changes and modifications may be made, by those of ordinary skill in the art, within the scope of the following claims, which may include any combination of features from different embodiments described above.

[0076] The terms used in the claims should be construed to have the broadest reasonable interpretation consistent with the foregoing description. For example, the use of the article “a” or “the” in introducing an element should not be interpreted as being exclusive of a plurality of elements. Likewise, the recitation of “or” should be interpreted as being inclusive, such that the recitation of “A or B” is not exclusive of “A and B,” unless it is clear from the context or the foregoing description that only one of A and B is intended. Further, the recitation of “at least one of A, B and C” should be interpreted as one or more of a group of elements consisting of A, B and C, and should not be interpreted as requiring at least one of each of the listed elements A, B and C, regardless of whether A, B and C are related as categories or otherwise. Moreover, the recitation of “A, B and / or C” or “at least one of A, B or C” should be interpreted as including any singular entity from the listed elements, e.g., A, any subset from the listed elements, e.g., A and B, or the entire list of elements A, B and C.

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented method for guided programming of autonomous agents using a meta agent, the computer-implemented method comprising: generating a first task that includes a start tag, script agents involved in the first task, and a timer associated with the first task or the script agents; generating a prompt for each script agent of the script agents associated with the first task that includes the timer, wherein each script agent is configured to generate a script that corresponds to the prompt and includes an end tag for executing a second task associated with the prompt; and determining an action based on the script agents returning a response to the prompt or upon expiration of the timer.

2. The computer-implemented method according to claim 1, wherein the action includes updating the prompt with all interactions occurring between the start tag and the end tag received in the response.

3. The computer-implemented method according to claim 2, further comprising: determining to restart the first task based on information provided by a large language model (LLM) that uses the updated prompt, the restarted first task including a new start tag and different input; and providing the restarted first task to the script agents.

4. The computer-implemented method according to claim 2 or claim 3, further comprising: determining to restart the first task based on the information provided by the LLM that uses the updated prompt, the restarted first task including a continue tag that asks about a failure from the script agents; and providing the restarted first task to the script agents.

5. The computer-implemented method according to any of the preceding claims, further comprising: receiving responses from the script agents; and tagging the first task with a continue tag or an end tag based on the responses from the script agents that are provided to a large language model (LLM).

6. The computer-implemented method according to claim 5, wherein the first task is tagged with an end tag, the method further comprising generating a next task that includes the start tag, the script agents involved in the next task, and the timer associated with the next task or the script agents.

7. The computer-implemented method according to claim 6, wherein the next task includes a symbol for an end of the next task, the method further comprising returning a plan associated with the first task and the next task.

8. The computer-implemented method according to any of the preceding claims, wherein the timer includes a counter of interactions between the script agents or between the meta agent and the script agents.

9. The computer-implemented method according to any of the preceding claims, wherein the prompt includes at least the start tag and an end tag associated with the first task.

10. The computer-implemented method according to any of the preceding claims, further comprising: instantiating the meta agent; collecting a list of agent types associated with the script agents; and specifying a number of the script agents and a type of the script agents, wherein generating the first task is based on the meta agent, the list of agent types, the number of script agents, and the type of the script agents.

11. The computer-implemented method according to claim 10, wherein instantiating the meta agent includes obtaining information about the first task and context for the first task.

12. The computer-implemented method according to any of the preceding claims, wherein the first task corresponds to determining a treatment plan, the script agents include a doctor agent and a patient agent, the prompt includes medical history of a patient and information about treatment practices, and wherein an end task for the first task includes the treatment plan with individual steps outlined for the patient and a doctor.

13. The computer-implemented method according to any of the preceding claims, wherein the first task corresponds to determining a cybersecurity operational plan, the script agents include a security operations center (SOC) agent and a cyber threat intelligence (CTI) agent, the prompt includes at least a current status of an information technology (IT) infrastructure, best practices of CTI teams, and CTI intelligence, and wherein an end task for the first task includes the cybersecurity operational plan with individual steps outlined for the SOC and CTI and a timeline for the cybersecurity operational plan.

14. A computer system for guided programming of autonomous agents using a meta agent, the computer system comprising one or more hardware processors which, alone or in combination, are configured to provide for execution of the following steps: generating a first task that includes a start tag, script agents involved in the first task, and a timer associated with the first task or the script agents;generating a prompt for each script agent of the script agents associated with the first task that includes the timer, wherein each script agent is configured to generate a script that corresponds to the prompt and includes an end tag for executing a second task associated with the prompt; and determining an action based on the script agents returning a response to the prompt or upon expiration of the timer.

15. A tangible, non-transitory computer-readable medium having instructions thereon which, upon being executed by one or more processors, provide for guided programming of autonomous agents using a meta agent by execution of the following steps: generating a first task that includes a start tag, script agents involved in the first task, and a timer associated with the first task or the script agents; generating a prompt for each script agent of the script agents associated with the first task that includes the timer, wherein each script agent is configured to generate a script that corresponds to the prompt and includes an end tag for executing a second task associated with the prompt; and determining an action based on the script agents returning a response to the prompt or upon expiration of the timer.

Images