Application management method and related equipment

Abstract

The invention discloses an application management method. The method comprises the steps that an interaction device receives a query of a user; the agent routing arrangement device identifies an agent requested to be queried by a user according to the query, retrieves a target agent matched with the query when querying an unrequested agent, then allocates a task for the target agent, and determines an execution sequence or a dependency relationship of the task; the agent routing arrangement device generates a standard operation program (SOP) through a language model according to the execution sequence or dependency relationship of the tasks and the target agent; and the agent execution device loads the target agent according to the standard operation program, executes the operation node in the target agent, and obtains an answer. According to the method, intention recognition can be carried out on the query of the user, the target agent matched with the query can be retrieved, the task can be allocated to the target agent, a static or dynamic SOP can be generated through task arrangement, diversified service requirements and a complex service environment can be met, and high flexibility and adaptability are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence (AI), and in particular to an application management method, an application management platform, a computing device cluster, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] In recent years, large language models (LLMs) have made remarkable achievements and gradually shown the potential to approach human intelligence. Based on this, more and more research has begun to use LLMs as core controllers to build agents with decision-making capabilities. Compared with reinforcement learning, LLM-based agents have richer internal world knowledge, and can support more reasoned actions even without training on specific domain data. In addition, LLM-based agents provide a natural language interface, which is convenient for human interaction.

[0003] Unlike zero-shot prompting for LLMs, agents support more complex interactions and coordination. Specifically, agents have control structures such as planning, looping, and reflection, and can fully utilize the internal reasoning capabilities of models such as LLMs to complete tasks. Further, agents can also combine tools, plugins, and function calls to perform a wider range of general tasks.

[0004] In the process of applying large language models to business agents, application development and running based on multiple agents is a key technical challenge. Related technologies often show insufficient flexibility and poor adaptability when facing diverse user needs and complex business environments. Specifically, different levels of difficulty of user queries, different types of business agents, and complex iterative interactions of multiple numbers or types of agents make it difficult for related technologies to achieve efficient and flexible application development and running. SUMMARY

[0005] The present application provides an application management method that can perform intent recognition on user queries, retrieve target agents matching the queries, and assign tasks to the target agents. By scheduling the target agents, static or dynamic SOPs can be generated to meet diverse business needs and complex business environments, with high flexibility and adaptability. The present application also provides an application management platform, a computing device cluster, a computer readable storage medium, and a computer program product corresponding to the above method.

[0006] In a first aspect, the present application provides an application management method. The method can be executed by an application management platform. The application management platform can be a software system, which can be a standalone software system or a software system integrated in other software. In some examples, the application management platform can be a multi-type / multi-agent application development and running framework supporting dynamic / static business processes. The software system can be provided to customers in the form of a software package for self-deployment or in the form of a cloud service for users to subscribe to the cloud service to use corresponding capabilities or functions. The software system can be deployed in a computing device cluster, and the computing device cluster executes the program code of the software system to execute the application management method of the present application.

[0007] The application management platform includes an interaction device, an agent routing arrangement device, and an agent execution device. In a specific implementation, the interaction device receives a user query, and the agent routing arrangement device identifies an agent requested by the user to query according to the query. When the query does not request an agent, the agent routing arrangement device retrieves a target agent matching the query. The agent routing arrangement device can assign a task to the target agent, determine an execution order or dependency relationship of the task, and generate a standard operating procedure (SOP) through a language model according to the execution order or dependency relationship of the task and the target agent. The agent execution device can load the target agent according to the SOP, execute an operation node in the target agent, and obtain an answer.

[0008] The method can perform intent recognition on a user query, retrieve a target agent matching the query, assign a task to the target agent, and generate a static or dynamic SOP by arranging the target agent, thereby meeting diversified business requirements and complex business environments and having high flexibility and adaptability.

[0009] In some possible implementations, the target agent includes multiple agents. The agent routing arrangement device can further perform correlation analysis on the multiple agents using a language model to obtain a cooperation evaluation result of the multiple agents. The agent routing arrangement device determines multiple-agent cooperation to respond to the query according to the cooperation evaluation result.

[0010] The method can evaluate the cooperation potential and effect between agents by performing correlation analysis on multiple agents in the target agent, start multiple-agent cooperation for agents with high cooperation potential and good effect, and thereby ensure the efficiency of cooperation.

[0011] In some possible implementation manners, the agent routing and arrangement apparatus can further acquire at least one of a length of the query, a question definiteness, domain knowledge or professional terms involved, lexical richness, and a grammatical structure; and predict, according to at least one of the length, the question definiteness, the domain knowledge or the professional terms, the lexical richness, and the grammatical structure, whether the query is a complex query. When the query is a complex query, the agent routing and arrangement apparatus retrieves, from the agent registration center, a target agent matching the query.

[0012] The method can analyze the complexity of the query first, and then start agent discovery and team formation for a complex query, and generate a standard operation procedure, which is not limited to a predefined standard operation procedure and has higher flexibility. Moreover, different types of standard operation procedures can be used for different types of queries, so that the method can avoid simple queries from occupying more resources and improve overall efficiency and resource utilization.

[0013] In some possible implementation manners, the interaction apparatus can further acquire a context of the query. The agent routing and arrangement apparatus can rewrite the query according to the context and a historical conversation. The rewritten query has clearer semantics and includes more information. Accordingly, the agent routing and arrangement apparatus can retrieve, from the agent registration center, a target agent matching the rewritten query. In this way, the accuracy of retrieval can be improved.

[0014] In some possible implementation manners, the agent execution apparatus can load a first agent in the target agent according to the standard operation procedure. The agent execution apparatus loads a first operation node in the first agent. Then the agent execution apparatus acquires a node type of the first operation node and node information of the first operation node. The node information of the first operation node includes at least one of an input / output parameter of the first operation node, an execution logic of the first operation node, and a dependency relationship of a task allocated to the first operation node. The dependency relationship of the task allocated to the first operation node includes a dependency relationship between the first agent and other agents in the target agent, a dependency relationship between the first operation node and other operation nodes in the first agent, or a dependency relationship between parameters used for executing the first operation node. The agent execution apparatus can execute the first operation node according to the node type of the first operation node and the node information of the first operation node. Then the agent execution apparatus loads a second agent in the target agent according to the standard operation procedure. The agent execution apparatus loads a second operation node in the second agent, acquires a node type of the second operation node and node information of the second operation node. The node information of the second operation node includes at least one of an input / output parameter of the second operation node, an execution logic of the second operation node, and a dependency relationship of a task allocated to the second operation node. The dependency relationship of the task allocated to the second operation node includes a dependency relationship between the second agent and other agents in the target agent, a dependency relationship between the second operation node and other operation nodes in the second agent, or a dependency relationship between parameters used for executing the second operation node. The agent execution apparatus executes the second operation node according to the node type of the second operation node and the node information of the second operation node.

[0015] The method can implement that the multiple agents execute the business process according to the dynamic standard operation procedure, and has high flexibility and adaptability.

[0016] In some possible implementation manners, before the agent execution apparatus loads the second agent in the target agent according to the standard operation procedure, the agent execution apparatus can also determine whether the first operation node is the last operation node of the first agent. If not, the agent execution apparatus executes the next operation node of the first agent. The method can ensure the accuracy of the business process by identifying whether the operation node is the last operation node of the agent, and executing the operation node of the next agent after the operation node in one agent is executed.

[0017] In some possible implementation manners, the application management platform further includes an agent development apparatus. The agent development apparatus can also receive metadata configured by a user for the agent. The metadata includes at least one of a prompt template identifier, a knowledge base identifier, a tool identifier, or a model identifier of a language model. The agent development apparatus can register the metadata of the agent to the agent registration center.

[0018] The method can adaptively enhance the capability of the agent in the running state by configuring metadata such as template identification, knowledge base identification, tool identification or model identification of a language model in the agent development state, thereby improving the acceptance rate of the answer of the agent.

[0019] In a second aspect, the application provides an application management platform. The application management platform comprises an interaction device, an agent routing arrangement device and an agent execution device.

[0020] The interaction device is configured to receive a query of a user.

[0021] The agent routing arrangement device is configured to identify an agent requested by the user according to the query, retrieve a target agent matched with the query when the query does not request an agent, assign a task to the target agent, and determine an execution order or a dependency relationship of the task.

[0022] According to the execution order or the dependency relationship of the task and the target agent, a standard job procedure is generated by using a language model.

[0023] The agent execution device is configured to load the target agent according to the standard job procedure, execute an operation node in the target agent, and obtain an answer.

[0024] In some possible implementation manners, the target agent comprises a plurality of agents, and the agent routing arrangement device is further configured to:

[0025] Perform correlation analysis on the plurality of agents by using a language model to obtain a cooperation evaluation result of the plurality of agents.

[0026] According to the cooperation evaluation result, the plurality of agents are determined to cooperate to respond to the query.

[0027] In some possible implementation manners, the agent routing arrangement device is further configured to:

[0028] Obtain at least one of a length, a problem definiteness, a related domain knowledge or a professional term, a lexical richness, and a grammatical structure of the query.

[0029] According to at least one of the length, the problem definiteness, the domain knowledge or the professional term, the lexical richness, and the grammatical structure, it is predicted whether the query is a complex query.

[0030] The agent routing arrangement device is specifically configured to:

[0031] When the query is a complex query, the target intelligent agent that matches the query is retrieved from the intelligent agent registry.

[0032] In some possible implementations, the interactive device is also used for:

[0033] Obtain the context of the query;

[0034] The intelligent agent routing and orchestration device is also used for:

[0035] Rewrite the query based on the context and historical sessions;

[0036] The intelligent agent routing and orchestration device is specifically used for:

[0037] Retrieve and rewrite the target agent from the agent registry to match the query.

[0038] In some possible implementations, the intelligent agent execution device is specifically used for:

[0039] According to the standard operating procedure, load the first agent in the target agent;

[0040] Load the first operation node in the first intelligent agent;

[0041] Obtain the node type and node information of the first operation node. The node information of the first operation node includes at least one of the input and output parameters of the first operation node, the execution logic of the first operation node, and the dependency relationship of the task assigned to the first operation node. The dependency relationship of the task assigned to the first operation node includes the dependency relationship between the first agent and other agents in the target agent, the dependency relationship between the first operation node and other operation nodes in the first agent, or the dependency relationship between the parameters used to execute the first operation node.

[0042] The first operation node is executed based on its node type and node information.

[0043] According to the standard operating procedure, load the second agent in the target agent;

[0044] Load the second operation node in the second intelligent agent;

[0045] Obtain the node type and node information of the second operation node. The node information of the second operation node includes at least one of the input and output parameters of the second operation node, the execution logic of the second operation node, and the dependency relationship of the task assigned to the second operation node. The dependency relationship of the task assigned to the second operation node includes the dependency relationship between the second agent and other agents in the target agent, the dependency relationship between the second operation node and other operation nodes in the second agent, or the dependency relationship between the parameters used to execute the second operation node.

[0046] The second operation node is executed based on its node type and node information.

[0047] In some possible implementations, the intelligent agent execution device is further used for:

[0048] Determine whether the first operation node is the last operation node of the first agent. If not, execute the next operation node of the first agent.

[0049] In some possible implementations, the platform also includes:

[0050] An intelligent agent development device is used to receive metadata configured by a user for the intelligent agent, the metadata including at least one of a prompt template identifier, a knowledge base identifier, a tool identifier, or a model identifier of a language model; and to register the metadata of the intelligent agent to an intelligent agent registration center.

[0051] Thirdly, this application provides a computing device cluster. The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is used to execute instructions stored in the at least one memory to cause the computing device or the computing device cluster to perform the application management method as described in the first aspect or any implementation thereof.

[0052] Fourthly, this application provides a computer-readable storage medium storing instructions that instruct a computing device or a cluster of computing devices to execute the application management method described in the first aspect or any implementation thereof.

[0053] Fifthly, this application provides a computer program product containing instructions that, when run on a computing device or a cluster of computing devices, causes the computing device or cluster of computing devices to execute the application management method described in the first aspect or any implementation thereof.

[0054] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0055] To more clearly illustrate the technical methods of this application, the accompanying drawings used will be briefly described below.

[0056] Figure 1 A schematic diagram of the architecture of an application management platform provided in this application;

[0057] Figure 2 A flowchart of an application management method provided in this application;

[0058] Figure 3 A flowchart illustrating the intelligent agent orchestration and routing in a multi-node dynamic SOP mode provided in this application;

[0059] Figure 4 A flowchart illustrating agent collaboration in a multi-node dynamic SOP mode provided in this application;

[0060] Figure 5 A flowchart of a multi-agent static SOP mode provided in this application;

[0061] Figure 6 A flowchart of a single-agent application mode provided in this application;

[0062] Figure 7 A flowchart of a single-agent application mode provided in this application;

[0063] Figure 8 A flowchart of a single-agent application mode provided in this application;

[0064] Figure 9 A flowchart of a single-agent application mode provided in this application;

[0065] Figure 10 A flowchart of an intelligent agent development method provided in this application;

[0066] Figure 11 A schematic diagram of the structure of a computing device provided in this application;

[0067] Figure 12 This application provides a schematic diagram of the structure of a computing device cluster;

[0068] Figure 13 This application provides a schematic diagram of another computing device cluster structure.

[0069] Figure 14 This is a schematic diagram of another computing device cluster provided in this application. Detailed Implementation

[0070] The terms "first" and "second" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0071] First, some technical terms involved in the embodiments of this application will be introduced.

[0072] Artificial intelligence (AI) is the ability to correctly interpret external data, learn knowledge from that data, and use that knowledge to achieve specific goals and tasks. Currently, a significant branch of AI is natural language processing (NLP). NLP uses algorithms such as machine learning (ML) and deep learning (DL) to build language models (LM), enabling computers to interpret, process, and understand human language.

[0073] A language model is a probability distribution model of words in a natural language. Language models can be used to predict the next most likely word based on the context of the input (e.g., several preceding words). Language models can also be categorized by parameter size into small language models and large language models (LLMs). Small language models can be simply referred to as small models, and large language models as large models. Large models can include, but are not limited to, transformer-based Bidirectional Encoder Representations from Transformers (BERT) models and Generative Pre-trained Transformer (GPT) models.

[0074] Language models, represented by LLMs, can serve as core controllers to construct intelligent agents with human-like decision-making capabilities, also known as AI agents. An intelligent agent is a computer program based on a language model, possessing planning and thinking abilities (also known as planning), memory capabilities, and the ability to use tool functions (or tool use), enabling it to autonomously complete given tasks. Planning ability can involve decomposing large tasks into sub-tasks, such as using Everything of Thought (XoT) to break down large tasks into sub-tasks and planning the execution process. Thinking ability allows for reflection and consideration of the task execution process, deciding whether to continue or terminate the task. Memory can include short-term and long-term memory. Short-term memory refers to the context generated and temporarily stored during task execution, which is cleared after task completion. Long-term memory refers to information retained for extended periods, such as external knowledge bases, typically using vector databases for storage and retrieval. Tools refer to external resources, services, or application programming interfaces (APIs) that the intelligent agent can use to perform specific tasks or enhance functionality. The APIs can include, but are not limited to, tool APIs such as calculators, search tools, code executors, and database query tools. These tools can serve as supplementary components, extending the functionality of the intelligent agent beyond its inherent language generation capabilities.

[0075] Compared to reinforcement learning, intelligent agents possess richer internal world knowledge. Even without training on domain-specific data, they can support more informed actions. Furthermore, LLM provides agents with a natural language interface, facilitating human interaction and offering greater flexibility and interpretability. LLM allows users to input into unrestricted text boxes, obtaining inference results without additional input. Compared to LLM, intelligent agents support more complex interactions and coordination. Specifically, agents possess control structures such as planning, memory, and thinking (e.g., reflection), fully utilizing the inherent reasoning capabilities of LLM to complete tasks. Moreover, by combining tools, plugins, and function calls, agents can perform a wider range of general tasks.

[0076] Agent-based applications can include single-agent or multi-agent architectures. A single-agent architecture involves a single agent solving a problem. A single agent is typically driven by a single language model, and all reasoning, planning, and tool execution are performed by that agent itself. In a single-agent architecture, there is no feedback mechanism from other AI agents, but humans can provide guidance. In a multi-agent architecture, each agent is assigned a persona and possesses various tools to help it complete tasks independently or as part of a team. Some agents also include memory components to store and load information. It's important to note that each agent can use the same language model or a set of different language models. When using the same language model, different roles can be assigned to agents by modifying prompts. For example, by modifying prompts, agents can act as both programming experts and code reviewers. Agents may possess the same tools or different tools.

[0077] In current multi-agent architectures, multiple agents typically execute tasks according to predefined static Standard Operating Procedures (SOPs). An SOP is an internally designed procedure for performing complex tasks within limited time and resources, usually including standard operating steps and requirements. A static SOP refers to an SOP with fixed operating steps (fixed process). Taking a software development process as an example, the software development process can be divided into multiple phases, such as design, development, testing, and documentation. Each atomic task in each phase is completed by predefined roles. These predefined roles may include Chief Product Officer (CPO), Chief Technology Officer (CTO), programmer, test engineer, and designer. Each atomic task in each phase involves multiple communications between several agents. Each phase includes atomic tasks executed in a fixed order. For example, the development phase includes two atomic tasks: the CTO and programmer generate complete code, and the designer and programmer design the graphical user interface. The testing phase includes two atomic tasks: code review by programmers and reviewers, and system testing by programmers and test engineers.

[0078] The aforementioned scheme predefines multiple stages, atomic tasks, and agent roles, and the order of atomic tasks is also fixed. The business process does not support dynamic SOPs, remaining limited to predefined static SOPs and unable to proactively explore new problems. Dynamic SOPs, on the other hand, refer to SOPs with variable operation steps or execution order. When facing diverse user needs and complex business environments, they often exhibit insufficient flexibility and poor adaptability. Specifically, issues such as user queries of varying difficulty, different types of business agents, and complex iterative interactions between multiple agents or agent types make it difficult for related technologies to achieve efficient and flexible application development and operation.

[0079] In view of this, this application provides an application management method based on intelligent agents. This method can be executed by an application management platform. The application management platform can be a software system, which can be a standalone software system or a software system integrated with other software. In some examples, the application management platform can be a framework for developing and running multiple types / multiple intelligent agent applications that support dynamic / static business processes. The software system can be provided to customers as a software package for self-deployment, or provided to users as a cloud service, with users subscribing to the cloud service to use the corresponding capabilities or functions. The aforementioned software system can be deployed in a computing device cluster, where the computing device cluster executes the program code of the software system, thereby executing the application management method of this application.

[0080] The application management platform includes an interaction device, an agent routing and orchestration device, and an agent execution device. The interaction device receives user queries, and the agent routing and orchestration device identifies the agent requested by the user based on the query. If no agent is requested in the query, the agent routing and orchestration device retrieves the target agent that matches the query. The agent routing and orchestration device can assign tasks to the target agent and determine the execution order or dependencies of the tasks. Based on the execution order or dependencies of the tasks and the target agent, the agent routing and orchestration device generates Standard Operating Procedures (SOPs) using a language model. The agent execution device can load the target agent according to the SOP, execute the operation nodes within the target agent, and obtain the answer.

[0081] This method can identify the intent of user queries, retrieve target agents matching the queries, and assign tasks to the target agents. By orchestrating the target agents, static or dynamic Standard Operating Procedures (SOPs) can be generated to meet diverse business needs and complex business environments, demonstrating high flexibility and adaptability. Furthermore, by applying dynamic and static SOP orchestration and execution to multiple types of agents, development efficiency can be improved.

[0082] To make the technical solution of this application clearer and easier to understand, the application management platform of this application will be described below with reference to the accompanying drawings.

[0083] See Figure 1 The schematic diagram of the application management platform architecture shown indicates that the application management platform 10 includes an interaction device 102, an agent routing and orchestration device 104, and an agent execution device 106. Further, the application management platform 10 may also include an agent registration center 108. The interaction device 102 serves as the business entry point and can be an integrated development environment (IDE), a browser, or an IDE plugin or browser plugin. In some instances, the interaction device 102 can also be an application (APP) or operating system (OS) on a mobile phone, computer, or other terminal. The agent routing and orchestration device 104 can be a plugin, extension, or functional module for implementing agent routing and orchestration. The agent execution device 106 is an agent executor. The agent registration center 108 can be an AI application marketplace. The agent registration center 108 can receive registration requests from developers, which are used to register agents developed by developers. It then registers the metadata of the agents based on the registration requests and publishes the registered agents for user access.

[0084] Interaction device 102 is used to receive user queries. Agent routing orchestration device 104 is used to identify the agent requested by the user based on the query; when the query does not request an agent, it retrieves the target agent that matches the query. Agent routing orchestration device 104 is also used to assign tasks to the target agent, determine the execution order or dependencies of the tasks, and generate SOPs based on the execution order or dependencies of the tasks and the target agent. In this application, agent routing orchestration device 104 can generate SOPs in various ways. One way is through language model generation. Specifically, agent routing orchestration device 104 is used to input the execution order and dependencies of the tasks assigned to the target agent into a language model, such as LLM, and generate SOPs through language model reasoning. Another way is that agent routing orchestration device 104 can directly retrieve SOPs with existing successful experiences from the knowledge base. Alternatively, agent routing orchestration device 104 can generate SOPs based on the retrieval augmented generation (RAG) algorithm. Another approach is for the agent routing orchestration device 104 to generate SOPs based on rules. For example, the agent routing orchestration device 104 can generate SOPs based on the dependencies of interface parameters or by direct sequential invocation. The agent execution device 106 is used to load the target agent according to the SOP, execute the operation nodes in the target agent, and obtain a response.

[0085] The intelligent agent routing and orchestration device 104 may include multiple modules. For example... Figure 1 As shown, the agent routing orchestration device 104 includes a dialogue management module 1042, a routing and orchestration control module 1044, and an agent discovery and teaming module 1046. In some possible implementations, the agent routing orchestration device 104 may not include the aforementioned dialogue management module. For example, if the agent routing orchestration device 104 does not require a memory function, the dialogue management module may not be included. Figure 1 The structure shown is merely an illustrative division of the application management platform 10. In other possible implementations of this application, other division methods may also be adopted. For example, the above-mentioned devices or modules may be split or combined to form new devices or modules.

[0086] The modules of the intelligent agent routing and orchestration device 104 are described in detail below.

[0087] The dialogue management module 1042 is used to implement session management, context management, and / or operational tracking. Session management may include, but is not limited to, session creation and maintenance. Context management includes saving and clearing dialogue history. Operational tracking refers to the implementation of data tracking points by operations teams for data analysis.

[0088] The routing and orchestration control module 1044 is used to implement unified routing and orchestration scheduling for one or more agents. The routing or orchestration Standard Operating Procedures (SOPs) can be distributed to the agent execution device 106 for execution. For simple queries or single-agent routing scenarios, the routing and orchestration control module 1044 can directly interface with the agent execution device 106. Single-agent applications can include codeless applications, low-code applications, or high-code applications. For complex queries, or scenarios requiring multi-agent discovery and orchestration, the routing and orchestration control module 1044 can first obtain the team information of one or more agents from the agent discovery and teaming module 1046, then generate an SOP and finally distribute it to the agent execution device 106. The SOPs can take the form of single-node SOPs (single-agent SOPs), multi-node static SOPs (or multi-agent static SOPs), and multi-node dynamic SOPs (or multi-agent dynamic SOPs).

[0089] The agent discovery and team formation module 1046 is used to retrieve and query matching target agents, determine whether target agent collaboration is needed, and plan and decompose collaboration among multiple target agents. Specifically, the agent discovery and team formation module 1046 retrieves the user's query-matched target agents from the agent registry center 108.

[0090] like Figure 1As shown in step 1, the interaction device 102 receives a user's query (denoted as query), where the query may or may not specify an agent. When the query includes an application identifier (app_id) or an agent identifier (agent_id), it indicates that the query specifies an agent. The application identifier is unique; for example, it can be a universally unique identifier (UUID) for the application. In some examples, the application identifier can be the application name (app_name). Similarly, the agent identifier is unique; for example, it can be the agent's UUID or agent name (agent_name). Next, referring to step 2, the interaction device 102 sends the query to the agent routing orchestration device 104. Optionally, the interaction device 102 may also send the application identifier to the agent routing orchestration device 104. When a user specifies multiple agents or the query is complex, the agent routing orchestration device 104 can retrieve target agents matching the query from the agent registry 108. For example, referring to steps 3a and 4a, when the user specifies multiple agents, the agent routing orchestration device 104 sends an application identifier, and the agent registry 108 can return the application type (app_type) and application information (app_info). The application type indicates whether the application is a no-code application, a low-code application (such as workflow), or a high-code application (such as agent). The application information can include any one or any combination of the following: application description (app_description), application provider (app_provider), application developer (app_creator), application version (app_version), application creation time (created_time), application update time (updated_time), and user-defined instructions (instruction). Referring to steps 3b and 4b, the agent routing orchestration device 104 can send a query to the inference service, which can plan the agents and return the execution order. The execution order can be represented by a workflow, or described in natural language, such as agent A as the first agent and agent B as the second agent. In some examples, the execution order can also be described using a Domain Specific Language (DSL). A DSL is a computer language specific to a particular problem domain, designed to solve specific types of problems in that domain. The agent routing orchestration device 104 can generate a Standard Operating Procedure (SOP) based on the tasks assigned to the target agent, their execution order, and the target agent itself.

[0091] As shown in step 5, the agent routing orchestration device 104 can send a query and application identifier to the agent execution device 106 according to the standard operating procedure (SOP). The application identifier can be the application identifier in the SOP. In some possible implementations, the agent routing orchestration device 104 can also send a context to the agent execution device 106, so that the agent execution device 106 can execute the operation nodes in the agent in conjunction with the context. The agent execution device 106 can load the corresponding agent according to the application identifier and execute the operation nodes in the agent to obtain a response. For example, the agent execution device 106 can first execute the operation nodes in the chat agent. After the operation nodes in the chat agent are executed, the next agent can be executed according to the SOP, such as the code agent or the development agent. After the operation nodes in the code agent are executed, the operation nodes in the check agent or the code inspection agent can be executed. After the operation nodes in the dev agent are executed, the request agent or the cloud agent can be executed. Figure 1 As shown in steps 6 to 11, the agent execution device 106 can send a prompt identifier (prompt_id) to the prompt word center and receive a prompt template returned by the prompt word center. Furthermore, the agent execution device 106 can also send search terms to the knowledge base, and when sending search terms, it can also send a knowledge base identifier (knowledgebase_id). Further, the agent execution device 106 can also send a knowledge base table identifier (knowledgeasset_id). In this way, the agent execution device 106 can perform a search on the corresponding knowledge base or knowledge base table to obtain search results, and the agent execution device 106 receives the search results from the knowledge base. The agent execution device 106 can also send a tool name and tool invocation parameters to the toolset to initiate a tool invocation request. The agent execution device 106 receives the execution results returned by the tool. It should be noted that before invoking a tool, the agent execution device 106 can also request the tool name and tool invocation parameters from the inference service. In this application, the intelligent agent execution device 106 can obtain answers by interacting once or multiple times with a reasoning service (e.g., a language model in the reasoning service), a toolset, or a knowledge base.

[0092] The agent execution device 106 can assemble a prompt according to the prompt template, based on the retrieval results, execution results, query, and context. This prompt can be a complete prompt. As in steps 12 and 13, the agent execution device 106 can send a prompt to the inference service. When sending the prompt, the agent execution device 106 can also send a model identifier to the inference service. The inference service can then use this model identifier to call the corresponding language model (such as LLM) for inference. The inference service can return a response to the agent execution device 106. This response can be a response from the language model. The agent execution device 106 can also send a response to the agent routing orchestration device 104. This response can also be returned to the dialogue management module 1042 for storage and later use. It should be noted that... Figure 1 The example illustrates how the response serves as a reasoning service. In practical applications, the response can also be a knowledge base retrieval result or a tool call result. For instance, if the similarity between the current query and historical queries reaches a threshold, the agent execution device 106 can return a knowledge base retrieval result. The agent routing orchestration device 104 can return the response to the interaction device 102, which then presents the response to the user, as shown in steps 14 to 16. The agent routing orchestration device 104 can return the response to the interaction device 102 via a pass-through method.

[0093] based on Figure 1 The application management platform 10 shown in this application also provides an application management method. This application management method allows for the loading and running of different agent application modes simply through configuration without intrusive modification of the business code during agent runtime. In some examples, agent application modes may include the following multiple modes: (1) a multi-node dynamic SOP mode where the user submits a complex query; (2) a user-specified multi-agent / multi-node static SOP mode; and (3) a specified single-agent application mode. It should be noted that in practical applications, the above application modes can also be combined to form a toolbox interface mode. The application management methods for different modes are described below.

[0094] See Figure 2 The flowchart illustrates an application management method, which can be executed by an application management platform 10. The application management platform 10 includes an interaction device 102, an agent routing and orchestration device 104, and an agent execution device 106. Further, the application management platform 10 may also include an agent registration center 108. The method includes the following steps:

[0095] S202, the interactive device 102 receives the user's query.

[0096] Specifically, the interactive device 102 can present a question-and-answer interactive interface to the user, receiving queries (or questions) entered by the user through the interface. These queries can be based on natural language or programming languages. The interactive interface can include a question-and-answer area or an editing area. Users can enter queries based on natural language in the question-and-answer area, or queries based on SQL, eXtensible Markup Language (XML), YAML, JavaScript Object Notation (JSON), MD, JAVA, or Python in the editing area. Queries can be applied throughout the entire application lifecycle, including application development and operation. For example, in the requirements gathering phase, queries can be used for requirements management, defect management, and project management; in the design phase, queries can be used for view design, design for X (DFX) design, API design, and data table design; in the code development phase, queries can be used for code continuation / completion, unit testing (UT) generation, code inspection / repair, code explanation / commenting, and code submission. During development, queries can also be used to access research and development knowledge. In the code review phase, queries can also be used for code review. Similarly, during the testing phase, queries are used to assist in test design, test case review, test script generation, or test execution and evaluation. For API design, development, and operations, queries are also used for managing cloud service resources, application cloud deployment design, code / script development, and cloud service operation / maintenance.

[0097] S204. The intelligent agent routing and orchestration device 104 identifies the intelligent agent requested by the user based on the query. If no intelligent agent is requested in the query, S206 is executed.

[0098] Specifically, the query request body sent by the front-end interaction device 102 may include a list of agents, which can be a list of agent_ids or agent_names selected by the user. The agent routing orchestration device 104 can parse the query request body to obtain the agent list. When the agent list is empty, it indicates that no agent was requested in the query.

[0099] In some possible implementations, the agent routing and orchestration device 104 can also identify the agent requested by the user through keyword matching. For example, if the user uses keywords when specifying the agent, such as the keyword "@" indicating mention, it can be identified whether the user's query requests an agent. If so, it can also identify the requested agent. The requested agent can be determined based on the field after "@" in the query.

[0100] In other possible implementations, the agent routing orchestration device 104 can also identify the agent requested by the user through semantic recognition based on the query. Specifically, the agent routing orchestration device 104 can use a semantic recognition model to perform semantic recognition on the query and identify the agent requested by the user.

[0101] When the identification result indicates that no agent has been requested, for example, the query does not include the keywords used by the specified agent, nor does it include the application identifier or agent identifier, it means that the user has not requested an agent, and the agent routing and orchestration device 104 can execute S206.

[0102] S206, the intelligent agent routing and orchestration device 104 determines whether the query is a complex query. If the query is a complex query, S208 is executed.

[0103] In a specific implementation, the intelligent agent routing orchestration device 104 can obtain at least one of the following: query length, question clarification, domain knowledge or professional terms involved, vocabulary richness, and grammatical structure. Then, based on at least one of the following: query length, question clarification, domain knowledge or professional terms involved, vocabulary richness, and grammatical structure, the intelligent agent routing orchestration device 104 uses a language model to predict whether the query is a complex query.

[0104] In some possible implementations, the agent routing orchestration device 104 can also determine whether a query is complex through question-and-answer communication using a language model. For example, the agent routing orchestration device 104 can send a hint to the LLM, which may include the user's query. The LLM can then infer whether the query is complex based on the hint. The agent routing orchestration device 104 receives the inference result from the LLM, thereby determining whether the query is complex.

[0105] It should be noted that step S206 above is optional, and the application management method of this application may not need to execute step S206. For example, the agent routing and orchestration device 104 can perform agent discovery and teaming for queries from agents that have not requested them.

[0106] S208, the agent routing and orchestration device 104 retrieves and queries the target agent that matches the target agent.

[0107] In specific implementation, the agent registry 108 can include agent applications registered by developers, including but not limited to GPT-based no-code applications, workflow-based low-code applications, and agent-based high-code applications. The agent routing orchestration device 104 can retrieve target agents matching the query from the agent registry 108. To improve retrieval efficiency and accuracy, the interaction device 102 can also obtain the query context, which can be obtained from the business system (or application environment). For example, if the query is entered in the business's question-and-answer area, the interaction device 102 can also obtain business-related context from the business window of the business system. Business-related context can include files opened in the business window and content loaded in the business window. For example, in chart applications, business-related context can include open data tables and graphs loaded in the data tables. The agent routing orchestration device 104 can rewrite the query based on this context and historical sessions. Accordingly, the agent routing orchestration device 104 retrieves target agents matching the rewritten query from the agent registry 108.

[0108] The target agent may include one or more agents. When the target agent includes multiple agents, the agent routing orchestration device 104 uses a language model to perform association analysis on the multiple agents and obtain the cooperation evaluation results of the multiple agents. The agent routing orchestration device 104 can determine the cooperation of multiple agents to respond to queries based on the cooperation evaluation results.

[0109] S210, the intelligent agent routing and orchestration device 104 assigns tasks to the target intelligent agent and determines the execution order or dependencies of the tasks.

[0110] Execution order refers to the sequence in which tasks are executed. Dependency relationships include the relationships between data that tasks depend on for execution. For example, one agent's execution of a task depends on the execution result of another agent. In this case, the dependency relationship can also reflect the execution order.

[0111] In practical implementation, the agent routing orchestration device 104 can use task decomposition and scheduling algorithms to assign tasks to the target agent, ensuring the rationality and effectiveness of task allocation and avoiding task conflicts and resource waste. In some possible implementations, the agent routing orchestration device 104 can also assign tasks to the target agent through a question-and-answer process using a language model.

[0112] In some possible implementations, the metadata of the agents stored in the agent registry 108 may include the agent's upstream and downstream agents. Based on this, when the agent routing orchestration device 104 retrieves a target agent from the agent registry 108, it can obtain the target agent's upstream and downstream agents based on the agent's metadata, thereby retrieving multiple target agents. Correspondingly, when assigning tasks to target agents, the agent routing orchestration device 104 can also combine the upstream and downstream relationships of the agents to perform task allocation.

[0113] S212, the agent routing and orchestration device 104 generates an SOP based on the execution order or dependencies of the tasks and the target agent.

[0114] Standard Operating Procedures (SOPs) are used to describe the action sequence, communication protocols, and coordination mechanisms of each agent in the target intelligent agent. In some examples, SOPs include task descriptions, timelines, resource allocation, and exception handling mechanisms, which ensure the efficiency and reliability of the execution process.

[0115] The agent routing orchestration device 104 can generate Standard Operating Procedures (SOPs) in several ways. One method is through language model generation. Specifically, the agent routing orchestration device 104 inputs the execution order and dependencies of tasks assigned to the target agent into a language model, such as an LLM (Low-Level Model), and generates the SOP through language model reasoning. Another method is for the agent routing orchestration device 104 to directly retrieve SOPs with existing successful experiences from the knowledge base. Alternatively, the agent routing orchestration device 104 can generate SOPs based on the retrieval augmented generation (RAG) algorithm using the retrieval results. Yet another method is for the agent routing orchestration device 104 to generate SOPs based on rules. For example, the agent routing orchestration device 104 can generate SOPs based on the dependencies of interface parameters or by direct sequential invocation. SOPs generated through the above methods can be dynamic SOPs, not limited to static SOPs.

[0116] S214. The agent execution device 106 loads the target agent according to the SOP, executes the operation nodes in the target agent, and obtains a response.

[0117] Specifically, the agent execution device 106 can load the first agent in the target agent according to the standard operating procedure (SOP), and then load the first operation node in the first agent. The agent execution device 106 can obtain the node type and node information of the first operation node. The node information of the first operation node includes at least one of the following: the input / output parameters of the first operation node, the execution logic of the first operation node, and the dependency relationship of the task assigned to the first operation node. The dependency relationship of the task assigned to the first operation node includes the dependency relationship between the first agent and other agents in the target agent, the dependency relationship between the first operation node and other operation nodes in the first agent, or the dependency relationship between the parameters used to execute the first operation node. The agent execution device 106 executes the first operation node according to the node type and node information of the first operation node.

[0118] The agent execution device 106 can also load a second agent from the target agent according to the standard operating procedure (SOP) and execute operation nodes in the second agent. The operation nodes in the second agent are called second operation nodes. The specific implementation of the agent execution device 106 executing the second operation node is similar to the specific implementation of the agent execution device 106 executing the first operation node. For example, the agent execution device 106 can load the second operation node from the second agent and obtain the node type and node information of the second operation node. The node information of the second operation node includes at least one of the following: the input / output parameters of the second operation node, the execution logic of the second operation node, and the dependency relationship of the task allocated to the second operation node. The dependency relationship of the task allocated to the second operation node includes the dependency relationship between the second agent and other agents in the target agent, the dependency relationship between the second operation node and other operation nodes in the second agent, or the dependency relationship between the parameters used to execute the second operation node. The agent execution device 106 executes the second operation node according to the node type and node information of the second operation node.

[0119] In some possible implementations, each agent may include one or more operation nodes. Based on this, before loading the second agent in the target agent according to the SOP, the agent execution device 106 determines whether the first operation node is the last operation node of the first agent. If not, the agent execution device 106 executes the next operation node of the first agent.

[0120] Based on the above description, this application provides an application management method. This method identifies whether a user's query specifies an agent. If no agent is specified, it retrieves a target agent matching the query and assigns tasks to the target agent. By orchestrating the target agents, static or dynamic standard operating procedures (SOPs) can be generated to meet diverse business needs and complex business environments, exhibiting high flexibility and adaptability.

[0121] To make the technical solution of this application clearer and easier to understand, the process of agent orchestration and routing in multi-node dynamic SOP mode is explained in detail below.

[0122] See Figure 3 The diagram illustrates a process for agent orchestration and routing in a multi-node dynamic SOP mode. This method is executed by an application management platform 10, which includes an interaction device 102, an agent routing orchestration device 104, an agent execution device 106, and an agent registration center 108. The agent routing orchestration device 104 includes a dialogue management module 1042, a routing and orchestration control module 1044, and an agent discovery and teaming module 1046. The method includes the following steps:

[0123] S302, the interactive device 102 receives the user's query.

[0124] S304, the dialogue management module 1042 in the intelligent agent routing and orchestration device 104 performs dialogue management based on the query.

[0125] Specifically, the dialogue management module 1042 creates a session identifier (ID), which associates each user interaction with the session ID to ensure continuity within the same session. The dialogue management module 1042 can use an in-memory database to maintain session state, ensuring efficient read and write operations. The in-memory database can be, but is not limited to, Redis or other in-memory databases. It should be noted that the dialogue management module 1042 can also maintain session state in a relational database or NoSQL database. When the in-memory database fails, the dialogue management module 1042 can recover the session state from the relational database or NoSQL database.

[0126] Furthermore, the dialogue management module 1042 can also save and manage historical multi-turn dialogues. Specifically, the user's historical queries (historical questions) and the language model's answers are stored in the session state object for reference when processing the current query. The dialogue management module 1042 can periodically clean and archive historical session data to ensure the validity of data storage and system performance.

[0127] S306, the routing and orchestration control module 1044 in the intelligent agent routing and orchestration device 104 identifies whether the user has specified an intelligent agent.

[0128] In this embodiment, the user can specify an agent using keywords. These keywords can include, but are not limited to, "@". Therefore, whether the user specifies an agent can include the following scenarios: a) @ a single agent; b) @ multiple agents; c) not @ any agent.

[0129] When a user mentions a single agent, it indicates that the agent application mode is a single-agent mode. If the user mentions a single agent, the agent routing orchestration device 104 determines the corresponding processing path based on the agent's name or role and routes the query to that agent. When a user mentions multiple agents, it indicates that the agent application mode is a multi-agent / multi-node static SOP mode. If the user mentions multiple agents, the agent routing orchestration device 104 initiates multi-agent planning / decomposition and starts the multi-agent fixed SOP collaborative processing flow. If the user does not mention any agent, the subsequent steps continue.

[0130] S308, Routing and Orchestration Control Module 1044 rewrite query.

[0131] The routing and orchestration control module 1044 can automatically supplement and rewrite queries using context and historical sessions (previous user interaction records) to ensure that the rewritten queries are semantically clearer and more specific. In some cases, the routing and orchestration control module 1044 can also request clarification from the user. For example, the routing and orchestration control module 1044 can use the interaction device 102 for multi-turn interactions to ensure an accurate understanding of the user's needs.

[0132] and Figure 2 Similar to the embodiments shown, S308 above is an optional step in this embodiment. The application management method of this application may also omit the above steps.

[0133] S310 and the routing and orchestration control module 1044 perform complexity analysis on the rewritten query.

[0134] In this embodiment, the routing and orchestration control module 1044 can perform complexity analysis on the rewritten query based on at least one of the following: query length, question specificity, relevant domain knowledge or technical terms, lexical richness (also known as lexical diversity), and grammatical structure. For example, the routing and orchestration control module 1044 can perform complexity analysis on the query through rule matching based on the query length, question specificity, relevant domain knowledge or technical terms, lexical richness, and grammatical structure.

[0135] In some possible implementations, the routing and orchestration control module 1044 can perform complexity analysis on the rewritten query through model question answering. For example, the routing and orchestration control module 1044 can input the rewritten query into the language model and request the language model to return the complexity of the rewritten query.

[0136] S312, the routing and orchestration control module 1044 determines whether the query is a simple query or a complex query. If it is a simple query, S314 is executed; if it is a complex query, S316 is executed.

[0137] Specifically, the routing and orchestration control module 1044 can determine whether a query is simple or complex based on the complexity analysis results. When the complexity analysis is qualitative, the routing and orchestration control module 1044 can directly determine whether the query is simple or complex based on the complexity analysis results. When the complexity analysis is quantitative, the routing and orchestration control module 1044 can compare the complexity with a threshold to determine whether the query is simple or complex. For example, if the complexity is greater than or equal to the threshold, the query is considered complex. If the complexity is less than the threshold, the query is considered simple.

[0138] S314, the routing and orchestration control module 1044 routes the query to the agent execution device 106, and the agent execution device 106 switches to the single agent application mode.

[0139] The single-agent application mode can include multiple modes, such as direct invocation mode, SOP mode, or reason-action (ReAct) mode.

[0140] S316, Agent Discovery and Team Formation Module 1046 retrieves target agents that match the rewritten query based on the rewritten query.

[0141] The agent discovery and team formation module 1046 retrieves target agents matching the rewritten query from the agent registry 108. The agent registry 108 maintains a list of agents, recording at least one of each agent's capabilities, areas of expertise, available tools, or processing speed. When retrieving target agents, the agent discovery and team formation module 1046 considers the agent's expertise and relationships. Specifically, it can use vector similarity search, keyword matching, or knowledge graphs to retrieve target agents. It should be noted that the agent registry 108 can establish detailed capability and relationship graphs for each agent to support rapid agent retrieval and matching.

[0142] S318, the agent discovery and teaming module 1046 performs association analysis on the target agents.

[0143] Specifically, when there are multiple target agents, the agent discovery and team formation module 1046 can invoke a language model (such as LLM) to perform association analysis on the multiple agents. The routing and orchestration control module 1044 can use LLM to perform in-depth analysis of user queries, including analyzing the query scope, complexity, and degree of knowledge domain overlap, thereby assessing whether multi-agent collaboration is necessary. Furthermore, the agent discovery and team formation module 1046 can also, based on historical successful experiences of multi-agent combined invocations, use algorithms such as cooperative game theory, collaborative filtering algorithms, Frequent Pattern Growth (FPGrowth) or Prefix-Projected Pattern Growth (PrefixSpan) to evaluate the collaboration potential and effectiveness between agents, ensuring efficient collaboration.

[0144] S320, the agent discovery and team formation module 1046 determines whether to perform agent collaboration based on the association analysis results. If yes, then execute S322; otherwise, execute S314.

[0145] Specifically, if the LLM evaluation requires collaboration, or if the collaboration potential between multiple agents is high and effective based on historical successful experiences of multi-agent combined calls, then agent collaboration is determined to occur. The agent discovery and teaming module 1046 can execute S322 to perform multi-agent collaboration. If the LLM evaluation does not require collaboration, or if the collaboration potential between multiple agents is low and effective based on historical failed experiences of multi-agent combined calls, then agent collaboration is determined not to occur. The agent discovery and teaming module 1046 can execute S314 to route the query to the agent execution device 106, which switches to single-agent application mode. A single agent processes the request through its API or service interface and returns the result. The agent execution device 106 can monitor the agent's processing to ensure the accuracy and timeliness of the response.

[0146] S322, Agent Discovery and Teaming Module 1046 assigns tasks to each agent.

[0147] Specifically, the agent discovery and teaming module 1046 can use task decomposition and scheduling algorithms to assign tasks to each agent, thereby clarifying the responsibilities and expected outputs of each agent, ensuring the rationality and effectiveness of task allocation, and avoiding task conflicts and resource waste.

[0148] S324, Agent Discovery and Team Formation Module 1046 determines the execution order and dependencies of tasks.

[0149] Specifically, the agent discovery and teaming module 1046 formulates a collaborative scheme among agents through a language model (such as LLM) to determine the execution order and dependencies of tasks.

[0150] S326, the routing and orchestration control module 1044 generates SOPs based on the execution order and dependencies of tasks and the target agent.

[0151] The routing and orchestration control module 1044 can load or use LLM to generate SOPs. The SOPs detail the action sequence, communication protocols, and coordination mechanisms for each agent. In this embodiment, the SOPs include task descriptions, time scheduling, resource allocation, and exception handling mechanisms to ensure the efficiency and reliability of the execution process.

[0152] S328, the intelligent agent execution device 106 executes the business process according to the multi-agent dynamic SOP.

[0153] S330 and agent execution device 106 execute business processes according to the multi-agent static SOP.

[0154] The specific implementation of the intelligent agent execution device 106 executing the business process according to the multi-agent dynamic SOP, or the intelligent agent execution device executing the business process according to the multi-agent static SOP, and the intelligent agent execution device 106 executing the business process of the single-agent application mode will be described in detail with reference to the embodiments.

[0155] The above describes the SOP generation process. The following section introduces the dynamic SOP execution process.

[0156] See Figure 4 The flowchart shown illustrates an agent collaboration method in a multi-node dynamic SOP mode. This method includes the following steps:

[0157] S402, the agent execution device 106 loads the next agent in the target agent according to the SOP.

[0158] The agent execution device 106 loads and executes each agent progressively through a cyclic mechanism, ensuring task continuity and efficient system operation. Specifically, the agent execution device 106 can load the next agent from the target agent group according to the Standard Operating Procedure (SOP). During the initialization phase, the next agent can be the first agent executed in the SOP, such as the first agent. During the execution phase, the next agent can be the next agent to be executed in the SOP, such as the second agent.

[0159] S404, The agent execution device 106 loads the next operation node in the agent.

[0160] Specifically, when an agent is loaded, the agent execution device 106 can load the agent's operation nodes to achieve refined task execution. For example, when the next agent is the first agent, the agent execution device 106 can load the first operation node of the first agent sequentially according to the standard operating procedure (SOP). As another example, when the next agent is the second agent, the agent execution device 106 can load the second operation node of the second agent sequentially according to the SOP. When loading operation nodes, the next operation node can be loaded only after the current operation node has been successfully executed.

[0161] S406, the intelligent agent execution device 106 acquires the node type and node information of the operation node.

[0162] Node types can be operation types, including knowledge query, data processing, API call, or model inference. Node information can include input and output parameters, execution logic, and dependencies of tasks assigned to the operation node. Dependencies of tasks assigned to the operation node include dependencies between the agent and other agents in the target agent, dependencies between the operation node and other operation nodes in the agent, or dependencies between the parameters used to execute the operation node.

[0163] In some possible implementations, the operation node of each agent is defined in a configuration file or database using JavaScript object notation (JSON) or YAML metadata to define the node type and node information. Based on this, the agent execution device 106 can obtain the node type and node information by reading the aforementioned metadata.

[0164] S408, the intelligent agent execution device 106 executes the operation node according to the node type and node information of the operation node.

[0165] S409, the intelligent agent execution device 106 stores the intermediate results of the operation node into the execution variable pool.

[0166] Node execution is the core of the operation of the agent execution device 106. The agent execution device 106 ensures that each operation node can execute correctly and records intermediate results. Specifically, the agent execution device 106 can maintain a global variable pool to store the intermediate results of each operation node. These intermediate results can include result data, metadata (such as timestamps and execution status), and dependencies, facilitating subsequent invocation and analysis by operation nodes. Furthermore, the agent execution device 106 can monitor the execution process of each operation node, recording execution time, resource usage, and execution results.

[0167] S410, the intelligent agent execution device 106 determines whether the execution was successful. If yes, then execute S412; otherwise, then execute S414.

[0168] Each operation node returns a status code and execution log after execution. The intelligent agent execution device 106 can determine whether the execution was successful based on the status code.

[0169] S412, the agent execution device 106 determines whether the operation node is the last operation node of the agent. If yes, then execute S416; if no, then execute S418.

[0170] Specifically, each agent can use a Directed Acyclic Graph (DAG) structure to manage the execution order of operation nodes, ensuring the dependencies and order of operation node execution. Based on this, the agent execution device 106 can identify whether the current operation node is the agent's last operation node according to the DAG.

[0171] S414, the intelligent agent execution device 106 executes the fault handling logic.

[0172] Specifically, the intelligent agent execution device 106 can record the cause of failure and execute fault handling logic through fault handling callbacks. Fault handling includes any one or any combination of retry mechanisms, error logging, skip execution, or notification mechanisms. Through fault handling, it can be ensured that problems are handled and reported in a timely manner. It should be noted that the fault handling callback method is business-related.

[0173] S416, the agent execution device 106 determines whether the agent is the last agent in the SOP. If yes, execute S420; otherwise, execute S422.

[0174] S418, the intelligent agent execution device 106 updates the node status and then returns to S404.

[0175] S420, the intelligent agent execution device 106 determines whether the user's query has been resolved. If yes, then execute S424; if no, then execute S426.

[0176] S422, the agent execution device 106 updates the agent state and then returns to S402.

[0177] S424, the agent execution device 106 returns the answer obtained from the agent execution to the routing and orchestration control module 1044.

[0178] S426, the agent execution device 106 instructs the agent discovery and teaming module 1046 to re-plan.

[0179] After all agents have completed their execution, the agent execution device 106 can determine whether the user's query (or question) has been resolved, and decide whether replanning or further processing is necessary. If the question is resolved, the generated answer can be returned to the user, completing the entire multi-agent collaboration process. If the question is not resolved, the process can return to the agent discovery and teaming module 1046, which will then re-plan / decompose the process. By recording previous execution processes and results, the agent discovery and teaming module 1046 adjusts its planning strategies and execution paths, learning from past failures to ensure that new execution paths can effectively solve the problem.

[0180] The above embodiments provide a detailed description of the multi-node dynamic SOP mode where the user does not mention any agents. The following describes the multi-node / multi-agent static SOP mode when the user mentions multiple agents.

[0181] When a user mentions multiple agents, pre-defined multi-agent orchestration logic can be used. This pre-defined multi-agent orchestration logic may include:

[0182] 1. Execute multiple agents in sequence: Provide the user's query and the output of the previous agent to the current agent, and return the output of the last agent to the user.

[0183] 2. Summarize after executing multiple agents in parallel: Provide the user's query to each agent, summarize the output of each agent, and return the summary result to the user.

[0184] It should be noted that a Standard Operating Procedure (SOP) is a pre-written workflow, which may include sequential execution logic and / or parallel execution logic. The sequential and parallel execution logic mentioned above is independent of the static and dynamic nature of the SOP.

[0185] Figure 5A flowchart of a multi-agent static SOP mode is shown. For the second logic mentioned above, the summary step can also be considered as an agent; therefore, both logics can be summarized using the same process. This process specifically includes the following steps:

[0186] S501, the routing and orchestration control module 1044 loads the DAG through dialogue management based on the user's query.

[0187] Here, the DAG can be the DAG corresponding to the built-in multi-agent static SOP.

[0188] S502, the agent execution device 106 loads the next agent according to the DAG.

[0189] S504, The agent execution device 106 loads the next operation node in the agent.

[0190] S506, the intelligent agent execution device 106 acquires the node type and node information of the operation node.

[0191] S508, the intelligent agent execution device 106 executes the operation node according to the node type and node information of the operation node.

[0192] S510, the intelligent agent execution device 106 records the intermediate results of the execution operation node to the execution variable pool.

[0193] S512, the intelligent agent execution device 106 determines whether the execution was successful. If yes, then execute S514; otherwise, then execute S516.

[0194] S514, the agent execution device 106 determines whether the operation node is the last operation node of the agent. If yes, then execute S518; otherwise, return to execute S506.

[0195] S516, the intelligent agent execution device 106 executes the fault handling logic.

[0196] S518, the agent execution device 106 determines whether the agent is the last agent in the DAG. If yes, execute S520; otherwise, return to execute S504.

[0197] S520, the agent execution device 106 returns the answer obtained from the agent execution to the routing and orchestration control module 1044.

[0198] in, Figure 5 The specific implementation of the relevant steps in the illustrated embodiments and Figure 4 The specific implementations of the relevant steps in the illustrated embodiments can be referenced from each other, and will not be repeated here.

[0199] When a user specifies an agent, or when it is recognized that only one agent is needed, the problem can be solved using a single-agent application mode. Specifically, if the user's query is identified as a simple query, the agent execution device 106 can directly invoke the capabilities of the language model (such as LLM) to answer it.

[0200] See Figure 6 The flowchart shown represents a single-agent application mode, specifically including the following steps:

[0201] S602, the intelligent agent execution device 106 assembles the prompt based on the context of the query.

[0202] S604, the intelligent agent execution device 106, based on the prompts, calls the language model to perform reasoning and obtain an answer.

[0203] S606, the intelligent agent execution device 106 returns a response to the routing and orchestration control module 1044.

[0204] Optionally, the agent execution device 106 can also add the answer to the history of the conversation.

[0205] In other embodiments, if it is recognized that a query can be resolved using an agent that is publicly available, such as a HighCode Agent, it can be implemented directly through routing and invocation.

[0206] See Figure 7 The flowchart shown represents a single-agent application mode, specifically including the following steps:

[0207] S701, the routing and orchestration control module 1044 routes the query to the specified agent based on the query and the agent specified in the query.

[0208] S702, the agent execution device 106 calls the agent to generate a query answer.

[0209] S704, the intelligent agent execution device 106 returns a response to the routing and orchestration control module 1044.

[0210] Optionally, the agent execution device 106 can also add the answer to the history of the conversation.

[0211] Figure 6 , Figure 7The direct invocation pattern of single-agent applications has been introduced. The reason-action (ReAct) pattern, or SOP pattern, will be discussed below. ReAct is a language model paradigm that uses natural language reasoning to solve complex tasks. ReAct is designed for tasks that allow LLMs to perform certain operations. For example, an LLM can interact with an external API to retrieve information. When a question is posed, the LLM can choose to perform an operation to retrieve information and then answer the question based on the retrieved information.

[0212] See Figure 8 The flowchart shown represents a single-agent application mode, specifically including the following steps:

[0213] S801, the routing and orchestration control module 1044 routes the query to the specified agent based on the query and the agent specified in the query.

[0214] S802, the agent execution device 106 assembles the ReAct prompt based on the query and context.

[0215] S804, the intelligent agent execution device 106 calls the language model to perform reasoning based on the ReAct prompt, and obtains the execution result Action and thought.

[0216] The execution result can be the result of the intelligent agent's execution device following the thinking process. The execution result can be an intermediate result or a final result, i.e., the final answer.

[0217] In some possible implementations, the agent execution device 106 can temporarily store the Action and Thought, for example, in a temporary register called ScratchPad.

[0218] S806, the intelligent agent execution device 106 determines whether the execution result is the final answer. If not, proceed to S808; if yes, proceed to S810.

[0219] S808, the intelligent agent execution device 106 calls the tool according to the Action and obtains the tool call result.

[0220] S810, the intelligent agent execution device 106 updates the ReAct prompt based on the call result. Then, S806 is executed.

[0221] In S08, the tools can be tools in a broad sense, which may include the services, APIs and other tools mentioned in the foregoing embodiments, as well as knowledge bases, models or other intelligent agents. This application does not limit them.

[0222] It should be noted that the agent execution device 106 can use tool call results as additional information (observation) to aid reasoning. Specifically, the agent execution device 106 can assemble the tool call results with the query and context into a ReAct hint. Furthermore, the agent execution device 106 can also add Thought and Action to the ReAct hint to further increase the amount of information in the hint, thereby improving the quality of the hint and accelerating reasoning.

[0223] S812, the intelligent agent execution device 106 returns a response to the routing and orchestration control module 1044.

[0224] See Figure 9 The flowchart shown represents a single-agent application mode, specifically including the following steps:

[0225] S901, the routing and orchestration control module 1044 loads the DAG through dialogue management based on the user's query.

[0226] Here, the DAG can be the DAG corresponding to a static SOP of a single agent.

[0227] S902, the agent execution device 106 loads the agent according to the DAG.

[0228] S904, the agent execution device 106 loads the next operation node in the agent.

[0229] S906, the intelligent agent execution device 106 acquires the node type and node information of the operation node.

[0230] S908, the intelligent agent execution device 106 executes the operation node according to the node type and node information of the operation node.

[0231] S910, the intelligent agent execution device 106 records the intermediate results of the execution operation node to the execution variable pool.

[0232] S912, the intelligent agent execution device 106 determines whether the execution was successful. If yes, then execute S914; otherwise, execute S916.

[0233] S914, the agent execution device 106 determines whether the operation node is the last operation node of the agent. If yes, then execute S918; otherwise, return to execute S906.

[0234] S916, the intelligent agent execution device 106 executes fault handling logic.

[0235] S918, the agent execution device 106 returns the answer obtained from the agent execution to the routing and orchestration control module 1044.

[0236] in, Figure 9 The specific implementation of the relevant steps in the illustrated embodiments and Figure 4 , Figure 5 The specific implementations of the relevant steps in the illustrated embodiments can be referenced from each other, and will not be repeated here.

[0237] The above provides a detailed introduction to application management methods during agent runtime. The following section introduces application management methods during agent development. Agent development includes different types such as no-code application configuration, low-code application SOP orchestration, and high-code application complex logic development. The metadata of the developed agent needs to be registered with the agent registry center 108. Agent development generally requires configuring at least one of the following: prompt, knowledge, tools, and model. In some possible implementations, agent development can also configure pre-processing strategies for the context and post-processing strategies for model results.

[0238] See Figure 10 The flowchart shown illustrates an agent development method, which includes:

[0239] S1002, Application Management Platform 10 receives business requests.

[0240] The business requirements section specifies the tasks that the business intelligence agent needs to complete. This information can include the task type, the agent ID (agent_id) of the business intelligence agent on which the task relies, and the application ID of the application built upon the business intelligence agent. The task type can include code continuation, unit testing (UT) generation, code debugging, or test code generation, test case checking, and optimization. The agent ID can include the agent name, such as CodeAgent, DesignAgent, or TestAgent. The application ID can be the application name.

[0241] Furthermore, business requirements may also include information about the developer who triggered the task, including the developer's affiliation, the programming language used, the programming framework used, or the type of interactive device. Developer affiliation information includes the organization or department to which the developer belongs, or the developer's employee ID.

[0242] In a concrete implementation, the application management platform 10 can present an agent configuration interface to the user and receive business requirements input by the user through the agent configuration interface. The application management platform 10 may also include an agent development device. In some examples, the agent development device can be an agent development production line, which can present an agent configuration interface to the user and receive business requirements input by the user.

[0243] S1004, Application Management Platform 10 receives the prompt icon configured by the user.

[0244] S1006, Application Management Platform 10 receives the knowledge base identifier configured by the user.

[0245] S1008, Application Management Platform 10 receives the tool identifier configured by the user.

[0246] S1010, the application management platform 10 receives the model identifier of the language model configured by the user.

[0247] Similar to business requirement configuration, the application management platform 10 can receive prompt identifiers, knowledge base identifiers, tool identifiers, or model identifiers configured by users through the agent configuration interface. Among them, the prompt identifier is a specific implementation of prompt auxiliary information. In practical applications, it can also be information from other users assisting in generating language models, such as prompt templates or parameter names in prompts (placeholder variables in agent prompt templates, prompt_key).

[0248] It should be noted that S1004 to S1010 can be executed in parallel or sequentially, and this application does not impose any restrictions on this.

[0249] S1012, Application Management Platform 10 displays the debugging interface to the user. When debugging is complete, S1014 is executed.

[0250] Specifically, the application management platform 10 can render a debugging interface based on configured business requirements, prompts, knowledge base identifiers, tool identifiers, or model identifiers, and then display the debugging interface to the user. If the current configuration meets the business requirements, the debugging is successful, and the application management platform 10 can execute S1014. If the current configuration does not meet the business requirements, the configuration can be modified until the debugging is successful.

[0251] S1014, Application Management Platform 10 registers the metadata of the intelligent agent in the intelligent agent registration center and publishes the intelligent agent.

[0252] This method utilizes a universal framework for the development and operation of multi-agent applications, enabling the unification of various agent types and complex orchestration logic, thereby reducing development and maintenance costs. Furthermore, it fully leverages the strengths of each agent, compensating for their weaknesses, and enhancing the ability to solve complex tasks. In addition, this method supports the rapid exploration and deployment of various agents, including uncoded, low-coded, and fully coded ones, promoting the development of the agent ecosystem. Moreover, this method supports various customer feedback and customized requirements, exhibiting high availability.

[0253] Based on the aforementioned application management method, this application also provides an application management platform 10. The structure of the application management platform 10 of this application is described below from the perspective of functional modularity. For example... Figure 1 As shown, the application management platform 10 includes

[0254] Interaction device 102, agent routing and orchestration device 104, and agent execution device 106;

[0255] The interactive device 102 is used to receive user queries;

[0256] The intelligent agent routing and orchestration device 104 is used to identify the intelligent agent requested by the user based on the query; when the query does not request an intelligent agent, it retrieves the target intelligent agent that matches the query; assigns tasks to the target intelligent agent and determines the execution order or dependency of the tasks; and generates a standard operating procedure through a language model based on the execution order or dependency of the tasks and the target intelligent agent.

[0257] The intelligent agent execution device 106 is used to load the target intelligent agent according to the standard operating procedure, execute the operation nodes in the target intelligent agent, and obtain a response.

[0258] For example, the above-mentioned interactive device 102, intelligent agent routing and orchestration device 104, and intelligent agent execution device 106 can be implemented in hardware or in software.

[0259] When implemented through software, the interaction device 102, the agent routing and orchestration device 104, and the agent execution device 106 can be applications running on computing devices. For example, the agent execution device 106 can be a computing engine running on a computing device. These applications can also be virtualized and provided to users as virtualization services. Virtualization services can include virtual machine (VM) services, bare metal server (BMS) services, or container services. VM services can be services that use virtualization technology to create virtual machine (VM) resource pools on multiple physical hosts to provide VMs for users to use on demand. BMS services are services that create virtual BMS resource pools on multiple physical hosts to provide BMS for users to use on demand. Container services are services that create virtual container resource pools on multiple physical hosts to provide containers for users to use on demand. A VM is a simulated virtual computer, that is, a logical computer. A BMS is a scalable, high-performance computing service with computing performance indistinguishable from traditional physical machines and features secure physical isolation. Containers are a kernel virtualization technology that provides lightweight virtualization to isolate user space, processes, and resources. It should be understood that the VM service, BMS service, and container service mentioned above are merely specific examples. In practical applications, virtualization services can also include other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0260] When implemented in hardware, the interaction device 102, the agent routing and orchestration device 104, and the agent execution device 106 may include at least one computing device, such as a server. Alternatively, the interaction device 102, the agent routing and orchestration device 104, and the agent execution device 106 may also be devices implemented using application-specific integrated circuits (ASICs) or programmable logic devices (PLDs). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0261] In some possible implementations, the target agent includes multiple agents, and the agent routing and orchestration device 104 is further configured to:

[0262] The language model is used to perform association analysis on the multiple agents to obtain the collaborative evaluation results of the multiple agents.

[0263] The multi-agent collaboration is determined based on the collaboration evaluation results in response to the query.

[0264] In some possible implementations, the agent routing and orchestration device 104 is further used for:

[0265] Obtain at least one of the following: length of the query, question specificity, domain knowledge or professional terminology involved, lexical richness, and grammatical structure;

[0266] Predict whether the query is a complex query based on at least one of the following: length, question specificity, domain knowledge or terminology, lexical richness, and grammatical structure.

[0267] The intelligent agent routing and orchestration device 104 is specifically used for:

[0268] When the query is a complex query, the target intelligent agent that matches the query is retrieved from the intelligent agent registry 108.

[0269] The agent registry 108 can be implemented in software or hardware. When implemented in software, the agent registry 108 can be an application running on a computing device. This application can also be virtualized and provided to users as virtualization services such as VM services, BMS services, or container services. When implemented in hardware, the agent registry 108 includes at least one computing device, such as a server. Alternatively, the agent registry 108 can also be a device implemented using an ASIC or a PLD.

[0270] In some possible implementations, the interaction device 102 is further used for:

[0271] Obtain the context of the query;

[0272] The intelligent agent routing and orchestration device 104 is also used for:

[0273] Rewrite the query based on the context and historical sessions;

[0274] The intelligent agent routing and orchestration device 104 is specifically used for:

[0275] The target agent is retrieved from the agent registry 108 and matched with the rewritten query.

[0276] In some possible implementations, the intelligent agent execution device 106 is specifically used for:

[0277] According to the standard operating procedure, load the first agent in the target agent;

[0278] Load the first operation node in the first intelligent agent;

[0279] Obtain the node type and node information of the first operation node. The node information of the first operation node includes at least one of the input and output parameters of the first operation node, the execution logic of the first operation node, and the dependency relationship of the task assigned to the first operation node. The dependency relationship of the task assigned to the first operation node includes the dependency relationship between the first agent and other agents in the target agent, the dependency relationship between the first operation node and other operation nodes in the first agent, or the dependency relationship between the parameters used to execute the first operation node.

[0280] The first operation node is executed based on its node type and node information.

[0281] According to the standard operating procedure, load the second agent in the target agent;

[0282] Load the second operation node in the second intelligent agent;

[0283] Obtain the node type and node information of the second operation node. The node information of the second operation node includes at least one of the input and output parameters of the second operation node, the execution logic of the second operation node, and the dependency relationship of the task assigned to the second operation node. The dependency relationship of the task assigned to the second operation node includes the dependency relationship between the second agent and other agents in the target agent, the dependency relationship between the second operation node and other operation nodes in the second agent, or the dependency relationship between the parameters used to execute the second operation node.

[0284] The second operation node is executed based on its node type and node information.

[0285] In some possible implementations, the intelligent agent execution device 106 is further configured to:

[0286] Determine whether the first operation node is the last operation node of the first agent. If not, execute the next operation node of the first agent.

[0287] In some possible implementations, the application management platform 10 also includes:

[0288] Intelligent Agent Development Device (Figure 1 (Not shown in the image) is used to receive metadata configured by the user for the agent, the metadata including at least one of prompt template identifier, knowledge base identifier, tool identifier or language model model identifier; and to register the metadata of the agent to the agent registration center 108.

[0289] This application also provides a computing device 1100. For example... Figure 11 As shown, the computing device 1100 includes a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate with each other via the bus 1102. The computing device 1100 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 1100.

[0290] Bus 1102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 11 The bus 1102 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1102 may include a path for transmitting information between various components of the computing device 1100 (e.g., memory 1106, processor 1104, communication interface 1108).

[0291] The processor 1104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0292] The memory 1106 may include volatile memory, such as random access memory (RAM). The memory 1106 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD). The memory 1106 stores executable program code, which the processor 1104 executes to implement the aforementioned application management method. Specifically, the memory 1106 stores instructions for the application management platform 10 to execute the application management method. For example, the memory 1106 may store instructions for implementing the functions of the interaction device 102, the agent routing and orchestration device 104, and the agent execution device 106. Further, the memory 1106 may also store instructions for implementing the agent registry center 108 and the agent development device (…). Figure 1 Instructions for functions (not shown in the text).

[0293] The communication interface 1108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1100 and other devices or communication networks.

[0294] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0295] like Figure 12 As shown, the computing device cluster includes at least one computing device 1100. The memory 1106 of one or more computing devices 1100 in the computing device cluster may store instructions from the same application management platform 10 for executing application management methods.

[0296] In some possible implementations, one or more computing devices 1100 in the computing device cluster can also be used to execute some of the instructions used by the application management platform 10 to execute application management methods. In other words, a combination of one or more computing devices 1100 can jointly execute the instructions used by the application management platform 10 to execute application management methods.

[0297] It should be noted that the memory 1106 in different computing devices 1100 in the computing device cluster can store different instructions for executing some functions of the application management platform 10.

[0298] Figure 13 One possible implementation is shown. For example... Figure 13 As shown, two computing devices 1100A and 1100B are connected via a communication interface 1108. The memory in computing device 1100A stores instructions for executing the functions of the interaction device 102 and the agent routing orchestration device 104. The memory in computing device 1100B stores instructions for executing the functions of the agent execution device 106. In other words, the memory 1106 of computing devices 1100A and 1100B jointly stores instructions from the application management platform 10 for executing application management methods. Furthermore, the memory in computing device 1100A can also store instructions for executing the functions of the agent registry center, facilitating retrieval by the agent routing orchestration device 104. The memory in computing device 1100B can also store instructions for executing the functions of the agent development device (…). Figure 1 Instructions for functions (not shown in the text).

[0299] Figure 13 The connection method between the computing device clusters shown can be considered because the application management method provided in this application requires more resources to execute the operation nodes in the agent. Therefore, it is considered that the interaction device 102 and the agent routing and orchestration device 104 are executed by the computing device 1100A, and the functions implemented by the agent execution device 106 are executed by the computing device 1100B.

[0300] It should be understood that Figure 13 The functions of computing device 1100A shown can also be performed by multiple computing devices 1100. Similarly, the functions of computing device 1100B can also be performed by multiple computing devices 1100.

[0301] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 14 One possible implementation is shown. For example... Figure 14As shown, the two computing devices 1100C and 1100D are connected via a network. Specifically, they are connected to the network through the communication interfaces in each computing device. In this possible implementation, the memory 1106 in computing device 1100C stores instructions for executing the functions of the interaction device 102 and the agent routing and orchestration device 104. Simultaneously, the memory 1106 in computing device 1100D stores instructions for executing the functions of the agent execution device 106. When the application management platform 10 also includes an agent registry 108 or an agent development device, the memory in computing device 1100C can also store instructions for executing the functions of the agent registry 108, and the memory in computing device 1100D can also store instructions for executing the functions of the agent development device.

[0302] Figure 14 The connection method between the computing device clusters shown can be that, considering that the application management method provided in this application requires a large amount of resources to execute the operation nodes in the agent, the function implemented by the agent execution device 106 is handed over to the computing device 1100D for execution.

[0303] It should be understood that Figure 14 The functions of the computing device 1100C shown can also be performed by multiple computing devices 1100. Similarly, the functions of the computing device 1100D can also be performed by multiple computing devices 1100.

[0304] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the application management method described above, applied to the application management platform 10.

[0305] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute the application management method described above.

[0306] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. An application management method, characterized in that, The method is executed by an application management platform, which includes an interaction device, an agent routing and orchestration device, and an agent execution device. The interactive device receives user queries; The intelligent agent routing and orchestration device identifies the intelligent agent requested by the user based on the query. When the query does not request an agent, the agent routing orchestration device retrieves a target agent that matches the query; The agent routing and orchestration device assigns tasks to the target agent and determines the execution order or dependencies of the tasks. The intelligent agent routing orchestration device generates a standard operating procedure based on the execution order or dependencies of the tasks and the target intelligent agent through a language model. The intelligent agent execution device loads the target intelligent agent according to the standard operating procedure, executes the operation nodes in the target intelligent agent, and obtains a response.

2. The method according to claim 1, characterized in that, The target intelligent agent includes multiple intelligent agents, and the method further includes: The agent routing orchestration device uses a language model to perform association analysis on the multiple agents and obtain the collaborative evaluation results of the multiple agents. The agent routing orchestration device determines the multi-agent collaboration in response to the query based on the collaboration evaluation results.

3. The method according to claim 1 or 2, characterized in that, The method includes: The intelligent agent routing orchestration device acquires at least one of the following: query length, question specificity, domain knowledge or professional terminology involved, lexical richness, and grammatical structure. The intelligent agent routing orchestration device predicts whether the query is a complex query based on at least one of the following: length, question specificity, domain knowledge or technical terminology, lexical richness, and grammatical structure. The agent routing orchestration device retrieves target agents that match the query, including: When the query is a complex query, the agent routing orchestration device retrieves the target agent that matches the query from the agent registry.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The interactive device obtains the context of the query; The intelligent agent routing orchestration device rewrites the query based on the context and historical sessions; The agent routing orchestration device retrieves target agents that match the query, including: The agent routing orchestration device retrieves and rewrites the target agent that matches the query from the agent registry center.

5. The method according to any one of claims 1 to 4, characterized in that, The intelligent agent execution device loads the target intelligent agent according to the standard operating procedure, executes the operation nodes in the target intelligent agent, and obtains a response, including: The intelligent agent execution device loads the first intelligent agent in the target intelligent agent according to the standard operating procedure; The intelligent agent execution device loads the first operation node in the first intelligent agent; The intelligent agent execution device acquires the node type and node information of the first operation node. The node information of the first operation node includes at least one of the input and output parameters of the first operation node, the execution logic of the first operation node, and the dependency relationship of the task allocated to the first operation node. The dependency relationship of the task allocated to the first operation node includes the dependency relationship between the first intelligent agent and other intelligent agents in the target intelligent agent, the dependency relationship between the first operation node and other operation nodes in the first intelligent agent, or the dependency relationship between the parameters used to execute the first operation node. The intelligent agent execution device executes the first operation node according to the node type and node information of the first operation node; The intelligent agent execution device loads the second intelligent agent from the target intelligent agent according to the standard operating procedure; The intelligent agent execution device loads the second operation node in the second intelligent agent; The intelligent agent execution device acquires the node type and node information of the second operation node. The node information of the second operation node includes at least one of the input and output parameters of the second operation node, the execution logic of the second operation node, and the dependency relationship of the task allocated to the second operation node. The dependency relationship of the task allocated to the second operation node includes the dependency relationship between the second intelligent agent and other intelligent agents in the target intelligent agent, the dependency relationship between the second operation node and other operation nodes in the second intelligent agent, or the dependency relationship between the parameters used to execute the second operation node. The intelligent agent execution device executes the second operation node according to the node type and node information of the second operation node.

6. The method according to claim 5, characterized in that, Before the intelligent agent execution device loads the second intelligent agent from the target intelligent agent according to the standard operating procedure, the method further includes: The intelligent agent execution device determines whether the first operation node is the last operation node of the first intelligent agent. If not, the intelligent agent execution device executes the next operation node of the first intelligent agent.

7. The method according to any one of claims 1 to 6, characterized in that, The application management platform also includes an intelligent agent development device, and the method further includes: The intelligent agent development device receives metadata configured by the user for the intelligent agent, the metadata including at least one of prompt template identifier, knowledge base identifier, tool identifier, or language model model identifier; The agent development device registers the agent's metadata with the agent registration center.

8. An application management platform, characterized in that, The application management platform includes an interactive device, an intelligent agent routing and orchestration device, and an intelligent agent execution device; The interactive device is used to receive user queries; The intelligent agent routing and orchestration device is used to identify the intelligent agent requested by the user based on the query; when the query does not request an intelligent agent, it retrieves the target intelligent agent that matches the query; assigns tasks to the target intelligent agent and determines the execution order or dependencies of the tasks. Based on the execution order or dependencies of the tasks and the target intelligent agent, a standard operating procedure is generated using a language model. The intelligent agent execution device is used to load the target intelligent agent according to the standard operating procedure, execute the operation nodes in the target intelligent agent, and obtain a response.

9. The platform according to claim 8, characterized in that, The target intelligent agent includes multiple intelligent agents, and the intelligent agent routing and orchestration device is further used for: The language model is used to perform association analysis on the multiple agents to obtain the collaborative evaluation results of the multiple agents. The multi-agent collaboration is determined based on the collaboration evaluation results in response to the query.

10. The platform according to claim 8 or 9, characterized in that, The intelligent agent routing and orchestration device is also used for: Obtain at least one of the following: length of the query, question specificity, domain knowledge or professional terminology involved, lexical richness, and grammatical structure; Predict whether the query is a complex query based on at least one of the following: length, question specificity, domain knowledge or terminology, lexical richness, and grammatical structure. The intelligent agent routing and orchestration device is specifically used for: When the query is a complex query, the target intelligent agent that matches the query is retrieved from the intelligent agent registry.

11. The platform according to any one of claims 8 to 10, characterized in that, The interactive device is also used for: Obtain the context of the query; The intelligent agent routing orchestration device rewrites the query based on the context and historical sessions; The agent routing orchestration device retrieves target agents that match the query, including: The agent routing orchestration device retrieves and rewrites the target agent that matches the query from the agent registry center.

12. The platform according to any one of claims 8 to 11, characterized in that, The intelligent agent execution device is specifically used for: According to the standard operating procedure, load the first agent in the target agent; Load the first operation node in the first intelligent agent; Obtain the node type and node information of the first operation node. The node information of the first operation node includes at least one of the input and output parameters of the first operation node, the execution logic of the first operation node, and the dependency relationship of the task assigned to the first operation node. The dependency relationship of the task assigned to the first operation node includes the dependency relationship between the first agent and other agents in the target agent, the dependency relationship between the first operation node and other operation nodes in the first agent, or the dependency relationship between the parameters used to execute the first operation node. The first operation node is executed based on its node type and node information. According to the standard operating procedure, load the second agent in the target agent; Load the second operation node in the second intelligent agent; Obtain the node type and node information of the second operation node. The node information of the second operation node includes at least one of the input and output parameters of the second operation node, the execution logic of the second operation node, and the dependency relationship of the task assigned to the second operation node. The dependency relationship of the task assigned to the second operation node includes the dependency relationship between the second agent and other agents in the target agent, the dependency relationship between the second operation node and other operation nodes in the second agent, or the dependency relationship between the parameters used to execute the second operation node. The second operation node is executed based on its node type and node information.

13. The platform according to claim 12, characterized in that, The intelligent agent execution device is also used for: Determine whether the first operation node is the last operation node of the first agent. If not, execute the next operation node of the first agent.

14. The platform according to any one of claims 8 to 13, characterized in that, The platform also includes: An intelligent agent development device is used to receive metadata configured by a user for the intelligent agent, the metadata including at least one of a prompt template identifier, a knowledge base identifier, a tool identifier, or a model identifier of a language model; and to register the metadata of the intelligent agent to an intelligent agent registration center.

15. A computing device cluster, characterized in that, The computing device cluster includes at least one computing device, the at least one computing device including at least one processor and at least one memory, the at least one memory storing computer-readable instructions; the at least one processor executes the computer-readable instructions to cause the computing device cluster to perform the application management method as described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, Includes computer-readable instructions; the computer-readable instructions are used to implement the application management method according to any one of claims 1 to 7.

17. A computer program product, characterized in that, Includes computer-readable instructions; the computer-readable instructions are used to implement the application management method according to any one of claims 1 to 7.

Images