Task scheduling method and device based on intelligent agent and computer equipment

Abstract

The invention relates to an agent-based task scheduling method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps that when a content query task is received, a plurality of candidate agents are determined according to the task type of the content query task, and resource state information of system computing resources is acquired; according to the task type, the resource state information and the respective capability description metadata of the plurality of candidate agents, scheduling priority ranking is carried out on the plurality of candidate agents, and a task processing agent is determined; according to the resource state information, determining a task execution mode, based on the task execution mode and a content query task, generating a task message, and distributing the task message to a task processing agent; and the task processing agent is used for obtaining the computing resource execution content query task from the system computing resource according to the task execution mode in the task message to obtain a content query result. By adopting the method, the resource utilization efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a task scheduling method and device based on agent, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] With the development of computer technology, a multi-agent system based on a large language model appears, which usually has the ability to process received content query tasks and quickly feedback content query results.

[0003] In related technologies, the multi-agent system based on the large language model usually includes multiple agent instances. When scheduling the agent instances to process the content query tasks, a static weighted round-robin scheduling method is usually used, that is, a scheduler allocates a fixed weight to each agent instance, and then distributes the content query tasks according to the weight proportion.

[0004] However, the traditional method, the static mode cannot adapt to resource changes, and there is a situation that the system outputs brief results when it is idle, and generates detailed results when it is busy, which easily causes waste of computing resources or system overload, and has the problem of low resource utilization efficiency. SUMMARY

[0005] Therefore, it is necessary to provide a task scheduling method, device, computer equipment, computer readable storage medium and computer program product based on agent, which can improve the resource utilization efficiency.

[0006] In a first aspect, the present application provides a task scheduling method based on agent, comprising:

[0007] When receiving a content query task, determining multiple candidate agents according to the task type of the content query task, and obtaining resource state information of system computing resources;

[0008] According to the task type, the resource state information and the ability description metadata of each of the multiple candidate agents, the multiple candidate agents are prioritized and sorted, and a task processing agent is determined;

[0009] According to the resource state information, a task execution mode is determined, a task message is generated based on the task execution mode and the content query task, and the task message is distributed to the task processing agent; the task processing agent is used to obtain computing resources from the system computing resources according to the task execution mode in the task message to execute the content query task and obtain a content query result.

[0010] In a second aspect, the present application further provides an agent-based task scheduling device, comprising:

[0011] a task processing module, configured to, when receiving a content query task, determine a plurality of candidate agents according to a task type of the content query task, and acquire resource state information of system computing resources;

[0012] a scheduling module, configured to, according to the task type, the resource state information and capability description metadata of each of the plurality of candidate agents, perform scheduling priority sorting on the plurality of candidate agents, and determine a task processing agent;

[0013] a task distribution module, configured to, according to the resource state information, determine a task execution mode, generate a task message based on the task execution mode and the content query task, and distribute the task message to the task processing agent; the task processing agent is configured to, according to the task execution mode in the task message, acquire computing resources from the system computing resources to execute the content query task, and obtain a content query result.

[0014] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0015] when receiving a content query task, determining a plurality of candidate agents according to a task type of the content query task, and acquiring resource state information of system computing resources;

[0016] according to the task type, the resource state information and capability description metadata of each of the plurality of candidate agents, performing scheduling priority sorting on the plurality of candidate agents, and determining a task processing agent;

[0017] according to the resource state information, determining a task execution mode, generating a task message based on the task execution mode and the content query task, and distributing the task message to the task processing agent; the task processing agent is configured to, according to the task execution mode in the task message, acquire computing resources from the system computing resources to execute the content query task, and obtain a content query result.

[0018] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0019] when receiving a content query task, determining a plurality of candidate agents according to a task type of the content query task, and acquiring resource state information of system computing resources;

[0020] According to the task type, the resource state information, and the capability description metadata of each of the plurality of candidate agents, the plurality of candidate agents are prioritized for scheduling, and a task processing agent is determined;

[0021] According to the resource state information, a task execution mode is determined, a task message is generated based on the task execution mode and the content query task, and the task message is distributed to the task processing agent; the task processing agent is configured to acquire computing resources from the system computing resources to execute the content query task according to the task execution mode in the task message, and obtain a content query result.

[0022] In a fifth aspect, the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the following steps:

[0023] When a content query task is received, a plurality of candidate agents are determined according to the task type of the content query task, and resource state information of system computing resources is acquired;

[0024] According to the task type, the resource state information, and the capability description metadata of each of the plurality of candidate agents, the plurality of candidate agents are prioritized for scheduling, and a task processing agent is determined;

[0025] According to the resource state information, a task execution mode is determined, a task message is generated based on the task execution mode and the content query task, and the task message is distributed to the task processing agent; the task processing agent is configured to acquire computing resources from the system computing resources to execute the content query task according to the task execution mode in the task message, and obtain a content query result.

[0026] The agent-based task scheduling method, apparatus, computer device, computer readable storage medium and computer program product can determine the multiple candidate agents that can process the content query task when receiving the content query task, and can accurately select the task processing agent for processing the content query task from the multiple candidate agents by using the task type, the resource state information of the system computing resources and the capability description metadata of the multiple candidate agents in a scheduling priority ordering manner, generate the task message based on the task execution mode and the content query task, and distribute the task message to the task processing agent, so that the content query task of the determined task execution mode can be issued, and the computing resources can be obtained from the system computing resources to execute the content query task according to the task execution mode in the task message, and the content query result can be obtained. The whole process can select the task processing agent and determine the task execution mode based on the resource state information of the system computing resources, the task type and the capability description metadata, so that the task processing agent can obtain the computing resources to execute the content query task according to the task execution mode, the resource state information is strongly associated with the scheduling content query task, the resource utilization rate is maximized, and the resource utilization efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0028] Figure 1 An application environment diagram of an agent-based task scheduling method in an embodiment;

[0029] Figure 2 A flowchart of an agent-based task scheduling method in an embodiment;

[0030] Figure 3 A schematic diagram of generating a task message in an embodiment;

[0031] Figure 4 A schematic diagram of determining the priority score of a candidate agent in an embodiment;

[0032] Figure 5 A schematic diagram of predefining a state judgment strategy in an embodiment;

[0033] Figure 6 A diagram for recording data of an agent instance in an embodiment;

[0034] Figure 7 A flow diagram for an agent-based task scheduling method in another embodiment;

[0035] Figure 8 A diagram for performing a content query task in an embodiment;

[0036] Figure 9 A system overall architecture diagram of a double-layer elastic system in an embodiment;

[0037] Figure 10 A workflow diagram of a resource detector in an embodiment;

[0038] Figure 11 A flow diagram for an agent-based task scheduling method in yet another embodiment;

[0039] Figure 12 A structural block diagram of an agent-based task scheduling apparatus in an embodiment;

[0040] Figure 13 An internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0042] It should be noted that the terms "first", "second" and the like used herein can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "include" and "have" and any variations thereof used herein are intended to cover the non-exclusive inclusion. The term "multiple" used herein refers to two or more. The term "and / or" used herein refers to one of the options or any combination of multiple options.

[0043] In order to describe the technical scheme of the present application clearly and facilitate the understanding of the technical scheme of the present application, the key concepts involved in the present application will be explained first as follows.

[0044] 1. Candidate agent.

[0045] A candidate agent refers to an agent instance that is a candidate and has the ability to process content query tasks. An agent instance refers to a specific agent individual that is created and run and has certain autonomous ability. An agent is a software entity that can perceive the environment, reason, and perform actions to achieve goals. It can be understood that an agent is an abstract definition, and an agent instance is a specific running entity that is instantiated in an independent process or thread, which needs to consume resources. Multiple agent instances can be deployed for the same agent, and the amount of resources consumed by different agent instances can be different. For example, a candidate agent can specifically refer to an agent instance in a multi-agent system based on a large language model.

[0046] 2. System computing resources.

[0047] System computing resources refer to a collection of various basic resources in the system that support computing tasks and business operations. These basic resources provide necessary computing and storage capabilities for agent instances to ensure efficient operation of the system and smooth execution of tasks. For example, system computing resources can specifically include processing resources, storage resources, etc. Processing resources can specifically include GPUs (Graphics Processing Units) and CPUs (Central Processing Units). Storage resources can specifically include memory.

[0048] 3. Resource state information.

[0049] Resource state information refers to information used to generally describe the state of system computing resources, which can also be referred to as resource state signals of system computing resources. For example, resource state information can specifically include system global state and recommended execution mode. The system global state refers to the overall operation of the system, which is obtained by integrating the usage of hardware resources, the running state of software resources, and the distribution of business load, etc., and reflects the overall health status and performance of the system. For example, the system global state can be normal state, high load state, and critical state, etc. The recommended execution mode refers to the task execution mode suggested according to the system global state.

[0050] 4. Task execution mode.

[0051] The task execution mode refers to a specific way or strategy adopted by the task processing agent to execute the content query task, which determines the decomposition, execution and feedback of the content query task. For example, the task execution mode can specifically include a core mode and a complete mode. The core mode refers to an operation mode that prioritizes efficiency and controls cost. In the core mode, the task processing agent is constrained within a strict estimated token consumption or computation time, and only generates and executes the most critical content and steps that guarantee the realization of the core goal of the task. The complete mode is an operation mode that prioritizes effectiveness, depth and accuracy. In the complete mode, the system can utilize more abundant computing resources to generate detailed analysis, multiple alternative solutions, in-depth reasoning processes and high-density information output.

[0052] 5. Capability description metadata.

[0053] The capability description metadata refers to metadata used to describe the capabilities and characteristics of the agent instance. For example, the capability description metadata can be metadata generated based on the capability meta description, which is used to describe the supported functions (action_type (action type)), input and output formats (input_schema, output_schema), resource requirements (such as expected_token_budget (estimated token consumption)), and real-time state (such as current_load (current load rate)) of the agent instance.

[0054] The agent-based task scheduling method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 The terminal 102 communicates with the server 104 through a network. The data storage system can store data required to be processed by the server 104. The data storage system can be separately arranged, can be integrated on the server 104, or can be placed on a cloud or other network server. When receiving a content query task sent through the terminal 102, the server 104 determines a plurality of candidate agents according to the task type of the content query task, obtains resource state information of system computing resources, performs priority ranking on the plurality of candidate agents according to the task type, the resource state information and the capability description metadata of each of the plurality of candidate agents, determines a task processing agent, determines a task execution mode according to the resource state information, generates a task message based on the task execution mode and the content query task, distributes the task message to the task processing agent, and the task processing agent is used to obtain computing resources from the system computing resources to execute the content query task according to the task execution mode in the task message, and obtain a content query result.

[0055] The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle-mounted device, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0056] In an exemplary embodiment, as shown in Figure 2 A task scheduling method based on an agent is provided. In this embodiment, the method is applied to a server for illustration. It should be understood that the method can also be applied to a terminal and a system including a terminal and a server, and can be implemented through the interaction of the terminal and the server. In this embodiment, the method includes the following steps 202 to 206. In this embodiment, the method includes the following steps 202 to 206.

[0057] Step 202, when receiving a content query task, determining a plurality of candidate agents according to the task type of the content query task, and obtaining resource state information of system computing resources.

[0058] The content query task refers to a query task submitted by a user to obtain specific information, usually carrying a content query statement. For example, the content query statement can be a plurality of keywords, a question, or a natural language description, etc.

[0059] The task type of the content query task refers to the type of the content query task determined according to the characteristics of the query content of the content query task. For example, the task type of the content query task can include, but is not limited to, a financial analysis task, a data summary task, a natural language processing task, an image recognition task, a prediction modeling task, a content recommendation task, etc. It should be understood that determining the task type of the content query task is conducive to more efficient processing and feedback of the content query result.

[0060] The candidate agent refers to an agent instance that has the ability to process content query tasks. The agent instance refers to a specific agent individual that is created and run and has certain autonomous ability. The agent is a software entity that can perceive the environment, reason, and perform actions to achieve goals. It can be understood that the agent is an abstract definition, and the agent instance is a specific running entity that is instantiated in an independent process or thread, which needs to consume resources. Multiple agent instances can be deployed for the same agent, and different agent instances can consume different amounts of resources. For example, the candidate agent can specifically refer to an agent instance in a multi-agent system based on a large language model. For example, the multi-agent system can be a task planning system. The large language model refers to a large artificial intelligence model that can understand and generate human language, and is the core of driving the agent to plan tasks.

[0061] The system computing resources refer to a collection of various basic resources in the system that support computing tasks and business operations. These basic resources provide necessary computing power and storage capacity for agent instances to ensure efficient operation of the system and smooth execution of tasks. For example, system computing resources can specifically include processing resources, storage resources, etc.

[0062] The processing resources can specifically include GPUs and CPUs. The GPU is a high-performance processor designed specifically for processing graphics and image data, widely used in computer graphics, deep learning, scientific computing, and other fields. The CPU is responsible for business logic, data preprocessing, input / output scheduling, and other non-GPU-intensive tasks. The storage resources can specifically include memory, which is used to temporarily store programs and data being processed by the processor, and has extremely fast access speed.

[0063] The resource state information refers to information that summarizes the state of system computing resources, which can also be referred to as system computing resource state signals. For example, the resource state information can specifically include system global state and recommended execution mode. The system global state refers to the overall operation of the system, which is obtained by integrating the usage of hardware resources, the running state of software resources, and the distribution of business load, etc., and reflects the overall health status and performance of the system. For example, the system global state can be normal state, high load state, and critical state, etc. It can be understood that different system global states reflect different overall operation of the system. The recommended execution mode refers to the task execution mode recommended according to the system global state.

[0064] Understandably, the recommended task execution mode may differ depending on the system's global state. Taking a task execution mode that includes both core mode and full mode, where the information content of the query results indicated by core mode is less than that indicated by full mode, as an example: In a normal system state, indicating ample system resources, the recommended execution mode is full mode. In a high-load or critical system state, indicating limited system resources, the recommended execution mode is core mode.

[0065] For example, when a content query task is received, the server parses the task to determine its task type, identifies multiple candidate agents based on the task type, and obtains the resource status information of the system's computing resources. Each candidate agent has the capability to process the content query task.

[0066] In practical applications, the server can use semantic analysis to parse content query tasks to determine the task type. This task type can be at least one of several predefined types, which can be configured according to the actual application scenario. For example, predefined types could include information retrieval tasks, data extraction tasks, knowledge question answering tasks, document search tasks, multimodal query tasks, predictive query tasks, and semantic query tasks.

[0067] In practical applications, the resource status information of the system's computing resources can be read in real time. When a content query task is received, the server can directly read the latest resource status information from a shared cache (such as Redis) or message bus. Furthermore, this resource status information can be published by a resource detector, which can be configured independently or integrated into the server.

[0068] In specific applications, the resource status information of the system's computing resources can also be generated in real time. When a content query task is received, the server needs to collect the resource indicator data of the system's computing resources, and then use the resource indicator data to determine the status information of the system's computing resources.

[0069] Step 204: Based on the task type, resource status information, and the capability description metadata of each of the multiple candidate agents, sort the scheduling priority of the multiple candidate agents to determine the task processing agent.

[0070] Capability description metadata refers to metadata used to describe the capabilities and characteristics of an agent instance. Specifically, capability description metadata can be metadata generated based on capability meta-descriptions, used to describe the functions (action_type), input / output formats (input_schema, output_schema) supported by the agent instance, resource requirements (e.g., expected_token_budget), and real-time status (e.g., current_load). Capability meta-description is a structured descriptive language used to declare the functions, input / output formats, resource requirements, and real-time status supported by an agent instance. A task-processing agent refers to an agent instance selected from multiple candidate agents for handling content query tasks.

[0071] For example, after determining multiple candidate agents, it is necessary to select the task processing agent from among the multiple candidate agents to handle the content query task. At this time, the server will sort the scheduling priority of the multiple candidate agents according to the task type, resource status information and the capability description metadata of each of the multiple candidate agents, so as to select the candidate agent with the highest priority as the task processing agent.

[0072] In practical applications, when selecting task processing agents, for each candidate agent, the server determines the priority score of the candidate agent based on the task type, resource status information, and the candidate agent's capability description metadata. Then, using the priority scores of multiple candidate agents, the server sorts the multiple candidate agents by scheduling priority to determine the task processing agent.

[0073] Step 206: Determine the task execution mode based on the resource status information; generate a task message based on the task execution mode and the content query task; and distribute the task message to the task processing agent. The task processing agent is used to obtain computing resources from the system computing resources to execute the content query task according to the task execution mode in the task message, and obtain the content query result.

[0074] The task execution mode refers to the specific method or strategy adopted by the task processing agent to execute content query tasks. It determines the decomposition, execution, and feedback methods of the content query task. For example, the task execution mode can specifically include a core mode and a full mode. The core mode is an operation mode that prioritizes efficiency and controllable costs. In the core mode, the task processing agent is constrained to a strict estimated token consumption or computation time, generating and executing only the most critical content and steps that ensure the achievement of the core task objective. The full mode is an operation mode that prioritizes effectiveness and pursues depth and accuracy. In the full mode, the system can utilize richer computing resources to generate detailed analysis, multiple alternative solutions, in-depth reasoning processes, and high-density information output.

[0075] It is understood that in this embodiment, when the task execution mode includes a core mode and a full mode, the computation time for processing content query tasks in the core mode is generally less than that in the full mode, and the amount of information in the content query results output under the core mode is generally less than that in the full mode. In this way, flexible report generation can be achieved, meaning the system can dynamically adjust the level of detail in the output content query results based on current resource availability. This aims to ensure the delivery of core information when resources are limited, and to provide richer and more in-depth analysis when resources are abundant.

[0076] The content query results refer to the query results generated by the task processing agent after executing the content query task. For example, the content query results can be structured query reports. Specifically, in the core task execution mode, the content query results can be structured brief query reports. In the full task execution mode, the content query results can be structured detailed query reports. It should be noted that the detailed query report has a longer word count than the brief query report. For example, a detailed query report can have 1000 words, while a brief query report can have 150 words.

[0077] For example, the server can determine the task execution mode based on the suggested execution mode in the resource status information. Based on the determined task execution mode, the server can generate a task message and distribute it to the task processing agent, so that upon receiving the task message, the task processing agent can execute the content query task according to the task execution mode in the task message.

[0078] In specific applications, such as Figure 3As shown, when generating a task message, if the server determines that no other constraints exist, it can generate the task message by assembling the task execution mode and the content query task. If the server determines that other constraints exist, it needs to generate the task message by assembling the task execution mode, the content query task, and other constraints. These other constraints refer to messages, besides the task execution mode, that constrain the task-processing agent to execute the content query task. For example, other constraints may specifically include query range constraints, language constraints, result format constraints, and resource constraints.

[0079] In practical applications, when a task message is sent to a task processing agent, the agent can retrieve computing resources from the system's computing resources based on the task execution mode in the task message to execute the content query task and obtain the query results. It's understandable that different task execution modes require different amounts of computing resources. This approach strongly correlates resource status information with the scheduled content query task, ensuring that an appropriate amount of computing resources are allocated from the system to process the query task, maximizing resource utilization and improving efficiency.

[0080] The aforementioned agent-based task scheduling method, upon receiving a content query task, can determine multiple candidate agents capable of handling the task based on its task type. By acquiring the resource status information of the system's computing resources, and utilizing the task type, resource status information, and metadata describing the capabilities of each candidate agent, it can accurately select the task-processing agent from among the candidates through a scheduling priority ranking. Based on the resource status information, it determines the task execution mode, generates a task message based on the task execution mode and the content query task, and distributes the task message to the task-processing agent. This enables the distribution of content query tasks with determined execution modes. The task-processing agent can then retrieve computing resources from the system's computing resources according to the task execution mode in the task message to execute the content query task and obtain the content query results. The entire process can select task processing agents and determine task execution modes based on the resource status information, task types, and capability description metadata of the system's computing resources. This allows the task processing agents to obtain computing resources to execute content query tasks according to the task execution mode, strongly correlate resource status information with scheduling content query tasks, maximize resource utilization, and improve resource utilization efficiency.

[0081] In an exemplary embodiment, the task processing agent is determined by prioritizing multiple candidate agents based on task type, resource status information, and capability description metadata of each agent. This includes:

[0082] For each candidate agent, a priority score is determined based on the task type, resource status information, and metadata describing the agent's capabilities.

[0083] Based on the priority scores of multiple candidate agents, the scheduling priority of the multiple candidate agents is sorted to determine the task processing agent.

[0084] The priority score is a quantitative indicator used to characterize the scheduling priority of candidate agents. In other words, a higher priority score indicates a higher scheduling priority for the candidate agent, and a lower priority score indicates a lower scheduling priority.

[0085] For example, for each candidate agent, the server determines the priority score of the candidate agent based on the task type, resource status information and the candidate agent's capability description metadata. Then, using the priority scores of multiple candidate agents, the server sorts the multiple candidate agents by scheduling priority and selects the candidate agent with the highest priority score from the multiple candidate agents as the task processing agent.

[0086] In practical applications, for each candidate agent, the server can determine the candidate agent's load parameters from the candidate agent's capability description metadata, and determine the first estimated token consumption from the candidate agent's capability description metadata based on the task type and resource status information. Then, using the candidate agent's load parameters and the first estimated token consumption, the server can calculate the candidate agent's priority score.

[0087] In a specific application, the server can determine the load factor score based on the candidate agent's load parameters, and the cost factor score based on the first estimated token consumption. Then, based on the load factor score and the cost factor score, the server can determine the candidate agent's priority score. Furthermore, when determining the candidate agent's priority score, the server can obtain the priority score by weighted summing of the load factor score and the cost factor score. The server can also additionally obtain other factor scores and then perform weighted summing of the load factor score, cost factor score, and other factor scores to obtain the priority score.

[0088] It should be noted that, in this embodiment, the method for calculating the load factor score and the cost factor score, as well as the weighted summation coefficient for each factor score when performing weighted summation, can be configured according to the actual application scenario.

[0089] In this embodiment, for each candidate agent, the priority score of the candidate agent can be accurately determined by combining the task type, resource status information and the candidate agent's capability description metadata. Then, by quantifying the priority scores of multiple candidate agents, the scheduling priority of multiple candidate agents can be objectively sorted, thereby achieving accurate selection of the task processing agent.

[0090] In an exemplary embodiment, for each candidate agent, a priority score is determined based on the task type, resource status information, and the candidate agent's capability description metadata, including:

[0091] For each candidate agent, the agent load parameters and agent performance parameters of the candidate agent are determined from the agent's capability description metadata. Based on the task type and resource status information, the first estimated token consumption is determined from the agent's capability description metadata.

[0092] Determine the load factor score based on the agent's load parameters;

[0093] Determine the performance factor score based on the agent's performance parameters;

[0094] Determine the cost factor score based on the first estimated token consumption;

[0095] The priority scores of candidate agents are determined based on the load factor score, performance factor score, and cost factor score.

[0096] Among them, agent load parameters refer to parameters used to describe the current load of the agent. For example, agent load parameters can specifically refer to the current load rate of the agent, and can be values ​​between 0 and 1. Agent performance parameters refer to parameters used to describe the performance of the agent. For example, agent performance parameters can specifically refer to at least one of the agent's historical success rate and average response time when processing tasks. The first estimated token consumption refers to the estimated number of tokens required for the agent to process this type of task under the suggested execution mode in the resource status information.

[0097] Among them, the load factor score is a metric that quantitatively evaluates the current load of a candidate agent. The performance factor score is a metric that quantitatively evaluates the performance of a candidate agent. The cost factor score is a metric that quantitatively evaluates the cost of using a candidate agent.

[0098] For example, such as Figure 4As shown, for each candidate agent, the server determines the agent load parameters and agent performance parameters from the agent's capability description metadata. Based on the task type and resource status information, it determines the first estimated token consumption from the agent's capability description metadata. Then, based on the agent load parameters, it determines the load factor score; based on the agent performance parameters, it determines the performance factor score; and based on the first estimated token consumption, it determines the cost factor score. Finally, by combining the load factor score, performance factor score, and cost factor score, the server determines the candidate agent's priority score.

[0099] In practical applications, the agent's load parameters can be calculated by the candidate agent itself. This embodiment does not limit the method for calculating the agent's load parameters. Taking the current load rate as an example, the current load rate can be obtained through a predefined load calculation formula, which can be configured according to the actual application scenario. For further example, the predefined load calculation formula can be: Current load rate = (Current number of tasks processed / Maximum concurrency capacity) * Weight coefficient 1 + (GPU utilization in the last n minutes) * Weight coefficient 2, where weight coefficient 1, weight coefficient 2, and n can be configured according to the actual application scenario, the sum of weight coefficient 1 and weight coefficient 2 is 1, and n is a positive integer. For example, if weight coefficient 1 is 0.6, then weight coefficient 2 is 0.4.

[0100] In practical applications, the metadata describing the capabilities of candidate agents includes the estimated token consumption of each type of task processed by the candidate agent in each task execution mode. After obtaining the metadata describing the capabilities of candidate agents, the server can determine the first estimated token consumption by querying the metadata describing the capabilities of candidate agents based on the suggested execution mode and task type in the resource status information.

[0101] In practical applications, the server can calculate the load factor score using a predefined formula, which can be configured according to the actual application scenario. For example, taking the agent's load parameter as the current load rate, the specific formula for calculating the load factor score can be: Load factor score = 1 - Current load rate. That is, the load factor score is negatively correlated with the current load rate; the lower the current load rate, the higher the load factor score.

[0102] In practical applications, the server can calculate the performance factor score using a predefined formula, which can be configured according to the actual application scenario. For example, taking the agent's performance parameters, including historical success rate and average response time, as an example, the performance factor score calculation formula can be: Performance Factor Score = Historical Success Rate * (1 / max(Average Response Time, 0.1)), where max(Average Response Time, 0.1) represents the larger value between the average response time and 0.1, meaning the performance factor score is positively correlated with both historical success rate and average response time.

[0103] In practical applications, the server can calculate the cost factor score using a predefined formula, which can be configured according to the actual application scenario. For example, the cost factor score calculation formula could be: Cost Factor Score = First Estimated Token Consumption, or Cost Factor Score = First Estimated Token Consumption / Baseline Token Consumption. The baseline token consumption refers to the token consumption used as a benchmark when calculating the cost factor score. Specifically, it can be obtained based on the token consumption of the candidate agent handling tasks of this task type in each execution mode. For example, the baseline token consumption could be the average token consumption of the candidate agent handling tasks of this task type in each execution mode. Furthermore, this average can be a weighted average, and the weight coefficients for each execution mode can be the same or different, depending on the specific application scenario.

[0104] In this embodiment, for each candidate agent, the agent load parameters and agent performance parameters of the candidate agent are determined from the agent's capability description metadata. Based on the task type and resource status information, the first estimated token consumption is determined from the agent's capability description metadata. This enables accurate acquisition of the parameters for calculating the priority score. Then, the load factor score can be determined based on the agent load parameters, the performance factor score can be determined based on the agent performance parameters, and the cost factor score can be determined based on the first estimated token consumption. By using the load factor score, performance factor score, and cost factor score, the priority score of the candidate agent can be accurately determined.

[0105] In an exemplary embodiment, determining the priority score of a candidate agent based on the load factor score, performance factor score, and cost factor score includes:

[0106] Obtain the first weighting coefficient of the load factor score, the second weighting coefficient of the performance factor score, and the third weighting coefficient of the cost factor score;

[0107] Based on the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient, the load factor score, performance factor score, and cost factor score are weighted to obtain the priority score of the candidate agent.

[0108] For example, the server can calculate the priority score of the candidate agent by obtaining the first weighting coefficient of the load factor score, the second weighting coefficient of the performance factor score, and the third weighting coefficient of the cost factor score, and by weighting the load factor score, performance factor score, and cost factor score based on the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient.

[0109] In specific applications, when performing weighting, the priority score of the candidate agent can be calculated by weighted summation or by weighted averaging. The weight coefficient of each score can be configured according to the actual application scenario. In this embodiment, no specific limitation is made here. For example, the sum of the weight coefficients of the three scores can be 1.

[0110] In a specific application, the priority score of a candidate agent can be calculated using the following formula: Priority score = Load factor score * w1 + Performance factor score * w2 + Cost factor score * w3, where w1 + w2 + w3 = 1, w1 is the first weight coefficient, w2 is the second weight coefficient, and w3 is the third weight coefficient. The three weight coefficients can be configured according to the actual application scenario.

[0111] In this embodiment, by using the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient to weight the load factor score, the performance factor score, and the cost factor score, the load factor, the performance factor, and the cost factor can be comprehensively considered to achieve an accurate calculation of the priority score.

[0112] In an exemplary embodiment, determining the priority score of a candidate agent based on the load factor score, performance factor score, and cost factor score includes:

[0113] Obtain historical task processing records, and extract agent usage records from these records.

[0114] Based on the agent's usage records, determine the affinity factor score of the candidate agents;

[0115] Obtain the first weighting coefficient of the load factor score, the second weighting coefficient of the performance factor score, the third weighting coefficient of the cost factor score, and the fourth weighting coefficient of the affinity factor score;

[0116] Based on the first, second, third, and fourth weighting coefficients, the load factor score, performance factor score, cost factor score, and affinity factor score are weighted to obtain the priority score of the candidate agent.

[0117] The historical task processing record refers to the server's record of tasks processed by agent instances within a certain period. Each record includes the agent instance used, the specific task processed by that agent instance, and the task processing result. This period can be configured according to the actual application scenario. For example, if the period can be one day, then the historical task processing record will be the server's record of tasks processed by agent instances within that day.

[0118] The agent usage record refers to the record of agent instances used, which may include the name and frequency of use of the agent instances. The affinity factor score is a quantitative indicator used to measure the degree of association or preferential cooperation tendency between a candidate agent and the recorded agent instances used. In this embodiment, the reliability of candidate agents can be evaluated through the affinity factor score. It is understood that used agent instances are more reliable than unused agent instances, and agent instances used more frequently are more reliable than those used less frequently.

[0119] For example, when determining the priority score of a candidate agent, the server obtains historical task processing records, extracts agent usage records from these records, compares the agent usage records with the candidate agents, determines the affinity factor score of the candidate agent, and then calculates the priority score of the candidate agent by weighting the load factor score, performance factor score, cost factor score, and affinity factor score based on the first weight coefficient, second weight coefficient, third weight coefficient, and fourth weight coefficient.

[0120] In practical applications, by comparing agent usage records with candidate agents, the server can determine the number of times a candidate agent has been used. This usage count can then be used to determine the affinity factor score of the candidate agent. In a specific application, when using the usage count of a candidate agent to determine its affinity factor score, the server can query a predefined affinity factor score table to map the usage count to the affinity factor score. It should be noted that the predefined affinity factor score table can be configured according to the actual application scenario. In this table, different usage counts correspond to different affinity factor scores, and the usage count is positively correlated with the affinity factor score; that is, the more times an agent is used, the higher its affinity factor score.

[0121] In specific applications, when performing weighting, the priority score of the candidate agent can be calculated by weighted summation or by weighted averaging. The weight coefficient of each score can be configured according to the actual application scenario. In this embodiment, no specific limitation is made here. For example, the sum of the weight coefficients of the four scores can be 1.

[0122] In a specific application, the priority score of a candidate agent can be calculated using the following formula: Priority score = Load factor score * w1 + Performance factor score * w2 + Cost factor score * w3 + Affinity factor score * w4, where w1 + w2 + w3 + w4 = 1, w1 is the first weight coefficient, w2 is the second weight coefficient, w3 is the third weight coefficient, and w4 is the fourth weight coefficient, which can be configured according to the actual application scenario.

[0123] In this embodiment, by acquiring historical task processing records and extracting agent usage records from them, the affinity factor score of candidate agents can be determined using these records. Then, the load factor score, performance factor score, cost factor score, and affinity factor score can be weighted using the first, second, third, and fourth weighting coefficients. By combining the load factor, performance factor, cost factor, and affinity factor, the priority score can be accurately calculated.

[0124] In an exemplary embodiment, the resource status information is generated by the resource detector based on the system global status and the suggested execution mode; the system global status and the suggested execution mode are obtained by the resource detector through status determination based on the resource index data of the system computing resources.

[0125] The resource detector is a resource monitoring tool that collects resource indicator data of the system's computing resources to determine the overall system status and provide suggested execution modes. Resource indicator data refers to a set of resource metrics used to describe the resource usage of the system's computing resources. For example, specific resource indicators may include GPU utilization, system load, available memory, and the number of concurrent tasks. GPU utilization refers to the usage rate of the GPU within a specific time period, usually expressed as a percentage, reflecting the GPU's workload when processing tasks. System load refers to the average load within a specific time period, usually expressed as an average load value over 1 minute, 5 minutes, or 15 minutes, reflecting the number of tasks waiting to run in the system. In this embodiment, the average load value within any specific time period can be selected as the system load, and this specific time period can be configured according to the actual application scenario. For example, if this specific time period is 15 minutes, then the system load refers to the average load value within 15 minutes. Available memory refers to the total amount of currently available memory in the system, expressed as a percentage, reflecting the proportion of unused memory resources in the system. The number of concurrent tasks refers to the number of tasks running simultaneously in the system, reflecting the system's concurrent processing capability.

[0126] For example, in order to obtain the resource status information of the system computing resources, the resource detector first collects the resource index data of the system computing resources, then makes a status judgment based on the resource index data, determines the global system status and the recommended execution mode, summarizes the global system status and the recommended execution mode, and generates the resource status information of the system computing resources.

[0127] In practical applications, during state determination, the resource detector maps at least one resource metric from the resource indicator data to the corresponding system global state and determines the recommended execution mode under that system global state. It is understandable that different values ​​of the at least one resource metric correspond to different system global states. Specifically, the system global state can be one of multiple candidate global states, including normal state, high-load state, and critical state. Each candidate global state corresponds to a different range of resource metric values.

[0128] In practical applications, a resource detector can use a lightweight daemon to collect resource metrics data of the system's computing resources. This lightweight daemon is a low-resource-consuming process that runs in the background and periodically performs data collection tasks to gather resource metrics data. In a specific application, the resource metrics data includes at least one resource indicator, and each indicator can be collected using different methods. Therefore, the lightweight daemon can collect different resource indicators using different methods.

[0129] For example, when the resource metric is GPU utilization, it can be collected by calling the NVML API (NVIDIA Management Library API, a set of functions provided by NVIDIA for managing and monitoring NVIDIA GPU devices). When the resource metric is system load, it can be collected by reading ` / proc / loadavg`, a virtual file in Linux containing the average system load. It records the average load over the past 1 minute, 5 minutes, and 15 minutes. When the resource metric is available memory, it can be obtained using the `psutil` library (a cross-platform library for obtaining system and process information). When the resource metric is the number of concurrent tasks, it can be obtained using the `psutil` library and internal counters. Internal counters are counters maintained internally by the system to record specific system states or events. When collecting the number of concurrent tasks, the internal counters can be process counters maintained by the system kernel.

[0130] In this embodiment, by collecting resource index data of the system's computing resources, the system can use the resource index data to determine the status, thereby determining the global status of the system and the suggested execution mode. Then, the system can use the global status of the system and the suggested execution mode to accurately generate resource status information of the system's computing resources.

[0131] In an exemplary embodiment, the system global state and suggested execution mode are obtained by the resource detector matching at least one resource metric from the resource metric data with the state determination conditions of multiple candidate global states in a predefined state determination strategy. Here, resource metrics refer to metrics used to describe the resource usage of system computing resources. For example, resource metrics may specifically include GPU utilization, system load, available memory, and the number of concurrent tasks. The predefined state determination strategy refers to a predefined strategy for determining the system global state, which can be configured according to the actual application scenario. The predefined state determination strategy includes the state determination conditions of multiple candidate global states.

[0132] Candidate global states refer to predefined global state options, i.e., possible system global states, which can be configured according to the actual application scenario. For example, multiple candidate global states may include normal state, high load state, and critical state, with each candidate global state corresponding to a different range of resource metric values. The state determination condition for a candidate global state refers to the conditions under which a system global state is determined to be a candidate global state. For example, the state determination condition for a candidate global state may specifically be a specific range of values ​​for at least one resource metric; that is, when at least one resource metric in the resource metric data falls within a specific value range, the system global state is considered a candidate global state.

[0133] For example, the resource detector matches at least one resource metric in the resource metric data with the state determination conditions of each of the multiple candidate global states in the predefined state determination strategy to determine the state determination conditions satisfied by at least one resource metric. The candidate global state corresponding to the satisfied state determination conditions is determined as the system global state corresponding to the resource metric data, and the suggested execution mode corresponding to the candidate global state is determined as the suggested execution mode corresponding to the resource metric data.

[0134] In practical applications, the state determination conditions for each candidate global state can be configured according to the actual application scenario, and the state determination conditions between multiple candidate global states must not conflict. It should be noted that for each candidate global state, the state determination conditions for the candidate global state can use only a portion of the resource metrics for determination, and the resource metrics used for the state determination conditions of different candidate global states do not have to be exactly the same, and the numerical ranges of the resource metrics in the state determination conditions of different candidate global states must not conflict.

[0135] In a specific application, taking multiple candidate global states including normal state, high load state and critical state, and at least one resource metric including GPU utilization, system load, available memory and number of concurrent tasks as an example, the normal state can be determined by GPU utilization, system load, available memory and number of concurrent tasks, the high load state can be determined by at least one of GPU utilization and number of concurrent tasks, and the critical state can be determined by GPU utilization, system load and available memory.

[0136] To further illustrate, a predefined state determination strategy can be specifically as follows: Figure 5As shown, if GPU utilization is greater than 85% or system load is greater than 4.0, and available memory is less than 10%, the system is considered to be in a critical state, and the recommended execution mode is kernel mode. If GPU utilization is greater than 60%, or the number of concurrent tasks is greater than 100, the system is considered to be in a high-load state, and the recommended execution mode is kernel mode. All other cases are considered to be in a normal state, and the recommended execution mode is full mode. It should be noted that when GPU utilization is greater than 60%, it is necessary to further determine whether GPU utilization is greater than 85%. If GPU utilization is greater than 85%, it is further determined whether available memory is less than 10%. If available memory is not less than 10%, the system can be considered to be in a high-load state.

[0137] In this embodiment, by using the matching result of at least one resource indicator quantity and the resource status determination condition, the system global status and suggested execution mode corresponding to the determined resource indicator data can be accurately determined.

[0138] In an exemplary embodiment, the task processing agent is further configured to generate system prompt words based on the task execution mode in the task message, and based on the system prompt words, retrieve computing resources from the system computing resources to execute a content query task and obtain content query results; the system prompt words are used to indicate the amount of information in the output content query results.

[0139] System prompts are words used to indicate the amount of information in the content query results output by the task processing agent. It is understandable that different task execution modes use different system prompts, and the amount of information in the output content query results indicated by different system prompts also varies.

[0140] The information content of a content query result refers to a quantitative indicator used to describe the richness of the content query result. For example, the information content of a content query result can specifically refer to the information length of the content query result, that is, the number of characters or words in the content query result.

[0141] For example, when a task message is sent to a task processing agent, the task processing agent can generate a system prompt word corresponding to the task execution mode in the task message to indicate the amount of information in the output content query result. Then, based on the system prompt word, the agent can obtain computing resources from the system computing resources to execute the content query task and obtain the content query result.

[0142] In practical applications, when retrieving computing resources from the system based on system prompts, the amount of computing resources required varies depending on the amount of information in the query results indicated by different system prompts. It can be understood that the less information in the query results indicated by a system prompt, the less computing resources are required to execute the query task; conversely, the more information in the query results indicated by a system prompt, the more computing resources are required. In other words, the amount of information in the query results indicated by a system prompt is positively correlated with the amount of computing resources required to execute the query task.

[0143] In this embodiment, by generating system prompts based on the task execution mode in the task message, the system prompts can be used to indicate the amount of information in the output content query results. Furthermore, based on the system prompts, computing resources can be obtained from the system's computing resources to execute the content query task and obtain the results. This entire process allows the task processing agent to obtain computing resources to execute the content query task according to the task execution mode, strongly associating the task execution mode with the execution of the content query task, maximizing resource utilization, and improving resource efficiency.

[0144] In one exemplary embodiment, the agent-based task scheduling method further includes:

[0145] When the task execution mode in the task message is core mode, the core mode prompt word corresponding to core mode is the system prompt word;

[0146] When the task execution mode is full mode, the full mode prompt word is the system prompt word; the full mode prompt word indicates that the amount of information in the output content query result is greater than the amount of information in the output content query result indicated by the core mode prompt word.

[0147] For example, when a task message is sent to a task processing agent, the task processing agent can generate a system prompt word based on the task execution mode in the task message. When generating the system prompt word, if the task execution mode in the task message is core mode, indicating that system resources are scarce, the task processing agent will determine the core mode prompt word corresponding to core mode as the system prompt word. If the task execution mode is full mode, indicating that system resources are sufficient, the task processing agent will determine the full mode prompt word corresponding to full mode as the system prompt word.

[0148] It should be noted that the information content of the content query results indicated by the full-mode prompt is greater than that indicated by the core-mode prompt. Taking the information content of the content query results as an example, the information length of the content query results indicated by the full-mode prompt is greater than that indicated by the core-mode prompt. In this embodiment, the full-mode prompt and the core-mode prompt can be configured according to the actual application scenario.

[0149] In practical applications, taking the information length of content query results, specifically the number of characters in the content query results, as an example, the format of the full mode prompt can be: "You are a detailed analyst. Please provide a comprehensive analysis report, including data, analysis, comparisons, and recommendations." The format of the core mode prompt can be: "You are an efficient analysis assistant. Please provide the core conclusions and key data directly in the most concise language within XXX words, without the need for analysis process or details."

[0150] In this embodiment, different system prompt words are determined for different task execution modes, so that the task processing agent can obtain the computing resource execution content query task according to the corresponding system prompt words of the task execution mode. This strongly associates the task execution mode with the execution content query task, thereby maximizing resource utilization and improving resource utilization efficiency.

[0151] In an exemplary embodiment, the task processing agent is further configured to adjust the content generation parameters of the task processing agent according to the resource constraint information in the task message, and obtain computing resources from the system computing resources to execute the content query task based on the system prompt words and the content generation parameters, thereby obtaining the content query results.

[0152] Resource constraint information refers to information used to constrain the acquisition of computing resources by the task processing agent. For example, resource constraint information could specifically be a token budget constraint, which is the upper limit on the number of tokens allocated to or allowed to be consumed by the task processing agent. Content generation parameters refer to the parameters used by the task processing agent to generate content. For example, content generation parameters could specifically include the maximum number of tokens.

[0153] For example, when generating system prompt words, the task processing agent needs to further determine whether there is resource constraint information in the task message. If there is resource constraint information in the task message, it needs to adjust the content generation parameters of the large language model in the task processing agent according to the resource constraint information in the task message. Then, based on the system prompt words and content generation parameters, it needs to obtain computing resources from the system computing resources to execute the content query task and obtain the content query results.

[0154] In practical applications, taking the resource constraint information as the token budget constraint information as an example, the task processing agent needs to set the maximum number of tokens in the content generation parameters as the token budget constraint amount to limit the upper limit of the number of tokens used when executing the content query task.

[0155] In this embodiment, by adjusting the content generation parameters in the task processing agent based on the resource constraint information in the task message, and then obtaining computing resources from the system computing resources to execute the content query task based on the system prompt words and content generation parameters, the content query results are obtained. This allows the acquisition of computing resources with system prompt words and content generation parameters as constraints, strongly correlates the task execution mode and resource constraint information with the execution of the content query task, maximizes resource utilization, and improves resource utilization efficiency.

[0156] In one exemplary embodiment, a task message is generated based on the task execution mode and the content query task, including:

[0157] Obtain resource constraint information for the task processing agent;

[0158] Assemble the task execution mode, resource constraint information, and content query task, and generate task messages.

[0159] Resource constraint information refers to information used to constrain the acquisition of computing resources by the task processing agent. For example, resource constraint information can specifically be the token budget constraint, which is the upper limit of the number of tokens allocated to or allowed to be consumed by the task processing agent.

[0160] For example, when generating a task message, the server obtains resource constraint information for the task processing agent, then assembles the task execution mode, resource constraint information, and content query task to generate the task message. In specific applications, the resource constraint information of the task processing agent can be obtained from the agent's capability description metadata. Taking the token budget constraint as an example, the server can determine at least one estimated token consumption of the task processing agent from its capability description metadata, and then determine the token budget constraint of the task processing agent based on that estimated token consumption.

[0161] In a specific application, at least one estimated token consumption, including a second estimated token consumption of the task processing agent in task execution mode, can be directly determined by the server as the token budget constraint of the task processing agent.

[0162] In this embodiment, based on the acquisition of resource constraint information, task messages can be accurately generated by assembling task execution modes, resource constraint information, and content query tasks, thereby enabling the task processing agent to execute content query tasks based on the task messages.

[0163] In one exemplary embodiment, obtaining resource constraint information for the task processing agent includes:

[0164] From the capability description metadata of the task processing agent, determine the second estimated token consumption of the task processing agent in the task execution mode, and determine the third estimated token consumption of the task processing agent in other execution modes.

[0165] Based on the second and third estimated token consumption, resource constraint information is generated for the task processing agent.

[0166] For example, when obtaining resource constraint information for a task processing agent, the server first determines the second estimated token consumption of the task processing agent in the task execution mode from the capability description metadata of the task processing agent, and determines the third estimated token consumption of the task processing agent in other execution modes. Then, based on the second estimated token consumption and the third estimated token consumption, the server generates resource constraint information for the task processing agent. Specifically, the resource constraint information may be a token budget constraint.

[0167] In practical applications, based on the second and third estimated token consumption values, the server performs a weighted sum of these values ​​to obtain resource constraint information for the task processing agent. The weighting coefficients used in the weighted summation can be configured according to the actual application scenario, corresponding to the execution mode, and the sum of the weighting coefficients for the second and third estimated token consumption values ​​is 1. It can be understood that when the first weighting coefficient corresponding to the second estimated token consumption value and the second weighting coefficient corresponding to the third estimated token consumption value are the same, it is equivalent to calculating the average estimated token consumption value of the second and third estimated token consumption values.

[0168] In a specific application, taking the example of 1 other execution mode, the formula for calculating resource constraint information can be: Resource constraint information = second estimated token consumption * w1 + third estimated token consumption * w2, where w1 is the first weight coefficient and w2 is the second weight coefficient, which can be configured according to the actual application scenario, and w1 + w2 = 1. When w1 = w2 = 0.5, the resource constraint information is the average estimated token consumption. When w1 and w2 are not equal to 0.5, the resource constraint information is the average value based on load weighting.

[0169] Furthermore, when the execution mode includes core mode and full mode, the weight coefficient corresponding to the estimated token consumption in core mode can usually be configured to be greater than the weight coefficient corresponding to the estimated token consumption in full mode. Taking the task execution mode as core mode and the other execution modes as full mode as an example, the first weight coefficient will be greater than the second weight coefficient. For example, the first weight coefficient can be set to 0.65 and the second weight coefficient to 0.35.

[0170] In this embodiment, by determining the second estimated token consumption of the task processing agent in the task execution mode and the third estimated token consumption of the task processing agent in other execution modes, the resource constraint information for the task processing agent can be accurately determined using the second estimated token consumption and the third estimated token consumption.

[0171] In one exemplary embodiment, the agent-based task scheduling method further includes:

[0172] When a heartbeat message is received from any agent instance, the capability description metadata of any agent instance is obtained from the heartbeat message and recorded.

[0173] Upon receiving a content query task, multiple candidate agents are determined based on the task type, including:

[0174] When a content query task is received, the task type of the content query task is matched with the action type in the capability description metadata of multiple agent instances to obtain the agent matching result.

[0175] Based on the agent matching results, multiple candidate agents are determined from multiple agent instances.

[0176] In this context, an agent instance refers to a specific, created, and running agent possessing a certain degree of autonomy. An agent is a software entity capable of perceiving its environment, reasoning, and executing actions to achieve a goal. It can be understood that an agent is an abstract definition, while an agent instance is the concrete, running entity of an agent within an independent process or thread, requiring resource consumption. Multiple agent instances can be deployed for the same agent, and the resource consumption of different instances can vary. Action type refers to the types of actions an agent instance can perform, that is, the types of tasks an agent instance can execute.

[0177] For example, upon receiving heartbeat information from any agent instance, the server retrieves the capability description metadata of that agent instance from the heartbeat information and records it. Upon receiving a content query task, the server determines the task type of the content query task and matches it with the action types in the capability description metadata of multiple agent instances to obtain agent matching results. Based on the matching degree between the task type in the agent matching results and the action type in the capability description metadata of the agent instances, multiple candidate agents with the capability to process the content query task are determined from the multiple agent instances.

[0178] Among them, candidate agents capable of handling content query tasks can be agent instances whose action type and task type in the capability description metadata match a threshold greater than a certain threshold. This threshold can be configured according to the actual application scenario. For example, the threshold can be 80%.

[0179] In practical applications, for each agent instance, after startup and registration, it periodically sends heartbeat information to the server. If no heartbeat information is received from the agent instance within a preset time interval, the server can mark the agent instance as offline. In other words, the normal operation of an agent instance can be determined by the received heartbeat information. The time interval for sending heartbeat information and the preset time interval can be configured according to the actual application scenario. For example, the time interval for sending heartbeat information can be 5 seconds, and the preset time interval can be twice the time interval for sending heartbeat information, i.e., 10 seconds.

[0180] In practical applications, when recording the capability description metadata of any agent instance, the server will synchronously record the time of the last heartbeat information received. This reception time can be used to determine whether the agent instance is running normally. If no heartbeat information is received from the agent instance within a preset time interval from the reception time, the server can mark the agent instance as offline.

[0181] In a specific application, the data of the agent instances recorded in the server can be as follows: Figure 6 As shown, this includes capability description metadata and the time of receiving the last heartbeat information. The capability description metadata includes the agent instance's agent identifier and action type (e.g., ...). Figure 6 The diagram shows "Financial Analysis" and "Data Summary"), input format, output format, and estimated token consumption for each execution mode (e.g., ...). Figure 6 Specifically, this can be shown as: "Core Mode": 500, "Full Mode": 2000) and the current load rate.

[0182] In this embodiment, when the heartbeat information of any intelligent agent instance is received, the capability description metadata of any intelligent agent instance is obtained from the heartbeat information and recorded. This enables the recording of the capability description metadata of intelligent agent instances. Thus, when a content query task is received, multiple candidate intelligent agents capable of processing the content query content can be selected from multiple intelligent agent instances by matching the task type of the content query task with the action type in the capability description metadata of each of the multiple intelligent agent instances. This achieves accurate selection of multiple candidate intelligent agents.

[0183] In one exemplary embodiment, such as Figure 7 As shown, a task scheduling method based on intelligent agents is provided. This embodiment illustrates the method by applying it to a server. It is understood that this method can also be applied to a terminal, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes steps 702 to 736. Wherein:

[0184] Step 702: When the heartbeat information of any agent instance is received, the capability description metadata of any agent instance is obtained from the heartbeat information and recorded.

[0185] Step 704: When a content query task is received, the task type of the content query task is matched with the action type in the capability description metadata of multiple agent instances to obtain the agent matching result.

[0186] Step 706: Based on the agent matching results, determine multiple candidate agents from multiple agent instances and collect resource index data of system computing resources.

[0187] Step 708: Match at least one resource indicator in the resource indicator data with the state determination conditions of each of the multiple candidate global states in the predefined state determination strategy to determine the system global state corresponding to the resource indicator data and the suggested execution mode.

[0188] Step 710: Based on the system global state and the suggested execution mode, generate resource status information of the system computing resources.

[0189] Step 712: For each candidate agent, determine the agent load parameters and agent performance parameters of the candidate agent from the agent's capability description metadata, and determine the first estimated token consumption from the agent's capability description metadata based on the task type and resource status information.

[0190] Step 714: Determine the load factor score based on the agent load parameters, determine the performance factor score based on the agent performance parameters, and determine the cost factor score based on the first estimated token consumption.

[0191] Step 716: Obtain historical task processing records and extract agent usage records from them.

[0192] Step 718: Determine the affinity factor score of the candidate agents based on the agent usage records.

[0193] Step 720: The load factor score, performance factor score, cost factor score and affinity factor score are weighted and summed to obtain the priority score of the candidate agent.

[0194] Step 722: Based on the priority scores of each candidate agent, sort the candidate agents by scheduling priority to determine the task processing agent.

[0195] Step 724: Based on the resource status information, determine the task execution mode, determine the second estimated token consumption of the task processing agent in the task execution mode from the capability description metadata of the task processing agent, and determine the third estimated token consumption of the task processing agent in other execution modes.

[0196] Step 726: Based on the second estimated token consumption and the third estimated token consumption, generate resource constraint information for the task processing agent.

[0197] Step 728: Assemble the task execution mode, resource constraint information, and content query task; generate a task message; and distribute the task message to the task processing agent so that the task processing agent can execute the following steps 730 to 736.

[0198] Step 730: If the task execution mode in the task message is core mode, determine the core mode prompt word corresponding to core mode as the system prompt word.

[0199] Step 732: When the task execution mode is full mode, the full mode prompt word corresponding to the full mode is determined as the system prompt word; the full mode prompt word indicates that the amount of information in the output content query result is greater than the amount of information in the output content query result indicated by the core mode prompt word, and the system prompt word is used to indicate the amount of information in the output content query result.

[0200] Step 734: Adjust the content generation parameters of the task processing agent based on the resource constraint information in the task message.

[0201] Step 736: Based on the system prompt words and content generation parameters, obtain computing resources from the system computing resources to execute the content query task and obtain the content query results.

[0202] In an exemplary embodiment, the agent-based task scheduling method of this application is used as an example to illustrate the application of the content query feedback method of a multi-agent system based on a large language model. The agent-based task scheduling method of this application can be applied to elastic output and agent instance scheduling in the content query feedback of a multi-agent system based on a large language model. Specifically, in the agent-based task scheduling method of this application, after receiving a content query task, the method can comprehensively consider the task type of the content query task, the capability description metadata of multiple candidate agents, and the resource status information of the system computing resources. By prioritizing the scheduling, the task processing agent for handling the content query task is determined, thereby realizing agent instance scheduling and the allocation of content query tasks. Furthermore, at the execution level, the task processing agent can adaptively adjust the system prompts and content generation parameters according to the task execution mode (specifically, core mode or full mode) and additional resource constraint information in the issued task message, outputting results with different levels of detail, thus achieving elastic output.

[0203] The following is an example of a task processing agent performing a content query task, such as... Figure 8 As shown, the task processing agent is built based on a large language model and has the ability to automatically decompose executable steps and design solutions for complex problems. Taking content query tasks as an example... Figure 8 Taking the example of "I enjoy hiking and want to stay in the mountains, so I need to buy a tent. Do you have any suggestions on what to buy?", the task processing agent can understand the problem of the content query task, then formulate a plan and automatically break it down into executable steps (such as...). Figure 8 The diagram includes executable steps 801, 802, and 803. These decomposed executable steps are displayed to the sender of the content query task, and then executed. After obtaining the execution result of each executable step, the content query result of the content query task is obtained and displayed to the sender of the content query task. It should be noted that, as... Figure 8 As shown, in the agent-based task scheduling method of this application, the amount of information in the content query results is related to the task execution mode. The amount of information in the content query results output by the task processing agent in the core mode is less than the amount of information in the content query results output by the task processing agent in the full mode. Figure 8 In this context, the number of "X" symbols represents the amount of information; the more "X" symbols there are, the greater the amount of information.

[0204] In related technologies, multi-agent systems based on large language models typically include multiple agent instances. When scheduling agent instances to handle content query tasks, two common methods are user-selected output modes or static weighted round-robin scheduling. User-selected output modes, where the AI ​​chat assistant provides "short reply," "long reply," or "more details" buttons, allow the user to manually choose the desired level of detail before or during the conversation. This is an external control mechanism entirely dependent on user subjective judgment. Static weighted round-robin scheduling, on the other hand, assigns a fixed weight to each agent instance and then allocates content query tasks according to these weights.

[0205] However, traditional methods have several drawbacks: First, the user experience is fragmented, requiring users to manually switch modes, increasing cognitive load, and making it difficult for users to accurately determine which mode is best suited for the current system state. Second, resource utilization efficiency is low. For methods where users manually select output modes, the system's output detail level is usually preset and fixed, unable to dynamically adjust according to the system's real-time resource status. This may lead to slow response or overflow when resources are scarce, while output may be insufficient when resources are idle. For static weighted round-robin scheduling, the static mode cannot adapt to resource changes, resulting in simplified results when the system is idle and detailed results when the system is busy, easily leading to wasted computing resources or system overload. Third, scheduling is unbalanced. Static scheduling strategies cannot perceive the real-time load of agent instances, easily leading to some agent instances being overloaded while others are idle, resulting in a decrease in overall system throughput.

[0206] Based on this, this application proposes an agent-based task scheduling method, namely, an agent-based elastic output and adaptive scheduling method based on a large language model of resource awareness and capability negotiation. It constructs a two-layer elastic system of resource awareness and capability adaptation to address the shortcomings of traditional technologies. Resource awareness refers to the system's ability to detect and perceive the status of underlying computing resources (such as GPU utilization, memory usage, available token budget, and network bandwidth) and business load (such as concurrent request count) in real time; that is, the system's ability to detect and perceive the status of its computing resources in real time. Elastic output refers to the system's ability to dynamically adjust the level of detail (core mode or full mode) of the output content according to the current resource availability, aiming to ensure the delivery of core information when resources are limited, and to provide richer and more in-depth analysis when resources are abundant.

[0207] In practical applications, the overall system architecture of a two-layer elastic system can be as follows: Figure 9 As shown, this application implements core functionalities by introducing a resource detector, a capability registry, and an elastic scheduler into a traditional architecture. The following section combines these functionalities with...Figure 9 The functions and technical implementation details of each module in the agent-based task scheduling method of this application are described.

[0208] In a specific application, such as Figure 9 As shown, the resource detector resides in the resource awareness layer. The data acquisition layer within the resource detector primarily collects resource metrics data of the system's computing resources periodically (e.g., every second) via a lightweight daemon process. For example, the data acquisition layer can collect GPU utilization by calling the NVML API, system load by reading ` / proc / loadavg`, available memory by using the `psutil` library, and the number of concurrent tasks by using the `psutil` library and an internal counter. After collecting the resource metrics data, the resource detector performs status judgment through a rule engine to obtain a unified resource status (i.e., resource status information, including the system's global status and suggested execution mode), and publishes the status information, either writing it to Redis (a shared cache) or publishing it via a message bus.

[0209] In a specific application, the rule engine can map resource metric data to enumerated states, which can be configured according to the actual application scenario. For example, enumerated states can specifically include normal state, high load state, and critical state, and the rules for state judgment can be predefined state judgment strategies. A specific example of a rule could be as follows: Figure 5 As shown, if GPU utilization is greater than 85% or system load is greater than 4.0, and available memory is less than 10%, the system global state is determined to be critical, and the corresponding recommended execution mode is kernel mode. If GPU utilization is greater than 60%, or the number of concurrent tasks is greater than 100, the system global state is determined to be high load, and the corresponding recommended execution mode is kernel mode. In all other cases, the system global state is determined to be normal, and the corresponding recommended execution mode is full mode. It should be noted that when GPU utilization is greater than 60%, it is necessary to further determine whether GPU utilization is greater than 85%. If GPU utilization is greater than 85%, it is further determined whether available memory is less than 10%. If available memory is not less than 10%, the system global state can be determined to be high load. It should be noted that the rules given in this embodiment are only examples of rules. The specific rules for state judgment can be configured according to the actual application scenario.

[0210] In a specific application, the workflow diagram of the resource detector can be as follows: Figure 10As shown, the resource detector mainly consists of a data acquisition layer and a data aggregation and status judgment layer. The data acquisition layer collects resource metrics such as GPU metrics (collected via NVML / dcgm), system metrics (collected via psutil / procfs), and business metrics (collected via internal counters). The data aggregation and status judgment layer uses predefined threshold strategies (i.e., predefined status judgment strategies) in the rule engine to map resource metric data to enumerated states, generating a unified resource status signal (i.e., resource status information, such as...). Figure 10 The example provided could be a system-wide state in a critical state (with the recommended execution mode being core mode), where the state is published via Redis Pub / Sub or through a message bus, allowing the elastic scheduler to subscribe.

[0211] In a specific application, such as Figure 9 As shown, the capability registry center resides in the capability registration layer. Specifically, the capability registry center can be a centralized database (such as Redis or a relational database) that records the metadata (i.e., capability description metadata) of all active agent instances. The data of the agent instances recorded in the capability registry center can be as follows: Figure 6 As shown, this includes capability description metadata and the time of receiving the last heartbeat information. The capability description metadata includes the agent instance's agent identifier and action type (e.g., ...). Figure 6 The table shows the "Financial Analysis" and "Data Summary" formats, input formats, estimated token consumption for each execution mode, and current load rate.

[0212] In a specific application, the capability registry center implements its functionality based on a heartbeat mechanism. Specifically, agent instances (such as...) Figure 9 The example shown illustrates how agent instances (such as Agent A and Agent B) register after startup and periodically (specifically every 5 seconds) send heartbeat information (including updated current load rates). If the capability registration center does not receive a heartbeat within a timeout (e.g., 10 seconds), the agent instance is marked as offline. The current load rate can be calculated by the agent instance itself. For example, the formula for calculating the current load rate could be: Current Load Rate = (Current Number of Tasks Processed / Maximum Concurrency Capacity) * 0.6 + (GPU Utilization in the Last Minute) * 0.4. It should be noted that the weighting coefficients 0.6 and 0.4 in the current load rate calculation formula are examples and can be configured according to actual needs.

[0213] In a specific application, such as Figure 9As shown, the decision-making process of the elastic scheduler is as follows: It receives user requests (i.e., content query tasks), obtains resource status information of system computing resources from the resource detector, queries the capability registry center according to the task type requested by the user, obtains multiple candidate agents and their status (i.e., capability description metadata), makes dynamic scheduling decisions, calculates dynamic scheduling priorities based on resource status and agent instance capabilities, selects the optimal agent instance, and issues content query tasks and execution mode instructions (i.e., task messages). When calculating dynamic scheduling priorities, a priority score needs to be calculated for each candidate agent. Specifically, a dynamic priority scheduling algorithm can be used, which refers to the algorithm by which the central scheduler (in this embodiment, the elastic scheduler) dynamically calculates and adjusts the task scheduling priorities of each candidate agent based on the capability description metadata (especially agent load parameters (specifically, the current load rate) and agent performance parameters) and the global system load (represented by resource status information) collected from multiple candidate agents.

[0214] In a specific application, a dynamic priority scheduling algorithm, including a dynamic priority scoring algorithm, is used to calculate a priority score for each candidate agent. The following example illustrates the dynamic priority scoring algorithm, using the agent's load parameter as the current load rate and performance parameters including historical success rate and average response time. The dynamic priority scoring algorithm primarily evaluates the priority of candidate agents through four dimensions: load factor, performance factor, cost factor, and affinity factor (optional).

[0215] In terms of load factor, the elastic scheduler can calculate the load factor score based on the current load rate. The specific formula for calculating the load factor score is: Load factor score = 1 - current load rate. That is, the load factor score is negatively correlated with the current load rate. The lower the current load rate, the higher the load factor score.

[0216] In terms of performance factor, the elastic scheduler can calculate the performance factor score based on historical success rate and average response time. The specific formula for calculating the performance factor score is: Performance factor score = Historical success rate * (1 / max(average response time, 0.1)), where max(average response time, 0.1) represents the larger value between the average response time and 0.1, that is, the performance factor score is positively correlated with historical success rate and average response time.

[0217] In terms of cost factors, the elastic scheduler can calculate cost factor scores using a predefined formula, which can be configured according to the actual application scenario. For example, the cost factor score calculation formula could be: Cost Factor Score = First Estimated Token Consumption, or Cost Factor Score = First Estimated Token Consumption / Baseline Token Consumption. The baseline token consumption refers to the token consumption used as a benchmark when calculating the cost factor score, and can be obtained based on the token consumption of candidate agents handling tasks of that task type in each execution mode.

[0218] In the affinity factor dimension, the main purpose is to prioritize agent instances that have cached relevant contexts. Based on this, the elastic scheduler will obtain historical task processing records, extract agent usage records from the historical task processing records, and then compare the agent usage records with the candidate agents to determine the affinity factor score of the candidate agents.

[0219] It should be noted that, since the affinity factor is an optional factor, the priority score of the candidate agent can be calculated in the following two ways in this embodiment. In a specific application, the priority score of the candidate agent can be calculated using the following formula: Priority Score = Load Factor Score * w1 + Performance Factor Score * w2 + Cost Factor Score * w3, where w1 + w2 + w3 = 1, w1 is the first weight coefficient, w2 is the second weight coefficient, and w3 is the third weight coefficient, which can be configured according to the actual application scenario. In a specific application, the priority score of the candidate agent can be calculated using the following formula: Priority Score = Load Factor Score * w1 + Performance Factor Score * w2 + Cost Factor Score * w3 + Affinity Factor Score * w4, where w1 + w2 + w3 + w4 = 1, w1 is the first weight coefficient, w2 is the second weight coefficient, w3 is the third weight coefficient, and w4 is the fourth weight coefficient, which can be configured according to the actual application scenario.

[0220] Understandably, when system resources are scarce and the recommended execution mode is core mode, the dynamic priority scheduling algorithm will prioritize agent instances with lower estimated token costs for executing core mode, in order to maximize resource utilization and protect the system. When system resources are abundant and the recommended execution mode is full mode, the dynamic priority scheduling algorithm will prioritize agent instances that can complete full mode tasks with high quality, in order to maximize the quality and depth of successful output.

[0221] Based on the priority scores of multiple candidate agents, the elastic scheduler selects the candidate agent with the highest score as the task processing agent. It distributes content query tasks to the task processing agent by distributing task messages. The task message body explicitly includes the task execution mode and the token_budget (token consumption, i.e., resource constraint information). The elastic scheduler can determine the final token_budget using strategies such as the minimum, average, or load-weighted average budget of the candidate agent in that execution mode.

[0222] In a specific application, such as Figure 9 As shown, after receiving the content query task and execution mode instruction (i.e., task message), the task processing agent will, according to the task execution mode in the task message, retrieve computing resources from the system's computing resources to execute the content query task, obtain the content query result, and return it to the user. Figure 9 As shown, the content query results can be divided into concise results and detailed results. Concise results are generated when the task execution mode is core mode, while detailed results are generated when the task execution mode is full mode.

[0223] In a specific application, the task processing agent receives a task message containing the task execution mode and token_budget. Based on the task execution mode, the agent dynamically adjusts the system prompts to adapt them, and simultaneously adjusts the generation parameters of the large language model. Specifically, the prompt for the core mode might be: "You are an efficient analysis assistant. Please provide the core conclusions and key data in the most concise language within XXX words, without needing to analyze the process or details." The prompt for the full mode might be: "You are a thorough analyst. Please provide a comprehensive analysis report, including data, analysis, comparisons, and recommendations." The generation parameters can be adjusted by setting `max_tokens` directly to `token_budget`, and `temperature` (a parameter controlling the randomness of the generated text, with a value between 0 and 1) may be lower in the core mode to ensure determinism.

[0224] In an exemplary embodiment, taking the evaluation of candidate agents' priorities using three dimensions—load factor, performance factor, and cost factor—in a dynamic priority scoring algorithm as an example, the agent-based task scheduling method of this application is illustrated. The core of the agent-based task scheduling method of this application lies in dynamic decision-making based on real-time state. The specific process can be as follows: Figure 11 As shown, it includes the following steps:

[0225] I. Task Submission: Users submit content query tasks to the system.

[0226] II. Task Parsing and Status Acquisition: After receiving the content query task, the elastic scheduler executes two operations in parallel:

[0227] Analyze task type: Analyze the content query task to determine the type of capability to be invoked (such as financial_analysis).

[0228] Get global resource status: Read the latest global resource status signals (i.e., resource status information) published by the resource detector from a shared cache (such as Redis) or message bus (e.g., {"level": "HIGH_LOAD (high load state)", "mode": "CORE (core mode)"}).

[0229] 3. Obtaining the list of candidate intelligent agents: Based on the parsed task type, the elastic scheduler queries the capability registry center to obtain a list of all candidate intelligent agents (i.e., multiple candidate intelligent agents) that can handle this type of task and are online, along with their real-time metadata (such as current load rate, historical performance, and estimated token cost).

[0230] IV. Dynamic Priority Scoring: This is the core decision-making step of this method. The elastic scheduler runs a multi-objective scoring algorithm for each candidate intelligence. This algorithm comprehensively considers:

[0231] Load factor score: The lower the real-time load of the candidate agent, the higher the score, in order to avoid assigning new tasks to overloaded instances. The specific calculation method can be 1 - current load rate;

[0232] Performance factor score: The higher the historical success rate and the shorter the average response time of the candidate agent, the higher the score, so that more reliable instances are selected first;

[0233] Cost factor score: The cost calculation method is determined based on the current global resource status to estimate the number of tokens. When resources are scarce (core mode recommended), candidate agents with lower estimated token costs for executing the core mode are prioritized. When resources are abundant, candidate agents that can complete the full mode with high quality are prioritized.

[0234] The elastic scheduler calculates the final priority score by weighting and summing the above factor scores for each candidate agent.

[0235] V. Selection and Distribution: The elastic scheduler selects the agent instance with the highest score as the task processing agent and distributes the task message containing the content query task to it. The distributed task message explicitly includes constraints such as the task execution mode (execution_mode) and the token budget (token_budget) for this task.

[0236] VI. Adaptive Agent Instance Execution: After the selected agent instance receives the task message:

[0237] Its internal configuration dynamically switches based on the task execution mode. Specifically, if the task execution mode is CORE, a concise version of the system prompt (i.e., the core mode prompt) and strict generation parameter restrictions are loaded to generate concise results containing only the core conclusions. If the task execution mode is COMPREHENSIVE, a detailed version of the system prompt (i.e., the full mode prompt) and relaxed generation parameters are loaded to generate detailed results containing thorough analysis and reasoning.

[0238] 7. Returning Results: The task processing agent returns the generated results to the elastic scheduler, which then presents them to the user, thus completing the task.

[0239] At this point, the system has completed an automated closed-loop process integrating resource awareness, intelligent scheduling, and elastic execution. This process ensures that the system can make optimal decisions under any resource conditions, maximizing resource utilization and user experience.

[0240] In an exemplary embodiment, an application example of the agent-based task scheduling method of this application is given. This agent-based task scheduling method can be applied to AI assistants, making them "considerate" and "efficient" partners. Users perceive an agent instance that can quickly and reliably provide "just the right" information in any situation. It will not make users wait too long when the system is busy, nor will it provide overly brief answers when the system is idle. For example, a user submits the same request (i.e., a content query task) in different scenarios: "Analyze the main highlights of A Automotive's Q2 financial report." In scenario 1 (sufficient system resources, GPU utilization <30%), the system will automatically select the task execution mode as "full mode." The report obtained by the user is a detailed report (approximately 1000 words) containing detailed revenue / profit data, analysis of each business line, comparison with competitors, future outlook, and a summary of investment recommendations. In scenario B (system resources are scarce, GPU utilization is >85%, high concurrency), the system will automatically switch the task execution mode to "core mode" and allocate a lower token budget to the task. The report obtained by the user is a concise summary (about 150 words) containing only the most critical conclusions (such as "revenue exceeded expectations, but profit margins are under pressure") and 3 core highlights, with extremely fast response speed.

[0241] In summary, the key technical points of the agent-based task scheduling method of this application include:

[0242] First, global resource detection and mode decision-making: The resource detector collects multi-dimensional resource indicators such as GPU utilization, memory usage, and current load rate in real time, and aggregates them into a unified resource status signal (i.e., resource status information) through the built-in rule engine, and decides the execution mode (core mode or full mode) for the whole or the task.

[0243] Second, agent capability registration and negotiation: After each agent instance starts up, it actively broadcasts its structured capability description metadata to the central registration center through a capability negotiation protocol. The registration center maintains a real-time updated pool of agent capabilities containing the real-time load of each agent instance through a heartbeat mechanism. The capability negotiation protocol refers to a lightweight communication specification used for capability broadcasting, state synchronization, and task matching between the central scheduler and multiple agent instances.

[0244] Third, dynamic scheduling based on multi-dimensional evaluation: After receiving a task, the elastic scheduler comprehensively matches the task type, the real-time load of the agent instance, historical performance, and the current system resource status, and calculates the priority through a multi-objective scoring algorithm to achieve optimal task allocation.

[0245] Fourth, the flexible output generator: At the execution level, the task processing agent adaptively adjusts its system prompts and generation parameters based on the task execution mode (core mode or full mode) issued by the scheduler and additional resource constraint information (such as the token limit (i.e., the token budget constraint)) to output results with different levels of detail.

[0246] Understandably, the agent-based task scheduling method of this application, by introducing an adaptive mechanism of resource awareness and capability negotiation, produces the following significant beneficial effects compared to related technologies:

[0247] First, intelligent resource utilization: It realizes the dynamic optimal matching of system computing resources and output content, avoids resource waste and overload risk, and significantly improves system stability and operational economy under large-scale service.

[0248] Secondly, a seamless user experience: users can always get the response speed and content depth that best match the current system capabilities without having to understand complex technical concepts or manually switch modes, resulting in a smooth and consistent experience.

[0249] Third, efficient load balancing: The "capacity negotiation-dynamic scheduling" mechanism ensures that tasks are allocated to the most suitable agent instance in real time, giving full play to the computing potential of the cluster and achieving higher overall throughput and lower average response latency.

[0250] Fourth, enhanced system robustness: The system has the ability to gracefully degrade (downgrade to core mode) or enhance services (upgrade to full mode) based on resource conditions, making it more adaptable to external resource fluctuations and sudden loads, thus improving the robustness of the service.

[0251] Fifth, it lays the framework foundation for resilient artificial intelligence: it provides a general and scalable framework for resilient artificial intelligence systems, which is not only applicable to output detail control, but also lays the foundation for more complex resource adaptation tasks in the future (such as model selection and pipeline optimization).

[0252] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0253] Based on the same inventive concept, this application also provides an agent-based task scheduling apparatus for implementing the agent-based task scheduling method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more agent-based task scheduling apparatus embodiments provided below can be found in the limitations of the agent-based task scheduling method described above, and will not be repeated here.

[0254] In one exemplary embodiment, such as Figure 12 As shown, a task scheduling device based on an intelligent agent is provided, including: a task processing module 1202, a scheduling module 1204, and a task distribution module 1206, wherein:

[0255] The task processing module 1202 is used to, when receiving a content query task, determine multiple candidate intelligent agents according to the task type of the content query task, and obtain the resource status information of the system computing resources;

[0256] The scheduling module 1204 is used to sort the scheduling priority of multiple candidate intelligent agents according to the task type, resource status information and the capability description metadata of each of the multiple candidate intelligent agents, and determine the task processing intelligent agent.

[0257] The task distribution module 1206 is used to determine the task execution mode based on resource status information, generate a task message based on the task execution mode and the content query task, and distribute the task message to the task processing agent. The task processing agent is used to obtain computing resources from the system computing resources to execute the content query task according to the task execution mode in the task message, and obtain the content query result.

[0258] The aforementioned agent-based task scheduling device, upon receiving a content query task, can determine multiple candidate agents capable of handling the task based on the task type. After acquiring the resource status information of the system's computing resources, it can accurately select the task processing agent from among the multiple candidate agents by using the task type, resource status information, and metadata describing the capabilities of each candidate agent, through a scheduling priority ranking method. By determining the task execution mode based on the resource status information, and generating a task message based on the task execution mode and the content query task, the device distributes the task message to the task processing agent. This enables the distribution of content query tasks with determined execution modes. The task processing agent can then retrieve computing resources from the system's computing resources according to the task execution mode in the task message to execute the content query task and obtain the content query result. The entire process can select task processing agents and determine task execution modes based on the resource status information, task types, and capability description metadata of the system's computing resources. This allows the task processing agents to obtain computing resources to execute content query tasks according to the task execution mode, strongly correlate resource status information with scheduling content query tasks, maximize resource utilization, and improve resource utilization efficiency.

[0259] In an exemplary embodiment, the scheduling module is further configured to determine the priority score of each candidate agent based on the task type, resource status information, and the candidate agent's capability description metadata, and to sort the scheduling priority of the multiple candidate agents based on their respective priority scores to determine the task processing agent.

[0260] In an exemplary embodiment, the scheduling module is further configured to, for each candidate agent, determine the agent load parameters and agent performance parameters of the candidate agent from the agent's capability description metadata, and determine a first estimated token consumption from the agent's capability description metadata based on task type and resource status information, determine a load factor score based on the agent load parameters, determine a performance factor score based on the agent performance parameters, determine a cost factor score based on the first estimated token consumption, and determine a priority score of the candidate agent based on the load factor score, performance factor score, and cost factor score.

[0261] In an exemplary embodiment, the scheduling module is further configured to obtain a first weighting coefficient of the load factor score, a second weighting coefficient of the performance factor score, and a third weighting coefficient of the cost factor score, and to weight the load factor score, performance factor score, and cost factor score based on the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient to obtain the priority score of the candidate agent.

[0262] In an exemplary embodiment, the scheduling module is further configured to obtain historical task processing records, extract agent usage records from the historical task processing records, determine the affinity factor score of candidate agents based on the agent usage records, obtain a first weighting coefficient of the load factor score, a second weighting coefficient of the performance factor score, a third weighting coefficient of the cost factor score, and a fourth weighting coefficient of the affinity factor score, and weight the load factor score, performance factor score, cost factor score, and affinity factor score based on the first weighting coefficient, the second weighting coefficient, the third weighting coefficient, and the fourth weighting coefficient to obtain the priority score of the candidate agents.

[0263] In one exemplary embodiment, the resource status information is generated by the resource detector based on the system global status and the suggested execution mode. The system global status and the suggested execution mode are obtained by the resource detector through status determination based on the resource indicator data of the system computing resources. In another exemplary embodiment, the system global status and the suggested execution mode are obtained by the resource detector matching at least one resource indicator quantity in the resource indicator data with the respective status determination conditions of multiple candidate global states in a predefined status determination strategy. In yet another exemplary embodiment, the task processing agent is further configured to generate system prompt words based on the task execution mode in the task message, and based on the system prompt words, retrieve the computing resources from the system computing resources to execute a content query task, thereby obtaining the content query results. The system prompt words are used to indicate the amount of information in the output content query results.

[0264] In an exemplary embodiment, when the task execution mode in the task message is core mode, the core mode prompt word corresponding to core mode is the system prompt word; when the task execution mode is full mode, the full mode prompt word corresponding to full mode is the system prompt word; the full mode prompt word indicates that the amount of information in the output content query result is greater than the amount of information in the output content query result indicated by the core mode prompt word.

[0265] In an exemplary embodiment, the task processing agent is further configured to adjust the content generation parameters of the task processing agent according to the resource constraint information in the task message, and obtain computing resources from the system computing resources to execute the content query task based on the system prompt words and the content generation parameters, thereby obtaining the content query results.

[0266] In an exemplary embodiment, the task distribution module is further configured to obtain resource constraint information for the task processing agent, assemble the task execution mode, resource constraint information, and content query task, and generate a task message.

[0267] In an exemplary embodiment, the task distribution module is further configured to determine, from the capability description metadata of the task processing agent, a second estimated token consumption of the task processing agent in the task execution mode, and a third estimated token consumption of the task processing agent in other execution modes, and generate resource constraint information for the task processing agent based on the second estimated token consumption and the third estimated token consumption.

[0268] In an exemplary embodiment, the agent-based task scheduling device further includes a capability registration module. The capability registration module is used to obtain the capability description metadata of any agent instance from the heartbeat information when it receives the heartbeat information of any agent instance, and record the capability description metadata of any agent instance. The task processing module is also used to match the task type of the content query task with the action type in the capability description metadata of the multiple agent instances when it receives the content query task, to obtain the agent matching result, and to determine multiple candidate agents from the multiple agent instances based on the agent matching result.

[0269] Each module in the aforementioned agent-based task scheduling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can invoke and execute the operations corresponding to each module.

[0270] In one exemplary embodiment, a computer device is provided, which can be a server or a terminal. Taking the computer device as a server as an example, its internal structure diagram can be as follows: Figure 13As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores metadata describing the capabilities of multiple agent instances. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements an agent-based task scheduling method.

[0271] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0272] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0273] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0274] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0275] It should be noted that the data involved in this application (including but not limited to data used for analysis, data stored, data displayed, etc.) are all data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0276] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0277] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0278] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A task scheduling method based on intelligent agents, characterized in that, The method includes: When a content query task is received, multiple candidate agents are determined based on the task type of the content query task, and the resource status information of the system computing resources is obtained. Based on the task type, the resource status information, and the capability description metadata of each of the multiple candidate intelligent agents, the multiple candidate intelligent agents are sorted by scheduling priority to determine the task processing intelligent agent; Based on the resource status information, a task execution mode is determined. Based on the task execution mode and the content query task, a task message is generated and distributed to the task processing agent. The task processing agent is used to obtain computing resources from the system computing resources according to the task execution mode in the task message, execute the content query task, and obtain the content query result.

2. The method according to claim 1, characterized in that, The step of prioritizing and determining the task processing agent based on the task type, resource status information, and capability description metadata of the multiple candidate agents includes: For each candidate agent, a priority score is determined based on the task type, the resource status information, and the candidate agent's capability description metadata. Based on the priority scores of the multiple candidate agents, the multiple candidate agents are sorted by scheduling priority to determine the task processing agent.

3. The method according to claim 2, characterized in that, For each candidate agent, the priority score of the candidate agent is determined based on the task type, the resource status information, and the candidate agent's capability description metadata, including: For each candidate agent, the agent load parameters and agent performance parameters of the candidate agent are determined from the agent's capability description metadata, and the first estimated token consumption is determined from the agent's capability description metadata based on the task type and the resource status information. Determine the load factor score based on the agent's load parameters; Based on the agent's performance parameters, determine the performance factor score; Determine the cost factor score based on the first estimated token consumption; The priority score of the candidate agent is determined based on the load factor score, the performance factor score, and the cost factor score.

4. The method according to claim 3, characterized in that, Determining the priority score of the candidate agent based on the load factor score, the performance factor score, and the cost factor score includes: Obtain the first weighting coefficient of the load factor score, the second weighting coefficient of the performance factor score, and the third weighting coefficient of the cost factor score; Based on the first weighting coefficient, the second weighting coefficient, and the third weighting coefficient, the load factor score, the performance factor score, and the cost factor score are weighted to obtain the priority score of the candidate agent.

5. The method according to claim 3, characterized in that, Determining the priority score of the candidate agent based on the load factor score, the performance factor score, and the cost factor score includes: Obtain historical task processing records, and extract agent usage records from the historical task processing records; Based on the agent's usage records, the affinity factor score of the candidate agent is determined; Obtain the first weighting coefficient of the load factor score, the second weighting coefficient of the performance factor score, the third weighting coefficient of the cost factor score, and the fourth weighting coefficient of the affinity factor score; Based on the first weighting coefficient, the second weighting coefficient, the third weighting coefficient, and the fourth weighting coefficient, the load factor score, the performance factor score, the cost factor score, and the affinity factor score are weighted to obtain the priority score of the candidate agent.

6. The method according to claim 1, characterized in that, The resource status information is generated by the resource detector based on the system global status and the suggested execution mode; the system global status and the suggested execution mode are obtained by the resource detector through status determination based on the resource index data of the system's computing resources.

7. The method according to claim 6, characterized in that, The system global state and the suggested execution mode are obtained by the resource detector matching at least one resource indicator quantity in the resource indicator data with the state determination conditions of multiple candidate global states in the predefined state determination strategy.

8. The method according to any one of claims 1 to 7, characterized in that, The task processing agent is further configured to generate system prompt words based on the task execution mode in the task message, and based on the system prompt words, obtain computing resources from the system computing resources to execute the content query task and obtain content query results; The system prompts are used to indicate the amount of information in the output query results.

9. The method according to claim 8, characterized in that, The method further includes: When the task execution mode in the task message is core mode, the core mode prompt word corresponding to the core mode is the system prompt word; When the task execution mode is full mode, the full mode prompt word corresponding to the full mode is the system prompt word; the information content of the content query result indicated by the full mode prompt word is greater than the information content of the content query result indicated by the core mode prompt word.

10. The method according to claim 8, characterized in that, The task processing agent is further configured to adjust the content generation parameters of the task processing agent according to the resource constraint information in the task message, and obtain computing resources from the system computing resources to execute the content query task based on the system prompt words and the content generation parameters, thereby obtaining the content query result.

11. The method according to any one of claims 1 to 7, characterized in that, The step of generating a task message based on the task execution mode and the content query task includes: Obtain resource constraint information for the task processing agent; Assemble the task execution mode, the resource constraint information, and the content query task to generate a task message.

12. The method according to claim 11, characterized in that, The step of obtaining resource constraint information for the task processing agent includes: From the capability description metadata of the task processing agent, determine the second estimated token consumption of the task processing agent in the task execution mode, and determine the third estimated token consumption of the task processing agent in other execution modes. Based on the second estimated token consumption and the third estimated token consumption, resource constraint information is generated for the task processing agent.

13. The method according to any one of claims 1 to 7, characterized in that, The method further includes: When a heartbeat message is received from any agent instance, the capability description metadata of the agent instance is obtained from the heartbeat message and recorded. When a content query task is received, multiple candidate agents are determined based on the task type of the content query task, including: When a content query task is received, the task type of the content query task is matched with the action type in the capability description metadata of multiple agent instances to obtain the agent matching result. Based on the agent matching results, multiple candidate agents are determined from the multiple agent instances.

14. A task scheduling device based on intelligent agents, characterized in that, The device includes: The task processing module is used to, when a content query task is received, determine multiple candidate intelligent agents according to the task type of the content query task, and obtain the resource status information of the system computing resources; The scheduling module is used to sort the scheduling priority of the multiple candidate intelligent agents according to the task type, the resource status information and the capability description metadata of the multiple candidate intelligent agents, and determine the task processing intelligent agent. The task distribution module is used to determine the task execution mode based on the resource status information, generate a task message based on the task execution mode and the content query task, and distribute the task message to the task processing agent; the task processing agent is used to obtain computing resources from the system computing resources to execute the content query task according to the task execution mode in the task message, and obtain the content query result.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 13.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 13.

Images