A multi-modal large model multi-agent collaborative scheduling and distribution method

Abstract

The application discloses a multi-modal large model multi-agent collaborative scheduling and distribution method, and relates to the technical field of agent deployment.The steps of the method include: receiving a complex task input by a user, decomposing the complex task into atomic subtasks, and generating a task decomposition graph; establishing an agent capability evaluation model, obtaining the comprehensive capability scores of the agents according to the capability behaviors of the agents in the candidate agent set; using the subtask types and the comprehensive capability scores of the agents to dynamically match and assign roles through a decision function, and generating a candidate agent set; and constructing a task-agent ant colony distribution algorithm according to the task decomposition graph and the comprehensive capability scores of the agents, and solving a distribution scheme.The application solves the problem that the existing multi-agent system lacks a dynamic role adjustment mechanism, cannot flexibly cooperate according to task progress and agent state, and is difficult to meet the requirements of an efficient and flexible task processing requirement of a multi-modal large model application development platform.

Description

Technical Field

[0001] This invention relates to the field of agent allocation technology, specifically a multimodal large-scale model multi-agent collaborative scheduling and allocation method. Background Technology

[0002] Multi-agent systems are increasingly being used for the collaborative scheduling and execution of complex tasks due to their advantages of being distributed, highly concurrent, and autonomous. However, current methods for integrating multimodal large models with multi-agent systems to automatically decompose, schedule, and allocate highly complex tasks with heterogeneous information flows still have many shortcomings.

[0003] For example, in some traditional task scheduling schemes, tasks are often assigned to agents in a pre-set order without fully considering the differences in the capabilities of agents and the complexity of the tasks. Once an agent fails, a transient timeout occurs, or the task priority changes abruptly, the system often cannot reschedule or reselect roles in a timely manner, resulting in a significant decrease in overall throughput and reliability.

[0004] In terms of role allocation, in most frameworks, the roles of master / sub agents are often pre-configured statically, lacking dynamic matching based on task type, agent expertise, and historical performance, making it difficult to adapt to complex and ever-changing task requirements. This leads to unreasonable task allocation when dealing with complex tasks, with some agents being overloaded while others are idle, reducing the overall operating efficiency of the multi-agent system.

[0005] Although some studies have attempted to apply metaheuristic algorithms (such as genetics, particle swarm optimization, and ant colony optimization) to multi-agent task allocation, most of them only focus on a single objective (minimizing time or cost) and do not adequately consider multi-dimensional factors such as resource consumption and allocation fairness. Furthermore, they lack fine-grained function design that takes into account the capabilities of agents, resulting in slow convergence speed and a tendency to get trapped in local optima.

[0006] Meanwhile, due to the lack of a dynamic role adjustment mechanism, it is impossible to flexibly collaborate based on task progress and agent status, making it difficult to meet the requirements of multimodal large model application development platforms for efficient and flexible task processing.

[0007] Therefore, this invention provides a multimodal large-scale model multi-agent cooperative scheduling and allocation method. Summary of the Invention

[0008] The purpose of this invention is to provide a multimodal large-scale model multi-agent cooperative scheduling and allocation method to solve the existing problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a multimodal large-scale model multi-agent cooperative scheduling and allocation method, comprising the following steps: S1. Receive complex tasks input by the user, break down the complex tasks into atomic-level subtasks, and generate a task decomposition diagram. S2. Establish an agent capability evaluation model and obtain a comprehensive capability score for each agent based on the capabilities and behaviors of each agent in the candidate agent set. S3 is used to dynamically match and assign roles based on the subtask type and the comprehensive ability score of the agent, and generate a set of candidate agents through a decision function. S4. Based on the task decomposition diagram and the comprehensive ability score of each agent, construct a task-agent ant colony allocation algorithm and solve the allocation scheme. A further improvement of this invention is that the task decomposition graph includes a set of tasks for a complex task input by the user. And set a maximum subtask length threshold. and minimum subtask length threshold Map the semantics of each sentence to the initial subtask. Construct a directed acyclic graph based on the initial subtasks. Each of the split subtasks All are considered as nodes in the directed acyclic graph, where the set of nodes is... Represents the set of all subtasks and edges. Indicates the dependencies between subtasks; For each pair of subtasks A bidirectional attention model is used to calculate semantic similarity and causal probability. Let be the dependency coefficient between subtasks; if Greater than the set confidence threshold Then, directed edges are established in the DAG, and tasks are merged using Tarjan's strongly connected component algorithm; finally, a DAG is obtained, which is used for topological sorting, and after generating the global subtask execution order, an ordered sequence is output to generate a task decomposition graph.

[0010] A further improvement of the present invention is that step S3 includes a role definition unit and a decision function creation unit. The role definition unit includes a predefined set of roles, including instructors and executors, and establishes a list of responsibilities for each role. Maintain a capability profile for each agent. When a task is generated, extract the task type and apply it through a decision function. Agent corresponding to the maximum value With mentor The combination of these elements enables intelligent agents The leader acts as the guide; and calculates the combination of decision functions for all combinations of agents and executors, sorts them from highest to lowest, obtains the agent matching sequence, and selects the top n agents as the task. Candidate intelligent agents.

[0011] A further improvement of this invention is that the decision function creation unit is used to construct a decision function, which is expressed as follows: first, the suitability between the agent's capabilities and role requirements is calculated. ,in This represents the minimum capability threshold set for role R; secondly, it calculates the real-time load penalty. , This indicates the current load rate of the agent. Represents the penalty coefficient; and the degree of fit between the agent's capabilities and role requirements. With real-time load penalty The decision function is obtained by weighted summation.

[0012] A further improvement of this invention is that the task-agent ant colony allocation algorithm includes: S41. Initialize the ant count M and pheromone matrix. and initial constant Heuristic function Importance of pheromones Importance of heuristic functions Initial volatility coefficient Initial pheromone intensity ; S42, Regarding ants Calculate each subtask sequentially. Select candidate agents Selection probability A smart agent is randomly selected through a roulette wheel betting method. As a sub-task of ant h The allocation is selected, and then the next subtask is allocated, until all subtasks have been allocated, resulting in the ants. Candidate agent selection path ; S43, Get all ants The selected path; S44. After each ant selection, run the reputation-volatility coefficient update strategy to update the volatility coefficient and perform localized volatility. ; S45. After each iteration, obtain the total cost for each ant. ;according to Output the optimal solution within the iteration ; S46. When the maximum number of iterations is reached... When the iteration ends, output the optimal solution at that point.

[0013] A further improvement of this invention is that the reputation-volatility coefficient update strategy specifically includes ant... In each selection of candidate agents Then, calculate the candidate agents. Reputation rating The volatile coefficient is updated based on the reputation score, and the updated volatile coefficient is expressed as follows: ,in, This indicates the set minimum evaporation rate. This indicates the minimum set evaporation rate.

[0014] A further improvement of this invention is that the reputation score is obtained by recording the historical task completion status of the intelligent agent j through the blockchain, including on-time rate, accuracy rate, and resource waste rate. The historical reputation score is obtained by weighted summation of the task completion status, and the next reputation score is obtained by predicting the next performance through an LSTM time series model, which serves as the input for the reputation-volatility coefficient update strategy.

[0015] A further improvement of this invention is that the heuristic function is derived from candidate agents. The comprehensive capability score of the intelligent agent is determined, and candidate intelligent agents are identified. The heuristic function is specifically represented as follows: ; in, It is Assigned to The cost is borne by the intelligent agent. Process this subtask The time and resource consumption factors are taken into account.

[0016] A further improvement of this invention is that the comprehensive ability score of the agent is obtained by weighted summation of the indicators of each ability behavior after normalization. Capabilities and behaviors include computing resources and professional knowledge reserves.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention firstly enables the system to perceive the agent's load and fault status in real time during subtask execution through a dynamic role allocation and rescheduling mechanism, and quickly adjust the scheduling strategy according to the task type and agent capabilities, thereby adapting to multimodal task scenarios with different complexities and urgency levels. 2. Secondly, a reputation evaporation coefficient strategy driven by the historical reputation score of the agent is introduced into the ant colony algorithm, which makes the pheromone of high reputation agents evaporate more slowly and low reputation agents evaporate more quickly, thereby accelerating the iterative aggregation of high-quality nodes and improving the scheduling convergence speed and scheme quality. 3. Integrate the agent's comprehensive capabilities and normalized allocation costs into the heuristic function, taking into account both capability bias and resource consumption, to guide ants to efficiently search for the optimal allocation scheme in the "capability + cost" space. Attached Figure Description

[0018] Figure 1 This is a flowchart of a multimodal large-scale model multi-agent cooperative scheduling and allocation method according to the present invention; Figure 2 This is a flowchart of the task-agent ant colony allocation algorithm in the multimodal large model multi-agent cooperative scheduling and allocation method of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0020] The term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone.

[0021] Example 1 Figure 1 The flowchart of a multimodal large-scale model multi-agent cooperative scheduling and allocation method disclosed in this embodiment is shown. The steps are as follows: S1. Receive complex tasks input by the user, break down the complex tasks into atomic-level subtasks, and generate a task decomposition graph; the task decomposition graph includes multimodal instructions, including text, images, and schematic diagrams, and performs sentence segmentation, entity recognition, and intent classification, outputting the preprocessing results; let the input task set be... And set a maximum subtask length threshold. and minimum subtask length threshold By combining a domain knowledge base (Ontology) and a Prompttuned LLM, the semantics of each sentence are mapped to the initial subtask. Construct a directed acyclic graph based on the initial subtasks. Each of the split subtasks All are considered as nodes in the directed acyclic graph, where the set of nodes is... Represents the set of all subtasks and edges. Indicates the dependencies between subtasks; For each pair of subtasks A bidirectional attention model is used to calculate semantic similarity and causal probability. Let be the dependency coefficient between subtasks; if Greater than the set confidence threshold Then, directed edges are established in the DAG, and tasks are merged using Tarjan's strongly connected component algorithm; finally, a DAG is obtained, which is used for topological sorting, and after generating the global subtask execution order, an ordered sequence is output to generate a task decomposition graph.

[0022] in, Indicates if to do first Do it again Dependency score: The closer to 1, the more the model believes there is a "sequential" or "causal" dependency; the closer to 0, the more the model believes that the two tasks may be independent or can be done in parallel.

[0023] Set a confidence threshold (For example, 0.5 or more strictly 0.7), meaning "it is only treated as a dependency edge if the model considers the probability sufficiently high." After performing the above BiAttention calculation for each pair of subtasks: if Just add a directed edge to the DAG. ; if If no edge is added, the two tasks can be considered as having "no direct priority relationship".

[0024] For example, subtasks "Download medical image dataset"; subtask "Preprocessing image data"; BiAttention will focus on keywords such as "download," "data," and "preprocessing," and if the model outputs... ;like ,but Connect a line in DAG This ensures that "preprocessing" is performed only after "downloading".

[0025] Tasks must be assigned in topological order (i.e., a task can only be assigned after all its predecessor tasks have been assigned).

[0026] S2. Establish an agent capability evaluation model and obtain a comprehensive capability score for each agent based on the capabilities and behaviors of each agent in the agent set. The overall capability score of the agent is obtained by weighted summation of the indicators of each capability behavior after normalization. The capabilities and behaviors include computing resources and professional knowledge reserves; computing resources include CPU FLOPS, GPU TensorOPS, and available memory, while professional knowledge reserves include domain knowledge graph coverage and the number of fine-tuned model training samples.

[0027] S3. Based on the subtask type and the agent's overall capability score, dynamically match and assign roles through a decision function; and collect the execution progress of each subtask and the agent's status in real time. When a timeout or failure is detected, immediately trigger task rescheduling or role reselection. It includes a role definition unit and a decision function creation unit. The role definition unit includes a predefined set of roles, including instructors and executors, and establishes a list of responsibilities for each role. Maintain a capability profile for each agent. When a task is generated, extract the task type and apply it through a decision function. Agent corresponding to the maximum value With mentor The combination of these elements enables intelligent agents The leader acts as the guide; and calculates the combination of decision functions for all combinations of agents and executors, sorts them from highest to lowest, obtains the agent matching sequence, and selects the top n agents as the task. Candidate intelligent agents.

[0028] The decision function creation unit is used to construct a decision function, which is expressed as follows: First, the suitability between the agent's capabilities and role requirements is calculated. ,in This represents the minimum capability threshold set for role R; secondly, it calculates the real-time load penalty. , This indicates the current load rate of the agent. Represents the penalty coefficient; and the degree of fit between the agent's capabilities and role requirements. With real-time load penalty The decision function is obtained by weighted summation.

[0029] S4. Based on the task decomposition diagram and the comprehensive ability scores of each agent, construct a task-agent ant colony allocation algorithm to solve for the allocation scheme, such as... Figure 2 As shown, the specific steps of the task-agent ant colony allocation algorithm include: S41. Initialize the ant count M and pheromone matrix. and initial constant Heuristic function Importance of pheromones Importance of heuristic functions Initial volatility coefficient Initial pheromone intensity ; This indicates that, under the current algorithm state, the subtasks are... Assigned to intelligent agents The residual "odor concentration (pheromone intensity)" along this path Control the extent to which ants refer to historical pheromones when selecting their mates; Control the extent to which ants refer to the heuristic value of the current path itself; Indicates if The larger the scale, the more it "relies" on the tracks left by ants in the past; if Larger networks tend to have greater "trust" in the low-cost characteristics of the current path; Heuristic functions are derived from candidate agents. The comprehensive capability score of the intelligent agent is determined, and candidate intelligent agents are identified. The heuristic function is specifically represented as follows: ; in, It is Assigned to Even with slightly higher costs, agents with high capability scores can still receive extra points in probability calculations, which is determined by the agent's cost. Process this subtask The determination is based on a combination of time and resource consumption factors, including taking data from the past Q times. Perform similar or identical tasks The average actual time taken is used as the estimated time, and the normalized CPU cycle consumption is used as the average time taken. Normalized memory usage (MB) Normalized GPU computational cost (FLOPs) Normalized I / O read / write volume (MB) The cost is obtained by performing a weighted summation and then a weighted summation with the estimated time.

[0030] S42, Regarding ants Calculate each subtask sequentially. Select candidate agents Selection probability A smart agent is randomly selected through a roulette wheel betting method. As a sub-task of ant h The allocation is selected, and then the next subtask is allocated, until all subtasks have been allocated, resulting in the ants. Candidate agent selection path ; S43, Get all ants The selected path; S44. After each ant selection, run the reputation-volatility coefficient update strategy to update the volatility coefficient. The reputation-volatility coefficient update strategy specifically includes Ant Group. In each selection of candidate agents Then, calculate the candidate agents. Reputation rating The reputation score is generated by recording the historical task completion status of agent j through the blockchain, including on-time rate, accuracy rate, and resource waste rate. The historical reputation score is obtained by weighted summation of task completion status. This historical score is then used to predict the next performance using an LSTM time-series model, which serves as the input for the reputation-volatile coefficient update strategy. The volatile coefficient is updated based on the reputation score, and the updated volatile coefficient is expressed as... ,in, This indicates the set minimum evaporation rate. This indicates the minimum set evaporation rate.

[0031] In this invention, if a certain intelligent agent An agent with consistently good performance (high reputation) needs its pheromones to "evaporate more slowly," making it easier for subsequent ants to follow its path; conversely, for agents with poor performance, pheromones should "evaporate more quickly" to reduce the likelihood of subsequent ants taking that path. At this time, it indicates extremely high credibility. Pheromones are almost non-volatile and continuously enhance their effects; when At this time, it indicates extremely poor credit. The pheromones evaporate quickly, further encouraging the exploration of other pathways; Then localized volatilization occurs. ; S45. After each iteration, obtain the total cost for each ant. ;according to Output the optimal solution within the iteration ; S46. When the maximum number of iterations is reached... When the iteration ends, output the optimal solution at that point.

[0032] S5 transmits task instructions, execution progress, and evaluation results among agents based on a structured message protocol, and ensures communication reliability and state synchronization through message queues and heartbeat mechanisms.

[0033] Establish a semantic understanding-based role interaction protocol, where agents interact through structured communication content, defining the message format as follows: ,in It contains structured information such as task instructions, execution progress, and evaluation results. The supervisor agent sends task instructions and guidance information to the executor agent through this protocol. The executor agent provides feedback on task execution progress and results to the supervisor agent. The evaluator agent sends its evaluation results to the supervisor agent for decision-making and adjustments.

[0034] The threshold and weight settings can be set by default according to the present invention, or they can be set by those skilled in the art.

[0035] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0036] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0037] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0038] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0039] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A multimodal large-scale model multi-agent cooperative scheduling and allocation method, characterized in that: Includes the following steps: S1. Receive complex tasks input by the user, break down the complex tasks into atomic-level subtasks, and generate a task decomposition diagram. S2. Establish an agent capability evaluation model and obtain a comprehensive capability score for each agent based on the capabilities and behaviors of each agent in the candidate agent set. S3 is used to dynamically match and assign roles based on the subtask type and the comprehensive ability score of the agent, and generate a set of candidate agents through a decision function. S4. Based on the task decomposition diagram and the comprehensive ability score of each agent, construct a task-agent ant colony allocation algorithm and solve the allocation scheme. The task decomposition graph includes a set of tasks for a complex task input by the user. And set a maximum subtask length threshold. and minimum subtask length threshold Map the semantics of each sentence to the initial subtask. ; Construct a directed acyclic graph based on the initial subtasks Each of the split subtasks All are considered as nodes in the directed acyclic graph, where the set of nodes is... Represents the set of all subtasks and edges. Indicates the dependencies between subtasks; For each pair of subtasks A bidirectional attention model is used to calculate semantic similarity and causal probability. Let be the dependency coefficient between subtasks; if Greater than the set confidence threshold Then, directed edges are established in the DAG, and tasks are merged using Tarjan's strongly connected component algorithm; finally, a DAG is obtained, which is used for topological sorting, and after generating the global subtask execution order, an ordered sequence is output to generate a task decomposition graph. The task-agent ant colony allocation algorithm includes: S41. Initialize the ant count M and pheromone matrix. and initial constant Heuristic function Importance of pheromones Importance of heuristic functions Initial volatility coefficient Initial pheromone intensity ; S42, Regarding ants Calculate each subtask sequentially. Select candidate agents Selection probability A smart agent is randomly selected through a roulette wheel betting method. As a sub-task of ant h The allocation is selected, and then the next subtask is allocated, until all subtasks have been allocated, resulting in the ants. Candidate agent selection path ; S43, Get all ants The selected path; S44. After each ant selection, run the reputation-volatility coefficient update strategy to update the volatility coefficient and perform localized volatility. ; S45. After each iteration, obtain the total cost for each ant. ;according to Output the optimal solution within the iteration ; S46. When the maximum number of iterations is reached... When the iteration ends, output the optimal solution at that point. The reputation-volatility coefficient update strategy specifically includes Ant Financial. In each selection of candidate agents Then, calculate the candidate agents. Reputation rating The volatile coefficient is updated based on the reputation score, and the updated volatile coefficient is expressed as follows: ,in, This indicates the set minimum evaporation rate. This indicates the minimum set evaporation rate; The reputation score is obtained by recording the historical task completion status of agent j through the blockchain, including on-time rate, accuracy rate, and resource waste rate. The historical reputation score is obtained by weighted summing of the task completion status. The next reputation score is obtained by predicting the next performance through an LSTM time series model, which serves as the input for the reputation-volatility coefficient update strategy.

2. The method for multimodal large-scale model multi-agent cooperative scheduling and allocation according to claim 1, characterized in that: Step S3 includes a role definition unit and a decision function creation unit. The role definition unit includes a predefined set of roles, including instructors and executors, and establishes a list of responsibilities for each role. Maintain a capability profile for each agent. When a task is generated, extract the task type and apply it through a decision function. Agent corresponding to the maximum value With mentor The combination of these elements enables intelligent agents The leader acts as the guide; and calculates the combination of decision functions for all combinations of agents and executors, sorts them from highest to lowest, obtains the agent matching sequence, and selects the top n agents as the task. Candidate intelligent agents.

3. The method for multimodal large-scale model multi-agent cooperative scheduling and allocation according to claim 2, characterized in that: The decision function creation unit is used to construct a decision function, which is expressed as follows: First, the suitability between the agent's capabilities and role requirements is calculated. ,in This represents the minimum capability threshold set for role R; secondly, it calculates the real-time load penalty. , This indicates the current load rate of the agent. Represents the penalty coefficient; and the degree of fit between the agent's capabilities and role requirements. With real-time load penalty The decision function is obtained by weighted summation.

4. The multimodal large-scale model multi-agent cooperative scheduling and allocation method according to claim 3, characterized in that: The heuristic function is derived from candidate agents. The comprehensive capability score of the intelligent agent is determined, and candidate intelligent agents are identified. The heuristic function is specifically represented as follows: ； in, It is Assigned to The cost is borne by the intelligent agent. Process this subtask The time and resource consumption factors are taken into account.

5. The method for multimodal large-scale model multi-agent cooperative scheduling and allocation according to claim 1, characterized in that: The overall capability score of the agent is obtained by weighted summation of the indicators of each capability behavior after normalization. Capabilities and behaviors include computing resources and professional knowledge reserves.

Images