A method, medium, and device for constructing a multi-agent cooperative system

Abstract

This application discloses a method, medium, and device for constructing a multi-agent collaborative system. The method includes: modeling a target business scenario and constructing a task dependency graph; setting initial collaboration strength weights for atomic task pairs with collaborative relationships, and generating a multi-stage collaborative training course from easy to difficult based on the collaborative complexity of the atomic tasks; controlling agents to execute tasks in the training course, collecting interaction behavior data, calculating the overall task completion efficiency, dynamically updating the collaboration weights, and inputting the interaction data and updated weights into a reinforcement learning model to iteratively optimize the collaboration strategy; and finally, solidifying the converged collaboration strategy into the constructed multi-agent collaborative system. This application, by combining course learning and reinforcement learning, can significantly improve the collaboration efficiency and robustness of the system in complex business scenarios.

Description

Technical Field

[0001] This application belongs to the field of multi-agent system cooperative control technology, specifically relating to a method, medium and device for constructing a multi-agent cooperative system. Background Technology

[0002] In the field of multi-agent systems, enabling multiple agents to collaborate efficiently to complete tasks in complex business scenarios is a key challenge. Existing methods typically rely on pre-defined fixed rules or simple negotiation mechanisms to guide collaboration between agents. However, these methods often exhibit insufficient adaptability when faced with dynamically changing task dependencies and resource competition. Specifically, due to the lack of quantitative evaluation of inter-task collaboration relationships and dynamic optimization mechanisms based on actual interaction behavior, agents are prone to excessive negotiation rounds and decision conflicts when performing task handover and conflict arbitration, resulting in low overall system task completion efficiency and failing to meet the requirements of collaborative efficiency and adaptability in complex application scenarios. Therefore, there is an urgent need for a technical solution that can systematically construct efficient and adaptive multi-agent collaborative systems. Summary of the Invention

[0003] In view of the above problems, this application provides a technical solution for constructing a multi-agent collaborative system to solve the technical problems of low agent collaboration efficiency, many conflicts, and difficulty in adapting to complex business scenarios caused by the lack of a systematic collaborative strategy training method in the prior art.

[0004] To achieve the above objectives, in a first aspect, this application provides a method for constructing a multi-agent cooperative system, the method comprising the following steps:

[0005] S1: Model the target business scenario to obtain a task set containing multiple atomic tasks, and construct a task dependency graph based on the dependency relationship between atomic tasks;

[0006] S2: For each pair of atomic tasks with a cooperative relationship in the task dependency graph, set an initial cooperation strength weight, and calculate the cooperation complexity of each atomic task based on the task dependency graph. Sort all atomic tasks in order from low to high according to the cooperation complexity to generate a multi-stage collaborative training course.

[0007] S3: Control multiple agents to perform tasks in the multi-stage collaborative training course. The multiple agents constitute a collaborative unit and execute an optimization loop for the collaborative strategy during the training process, specifically including:

[0008] S31: Collect interaction behavior data between multiple intelligent agents. The interaction behavior data includes the number of negotiation rounds between the initiator and the receiver during the task handover process, the frequency of decision conflict events caused by resource competition, and the average idle time of the intelligent agent in the task waiting state.

[0009] S32: Calculate the overall task completion efficiency of the collaborative unit, and dynamically update the initial collaboration intensity weight based on the negotiation rounds and the frequency of decision conflict events;

[0010] S33: Input the interaction behavior data and the updated cooperation strength weight into the reinforcement learning model, use the overall task completion efficiency as the reward signal, iteratively optimize the cooperation strategy, and use the optimized cooperation strategy as the initial cooperation strategy for the next stage of task execution. The cooperation strategy is used to guide the agent to perform task handover and conflict arbitration.

[0011] S4: After the multi-stage collaborative training course is fully executed and the optimization loop of the collaborative strategy converges, the final output collaborative strategy is solidified into a completed multi-agent collaborative system.

[0012] In a second aspect, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-agent cooperative system construction method as described in the first aspect of this application.

[0013] In a third aspect, this application provides an electronic device having a computer program stored thereon, including a processor and a storage medium, wherein the computer program is stored on the storage medium, and when executed by the processor, the computer program implements the multi-agent cooperative system construction method as described in the first aspect of this application.

[0014] Unlike existing technologies, the above technical solution involves a method, medium, and device for constructing a multi-agent collaborative system. The method includes: modeling a target business scenario and constructing a task dependency graph; setting initial collaboration strength weights for atomic task pairs with collaborative relationships, and generating a multi-stage collaborative training course from easy to difficult based on the collaborative complexity of the atomic tasks; controlling agents to execute tasks in the training course, collecting interaction behavior data, calculating overall task completion efficiency, dynamically updating collaboration weights, and inputting the interaction data and updated weights into a reinforcement learning model to iteratively optimize the collaboration strategy; finally, solidifying the converged collaboration strategy into the constructed multi-agent collaborative system. This application solves the problems of low collaboration efficiency, numerous conflicts, and poor adaptability in existing multi-agent systems due to a lack of systematic training by combining course learning and reinforcement learning, and can significantly improve the collaboration efficiency and robustness of the system in complex business scenarios.

[0015] The above description of the invention is merely an overview of the technical solution of this application. In order to enable those skilled in the art to better understand the technical solution of this application and to implement it based on the description and drawings, and to make the above-mentioned objectives and other objectives, features and advantages of this application easier to understand, the following description is provided in conjunction with the specific embodiments and drawings of this application. Attached Figure Description

[0016] The accompanying drawings are only used to illustrate the principles, implementation methods, applications, features, and effects of specific embodiments of this application and other related content, and should not be considered as limitations on this application.

[0017] In the accompanying drawings of the instruction manual:

[0018] Figure 1 This is a flowchart of the method for constructing a multi-agent cooperative system according to the first exemplary embodiment of this application;

[0019] Figure 2 This is a flowchart of the method for constructing a multi-agent cooperative system according to the second exemplary embodiment of this application;

[0020] Figure 3 This is a flowchart of the method for constructing a multi-agent cooperative system according to the third exemplary embodiment of this application;

[0021] Figure 4 This is a flowchart of the multi-agent cooperative system construction method described in the fourth exemplary embodiment of this application;

[0022] Figure 5 This is a flowchart of the multi-agent cooperative system construction method described in the fifth exemplary embodiment of this application;

[0023] Figure 6 This is a flowchart of the method for constructing a multi-agent cooperative system according to the sixth exemplary embodiment of this application;

[0024] Figure 7 This is a flowchart of the method for constructing a multi-agent cooperative system according to the seventh exemplary embodiment of this application;

[0025] Figure 8 This is a flowchart of the method for constructing a multi-agent cooperative system according to the eighth exemplary embodiment of this application;

[0026] Figure 9 This is a flowchart of the multi-agent cooperative system construction method described in the ninth exemplary embodiment of this application;

[0027] Figure 10 This is a schematic diagram of an electronic device according to an exemplary embodiment of this application;

[0028] The reference numerals used in the above figures are explained as follows:

[0029] 10. Electronic equipment; 101. Processor; 102. Storage medium. Detailed Implementation

[0030] To explain in detail the possible application scenarios, technical principles, specific feasible solutions, and the objectives and effects that this application can achieve, the following detailed description is provided in conjunction with the listed specific embodiments and accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of this application, and are therefore only examples, and should not be used to limit the scope of protection of this application.

[0031] like Figure 1 As shown, in a first aspect, this application provides a method for constructing a multi-agent cooperative system, the method comprising:

[0032] S1: Model the target business scenario to obtain a task set containing multiple atomic tasks, and construct a task dependency graph based on the dependency relationship between atomic tasks;

[0033] S2: For each pair of atomic tasks with a cooperative relationship in the task dependency graph, set an initial cooperation strength weight, and calculate the cooperation complexity of each atomic task based on the task dependency graph. Sort all atomic tasks in order from low to high according to the cooperation complexity to generate a multi-stage collaborative training course.

[0034] S3: Control multiple agents to perform tasks in the multi-stage collaborative training course. The multiple agents constitute a collaborative unit and execute an optimization loop for the collaborative strategy during the training process.

[0035] S4: After the multi-stage collaborative training course is fully executed and the optimization loop of the collaborative strategy converges, the final output collaborative strategy is solidified into a completed multi-agent collaborative system.

[0036] like Figure 2 As shown, step S3 specifically includes:

[0037] S31: Collect interaction behavior data between multiple intelligent agents. The interaction behavior data includes the number of negotiation rounds between the initiator and the receiver during the task handover process, the frequency of decision conflict events caused by resource competition, and the average idle time of the intelligent agent in the task waiting state.

[0038] S32: Calculate the overall task completion efficiency of the collaborative unit, and dynamically update the initial collaboration intensity weight based on the negotiation rounds and the frequency of decision conflict events;

[0039] S33: Input the interaction behavior data and the updated cooperation strength weights into the reinforcement learning model, use the overall task completion efficiency as the reward signal, iteratively optimize the cooperation strategy, and use the optimized cooperation strategy as the initial cooperation strategy for the next stage of task execution. The cooperation strategy is used to guide the agent to perform task handover and conflict arbitration.

[0040] In this embodiment, the target business scenario refers to a specific application scenario that requires multiple intelligent agents to collaborate in completing a task, such as the order processing scenario of an e-commerce platform (including sub-tasks such as order review, inventory deduction, and logistics allocation) and the production line collaboration scenario in intelligent manufacturing (including sub-tasks such as parts processing, assembly, and quality inspection), which have clear task processes and business rules.

[0041] An atomic task is the smallest unit of task that can be executed independently by a single intelligent agent. It forms the basis of complex business tasks. For example, in the order processing scenario, "order information verification" and "inventory quantity query" are both atomic tasks.

[0042] A task dependency graph is a structure that graphically represents the dependencies between atomic tasks. Nodes in the graph represent atomic tasks, and edges represent the dependencies between tasks (e.g., "order approval completed" is a prerequisite for "inventory deduction", and the two are connected by an edge).

[0043] The collaboration strength weight is used to quantify the degree of collaboration between a pair of atomic tasks that have a collaborative relationship. The higher the weight, the more interaction and coordination are required between the two tasks during the collaboration process.

[0044] Collaboration complexity is an indicator that measures the complexity of a single atomic task in a collaborative process. It is positively correlated with the in-degree (the number of dependent edges pointing to the task), the out-degree (the number of dependent edges originating from the task), and the number of agents competing for resources. The higher the in-degree / out-degree and the more agents competing for resources, the higher the collaboration complexity.

[0045] Multi-stage collaborative training courses refer to training sequences based on the sorting of atomic task collaboration complexity from low to high, simulating the human learning process of "from easy to difficult", allowing the agent to first master simple collaborative tasks and then gradually challenge complex collaborative tasks.

[0046] A collaborative unit is a group of agents temporarily formed during the training phase to perform specific training tasks. The number and type of agents are determined according to the requirements of the current phase. For example, in a simple collaborative phase, only 2 agents may be needed to form a collaborative unit; in a complex collaborative phase, 5-8 agents are needed to form a collaborative unit.

[0047] Interactive behavior data refers to data generated by agents during collaborative task execution that reflects the collaborative state. This includes task handover negotiation rounds (e.g., the number of times agent A communicates and confirms with agent B when handing over a task), frequency of decision conflict events (e.g., the number of times conflicts occur when multiple agents simultaneously request the same device resource), and average idle time of agents (e.g., the average time agent C waits for the next task assignment after completing the current task).

[0048] Overall task completion efficiency is the core indicator for measuring the effectiveness of collaborative units in completing tasks. It is the weighted average of the average task completion speed (the number of atomic tasks completed per unit time) and the average task queue clearing time (the average time from the generation of the task queue to the completion of all tasks). The weights can be preset according to the needs of the business scenario (e.g., in the emergency work order processing scenario, the weight of the average task completion speed is set to 0.6, and the weight of the average task queue clearing time is set to 0.4).

[0049] Reinforcement learning models are machine learning models used to optimize collaboration strategies. Through a "state-action-reward" loop mechanism, the model learns the optimal strategy by interacting with the environment (i.e., the collaborative task scenario). Here, the environment state includes interaction behavior data and collaboration intensity weights, actions include task handover decisions and conflict arbitration decisions, and rewards are determined based on the overall task completion efficiency.

[0050] The collaboration strategy is a set of rules used to guide multi-agent collaboration. It is a set of action strategies of all agents (the probability distribution of an individual agent choosing an action in the current state). Specifically, it is used to manage task handover (such as deciding whether to approve a handover request based on the historical success rate of task handover and the current load of the recipient) and conflict arbitration (such as allocating priority access to resources based on the current task criticality and historical resource usage efficiency of agents competing for resources).

[0051] In step S1, a comprehensive analysis of the target business scenario is first conducted to identify the smallest task units (i.e., atomic tasks) in the business process and form a task set. Then, the relationship between each atomic task and other tasks is analyzed (e.g., "parts processing" can only be executed after "raw material procurement" is completed, i.e., there is a time dependency; "product quality inspection" needs to obtain the data of "assembly completed" before it can be performed, i.e., there is a data dependency). Based on these dependencies, a task dependency graph is constructed to ensure that the task execution order conforms to the business logic.

[0052] In step S2, for atomic task pairs (task pairs with collaborative relationships) directly connected by dependency edges in the task dependency graph, an initial collaboration strength weight is set in combination with task attributes (such as dependency type, execution time, and required resources). The magnitude of the weight value reflects the level of collaboration requirements of the task.

[0053] When generating multi-stage training courses, the collaborative complexity of each atomic task is first calculated (based on in-degree, out-degree, and the number of agents competing for resources). Then, the atomic tasks are sorted from low to high collaborative complexity to form multi-stage training courses. For example, atomic tasks with a collaborative complexity of 1-3 are assigned to the first stage (simple collaborative stage), those with a complexity of 4-6 are assigned to the second stage (medium collaborative stage), and those with a complexity of 7-10 are assigned to the third stage (complex collaborative stage), ensuring that agents gradually accumulate collaborative experience.

[0054] In step S3, based on the task requirements of the current training phase, a suitable number and capability of agents are selected from the agent pool to form a temporary collaborative unit. For example, if the task is simple in the first phase, a collaborative unit of 2 agents is formed; if the task is complex in the third phase, a collaborative unit of 6 agents is formed.

[0055] In step S31, while the collaborative unit is executing the task of the current stage, the number of task handover negotiation rounds is recorded in real time (e.g., agent 1 hands over the "order review" task to agent 2, and the two parties reach an agreement after 3 rounds of negotiation, and the negotiation round is recorded as 3), the frequency of decision conflict events (e.g., agent 3 and agent 4 apply for "inventory query" resources at the same time, resulting in 1 conflict, and the conflict frequency is recorded as +1), and the average idle time of agents (e.g., agent 5 waits 2 minutes after completing a task to receive a new task, agent 6 waits 3 minutes, and the average idle time is 2.5 minutes), to ensure that the data fully reflects the collaborative status.

[0056] In step S32, based on the collected task completion data, the overall task completion efficiency of the collaborative unit is calculated (e.g., 8 atomic tasks are completed per unit time, the average task queue clearing time is 15 minutes, and the overall efficiency value is calculated according to the preset weight). At the same time, the initial collaboration intensity weight is dynamically updated in combination with the negotiation rounds and decision conflict frequency. Specifically, if a task pair has more negotiation rounds and higher conflict frequency, it indicates that the task has a higher demand for collaboration, and its collaboration intensity weight is appropriately increased; otherwise, the weight is reduced to make the weight more in line with the actual collaboration situation.

[0057] In step S33, the collected interaction behavior data and the updated collaboration strength weights are input into the reinforcement learning model. The overall task completion efficiency is used as the reward signal (the higher the efficiency, the higher the reward). The model optimizes the collaboration strategy through iterative learning. The optimized strategy will be used as the initial strategy for the next stage of training, realizing the cycle of "learning-optimization-relearning" and gradually improving the collaboration ability.

[0058] In step S4, when all multi-stage collaborative training courses are completed and the collaborative strategy optimization loop meets the convergence condition (the overall task completion efficiency fluctuates below the threshold for multiple consecutive rounds), it is determined that the collaborative strategy has reached its optimal stable state. Then, the final optimized collaborative strategy is written into the program module of the multi-agent collaborative system, enabling agents to directly perform task handover and conflict arbitration based on this strategy during actual system operation, ensuring the system completes business tasks stably and efficiently.

[0059] The above solution transforms complex business scenarios into structured atomic tasks and dependencies through task decomposition and dependency graph construction, ensuring that the collaborative system aligns with actual business logic and avoiding collaborative chaos caused by ambiguous task modeling. Multi-stage training courses are generated based on collaborative complexity, allowing agents to learn collaboration "from easy to difficult," reducing initial training difficulty, minimizing ineffective training iterations, and significantly improving training efficiency. By collecting interaction behavior data in real time and dynamically updating collaboration strength weights, combined with iterative optimization using reinforcement learning models, the collaboration strategy can be adjusted according to the actual collaboration state, adapting to changes in different business scenarios. For example, during peak order periods, the strategy can automatically optimize the task handover process, reducing negotiation costs. Strategy optimization focuses on overall task completion efficiency while also considering collaboration details such as negotiation rounds and conflict frequency, avoiding strategy bias caused by optimizing a single metric. The resulting collaborative system can efficiently complete tasks and cope with conflicts and delays during the collaboration process, significantly improving robustness.

[0060] In some embodiments, such as Figure 3 As shown, step S1 includes:

[0061] S11: Obtain historical work order data and business process specification documents for the target business scenario. The business process specification documents record process rules for defining task execution logic and relationships.

[0062] S12: Based on the process rules, the historical work order data is deconstructed into multiple atomic tasks that can be assigned to a single intelligent agent for independent execution, to obtain the task set. Each atomic task is associated with at least the following attributes: task type, estimated execution time, and set of required resource types.

[0063] S13: Parse the process rules in the business process specification document to obtain the process sequence and data flow rules, construct the temporal dependency relationship between multiple atomic tasks based on the process sequence, and construct the data dependency relationship between multiple atomic tasks based on the data flow rules;

[0064] S14: Based on the temporal and data dependencies, the task dependency graph is obtained.

[0065] In step S11, the business process specification document is a formal document that standardizes the execution logic and relationships of business tasks. It includes process rules (such as "inventory deduction can only be performed after order approval" and "logistics tracking number must be generated after logistics allocation is completed"), task execution standards (such as "order approval must be completed within 10 minutes"), etc.

[0066] When acquiring historical work order data, extract historical work order data from the past 1-3 months through the database system of the target business scenario (ensuring sufficient data volume to reflect routine business conditions). The data must include fields such as task ID, task name, executing agent ID, execution start time, execution end time, resources required for the task, and associated data of the task.

[0067] In step S12, the process rules in the business process specification document are first parsed sentence by sentence to identify the characteristics of the smallest task unit. For example, parsing "order review requires checking the integrity of order information and generating review result data after the review is passed" determines that "order review" is the smallest task unit that can be executed by a single intelligent agent.

[0068] The smallest task unit obtained from the parsing is then matched with historical work order data to verify whether the task unit exists independently in the work order (e.g., there is a separate "order review" task record in the work order data, and it is executed by a single agent). If it exists, it is determined to be an atomic task; if it does not exist, the task unit is further broken down until an independently executable atomic task is obtained.

[0069] Then, attributes are associated with each atomic task. The task type is determined according to the task function (e.g., "order review" is a review task). The estimated execution time is determined based on the average execution time of the task in the historical work order data (e.g., the average execution time of the past 100 "order review" tasks is 8 minutes, so the estimated time is set to 8 minutes). The set of required resource types is extracted from the "resources required for the task" field of the work order data (e.g., "order review" requires "order database access permissions" and "user information query resources"), forming a complete task set.

[0070] In step S13, based on the process sequence rules in the business process specification document, the execution order of atomic tasks is identified. For example, in the rule "Inventory deduction is performed after order approval," "Order approval" is a prerequisite task, and "Inventory deduction" is a subsequent task, establishing a temporal dependency between them. By traversing all process sequence rules, a temporal dependency network of all atomic tasks is constructed. Based on the data flow rules in the document, the data flow between atomic tasks is identified. For example, in the rule "Inventory deduction requires the result data of order approval, and the inventory data is passed to logistics allocation after inventory deduction is completed," there are data dependencies between "Order approval" and "Inventory deduction," and between "Inventory deduction" and "Logistics allocation." By extracting all data transfer relationships, a data dependency network of atomic tasks is constructed.

[0071] In step S14, a task dependency graph is constructed using atomic tasks as nodes and temporal dependencies and data dependencies as edges. For example, between node A (order approval) and node B (inventory deduction), edges labeled "temporal" represent temporal dependencies, and edges labeled "data" represent data dependencies. If a pair of tasks has both types of dependencies, two edges with corresponding labels are drawn between the nodes, ultimately forming a complete task dependency graph that intuitively reflects the association logic between tasks.

[0072] The above solution decomposes tasks based on both historical work order data and business process specification documents, avoiding unreasonable task granularity caused by relying solely on subjective experience (such as tasks that are too large to be executed by a single agent, or tasks that are too small, increasing collaboration costs), and ensuring that atomic tasks meet actual business execution needs. It distinguishes between temporal dependencies and data dependencies, so that the task dependency graph not only reflects the task execution order but also the data flow requirements, providing a more detailed basis for subsequent collaboration strategy optimization (such as prioritizing data transmission efficiency for data-dependent task pairs).

[0073] In some embodiments, such as Figure 4 As shown, in step S2, for each pair of atomic tasks with a cooperative relationship in the task dependency graph, an initial cooperation strength weight is set, including:

[0074] S21: Traverse all dependency edges in the task dependency graph and identify two atomic tasks directly connected by the dependency edge as a pair of atomic tasks with a cooperative relationship;

[0075] S22: For each of the atomic task pairs, calculate the initial cooperation strength weight based on the attributes of the atomic tasks, using the following formula:

[0076] W initial =α×F type +β×F complexity +γ×F coupling ;

[0077] Among them, W initial This represents the initial collaboration strength weight;

[0078] F type The dependency type factor is determined based on the dependency type between the atomic task pairs, wherein data dependencies are assigned a first weight value and temporal dependencies are assigned a second weight value.

[0079] F complexity This represents the task complexity factor, the value of which is determined based on the average of the estimated execution times of the two atomic tasks in the atomic task pair.

[0080] F coupling This represents the task coupling factor, the value of which is determined based on the size of the intersection of the sets of resource types required by the two atomic tasks in the atomic task pair.

[0081] α, β, γ are preset weighting coefficients.

[0082] The above scheme quantifies collaboration requirements through three dimensions: dependency type, task complexity, and task coupling. This ensures that the initial collaboration strength weights objectively reflect the actual collaboration needs of atomic task pairs, reducing ineffective exploration in the early stages of the model. The weighting coefficients can be flexibly adjusted according to different business scenarios. For example, in industrial production scenarios, task complexity has a greater impact on collaboration, so the value of β can be increased; in financial data processing scenarios, data dependency has a greater impact, so the value of α can be increased, allowing the initial weights to adapt to diverse application needs.

[0083] In some embodiments, such as Figure 5 As shown, step S33 specifically includes:

[0084] S331: Normalize the interaction behavior data and the updated collaboration strength weights, concatenate the normalized collaboration strength weight vector with the normalized interaction behavior data vector to obtain a comprehensive feature vector, and use the comprehensive feature vector as the state space S of the reinforcement learning model.

[0085] S332: Construct a composite reward function R that combines positive efficiency rewards with negative behavioral penalties to generate the reward signal. The calculation formula for the composite reward function R is as follows:

[0086] R=w e ×E-(w n ×N n +w c ×N c +w i ×T i );

[0087] Where E is the normalized overall task completion efficiency, and N n N represents the round of normalized task handover negotiation. c T represents the normalized frequency of decision-making conflicts caused by resource competition. i w represents the average idle time of the agent in the task waiting state after normalization. e ,w n ,w c ,w i These represent the preset weighting coefficients, used to balance the contribution of each indicator.

[0088] S333: Based on the state space S and the composite reward function R, the reinforcement learning model is iteratively optimized. When the change in the overall task completion efficiency within multiple consecutive training iterations is lower than a preset threshold, it is determined that the cooperative strategy has converged.

[0089] In this embodiment, the reinforcement learning model obtains the comprehensive feature vector of the current cooperative state from the state space S. Based on the current state, it outputs the action corresponding to the cooperative strategy (such as approving the request to "transfer task A to task B" or deciding that "agent 1 has priority to use resource C"). Based on the task completion status and interaction data after the action is executed, a reward signal R is calculated through a composite reward function. The reward signal R adjusts internal parameters (such as adjusting the judgment threshold for task transfer decisions) to make it easier to output actions that yield high rewards in the same or similar states, thus achieving strategy optimization. The overall task completion efficiency of each training iteration is recorded, and the change in efficiency (the absolute value of the efficiency of the next iteration minus the efficiency of the previous iteration) is calculated over multiple consecutive iterations (such as 10 consecutive iterations). If all changes are lower than a preset threshold (such as 0.05), it indicates that the model strategy has stabilized, the overall efficiency no longer significantly improves, the cooperative strategy is determined to have converged, and iterative optimization is stopped.

[0090] The above scheme uses normalization to unify the collaboration intensity weights and interaction behavior data across different dimensions to the same value range. This prevents the model from overemphasizing any single data point due to differences in data volume (e.g., negotiation rounds in single digits, idle time in minutes), ensuring the model learns all collaboration features fairly. The composite reward function simultaneously considers both positive efficiency and negative behavior, avoiding policy bias caused by a single efficiency indicator (e.g., ignoring conflicts in pursuit of efficiency, leading to subsequent task blockage). This ensures that the collaboration strategy learned by the model can efficiently complete tasks while reducing internal friction (negotiation, conflict, idle time) during the collaboration process, thus improving the strategy's practicality.

[0091] In some embodiments, such as Figure 6 As shown, in step S333, the iterative optimization of the reinforcement learning model based on the state space S and the composite reward function R includes the following steps:

[0092] S3331: Set up a centralized evaluation network, the input of which is the state space S, to evaluate the long-term value of the global cooperation strategy; configure a distributed policy network for each agent in the cooperation unit, each distributed policy network generating the individual action decision of the agent based on the local observation information of its corresponding agent and combined with the global cooperation relationship represented by the state space S.

[0093] S3332: Based on the state space S and the composite reward function R, the centralized evaluation network and all distributed policy networks are jointly iteratively trained, specifically including: using the reward signal generated by the composite reward function R, updating the parameters of the centralized evaluation network through temporal difference learning; based on the long-term value estimate output by the centralized evaluation network, generating the policy update gradient through the policy gradient method, and simultaneously optimizing the parameters of all distributed policy networks accordingly.

[0094] S3333: Repeat step S3332 until the reinforcement learning model converges, and deploy the final distributed policy network to each agent in the cooperative unit.

[0095] In this embodiment, the centralized evaluation network is the core module in the reinforcement learning model responsible for evaluating the long-term value of the collaborative strategy from a global perspective. It does not directly generate the agent's action decisions, but only outputs the cumulative reward that the current strategy may obtain in the long run through the input state space S (such as the comprehensive feature vector of the financial product data processing scenario, including the collaborative weights of the data cleaning and feature extraction task pairs, agent interaction data, etc.), providing a global value reference for strategy optimization, such as evaluating the long-term benefits of the strategy of "prioritizing the processing of high-net-worth customer product recommendation data".

[0096] A distributed policy network refers to a network module configured separately for each agent in a collaborative unit. Each network only obtains local observation information of the corresponding agent (such as the type of financial data being processed by the agent, the computing resources already occupied, and the task load status of surrounding agents), and at the same time combines the global collaborative relationship provided by the centralized evaluation network (such as the collaborative strength of other financial data processing task pairs) to generate the individual action decision of the agent.

[0097] Local observation information refers to information that a single agent can directly perceive and that is only relevant to itself, without including complete information about other agents. In the scenario of financial product data processing, for example, the local observation information of agent A includes "currently performing a high-net-worth client asset data cleaning task", "30% of computing resources have been used", "15 minutes of task processing time remaining", and "neighboring agent B is busy processing client risk assessment data", but does not include the task execution progress of agent C.

[0098] Temporal difference learning refers to an algorithm used to update the parameters of a centralized evaluation network. Its core principle is to adjust the network parameters using the difference between the "current reward + future value estimate" and the "current value estimate" (temporal difference error), making the network parameters more accurate in estimating long-term value. For example, in financial product recommendation data processing, if the current value estimate is 0.6, the current reward (the return for completing a batch of recommendation data processing) is 0.4, and the future value estimate is 0.5, then the temporal difference error = 0.4 + 0.5 - 0.6 = 0.3, and the network parameters are updated based on this error.

[0099] The policy gradient method refers to an algorithm used to optimize the parameters of a distributed policy network. It calculates the gradient of the reward signal with respect to the policy parameters (i.e., the degree to which parameter changes affect the reward), and adjusts the parameters along the gradient's ascending direction to make the policy more likely to output actions that yield high rewards. For example, in financial data conflict arbitration, a positive gradient for a certain parameter indicates that increasing its value (such as increasing the weight of "customer asset size" in resource allocation) will increase the reward; therefore, the algorithm updates the parameter in the direction of increasing that value.

[0100] In step S331, when constructing the centralized evaluation network, the input layer receives the comprehensive feature vector of the state space S (e.g., in a financial product data processing scenario, it contains the collaborative weights of 6 task pairs and 3 interaction data items, forming a 9-dimensional vector). The hidden layer consists of 2-3 fully connected layers (using ReLU as the activation function), and the output layer outputs a single numerical value (the long-term value estimate of the current strategy). For example, if the input vector is [0.52, 0.48, 0.61, 0.35, 0.42, 0.55, 0.28, 0.33, 0.21], the hidden layers are set to 64 neurons and 32 neurons, the output layer is 1 neuron, and the output long-term value estimate is 0.68, representing the long-term value of the current strategy of "prioritizing high-net-worth client data".

[0101] Then, an independent network is configured for each agent in the collaborative unit. The input layer receives the agent's local observation information vector (e.g., dimension 7, including task type, resource utilization, etc.) and the global collaborative relationship vector output by the centralized evaluation network (e.g., dimension 4, including the collaborative strength of other task pairs). After concatenation, the vector is input into two fully connected hidden layers, and the output layer is the action probability distribution. For example, when an agent processes financial product recommendation data, the probability of outputting "receive new recommendation data processing request" is 0.7, and the probability of "reject request" is 0.3. Each agent has the same network structure, but the parameters are independent.

[0102] In S3332, in each training iteration, the centralized evaluation network outputs a long-term value estimate V(S) (e.g., 0.68) based on the current state S. After the agent performs an action (e.g., receiving a request to process recommended data), it obtains a reward R (e.g., 0.4 reward for completing the processing) and the next state S', and obtains the long-term value estimate V(S') of the next state (e.g., 0.72). The temporal difference error δ = R + C × V(S') - V(S) (where C is a discount factor, set to 0.9, representing the importance of future value) is calculated, i.e., δ = 0.4 + 0.9 × 0.72 - 0.68 = 0.4 + 0.648 - 0.68 = 0.368. Based on δ, the parameters of the centralized evaluation network are updated using gradient descent to make V(S) closer to R + C × V(S'), thus improving the accuracy of value estimation.

[0103] Then, the centralized evaluation network outputs a long-term value estimate of the current state, V(S) (0.68), as a global value reference. Each distributed policy network generates an action probability distribution and executes an action based on the agent's local observation information (such as "current resource usage is 30%)" and V(S). The reward R (0.4) is calculated according to the composite reward function, and combined with V(S), the advantage function A=RV(S)=0.4-0.68=-0.28 (this is an example; the actual advantage function for efficient actions is positive). The gradient of the advantage function with respect to the parameters of the distributed policy network is calculated using the policy gradient method, and the parameters are updated along the gradient ascending direction to make the network more likely to generate actions with a positive advantage function, such as "prioritizing the processing of high-net-worth customer recommendation data".

[0104] In S3333, once the reinforcement learning model converges (e.g., the overall efficiency of financial product data processing fluctuates below 0.05 for 10 consecutive rounds), the final parameters of each distributed policy network are deployed to the corresponding agents. For example, in a financial product recommendation scenario, the agent responsible for processing customer risk data will have its network parameters deployed to guide it to "prioritize receiving risk data from high-net-worth clients and prioritize allocating computing resources to this type of task when resource conflicts occur," thus achieving a collaborative model of "centralized evaluation and distributed execution."

[0105] In the above scheme, the centralized evaluation network optimizes the policy from a global perspective, avoiding local optimal decisions by the agent due to incomplete information (such as focusing only on its own task progress while ignoring the overall financial data processing efficiency). The distributed policy network allows the agent to make autonomous decisions based on local information, improving the response speed to changes in the local environment (such as quickly adjusting the policy for receiving financial data tasks if an agent suddenly becomes idle), balancing global and local needs. Under this architecture, temporal difference learning and policy gradient methods optimize the two types of networks respectively. Value estimation and action decision-making promote each other. More accurate value estimation provides a reliable direction for policy optimization, and better action decisions provide high-quality data for value estimation, accelerating model convergence. Compared with a single network architecture, this approach can effectively improve convergence speed and shorten the system construction cycle.

[0106] In some embodiments, in step S33, counterfactual reasoning is used as an auxiliary optimization method during iterative optimization. This counterfactual reasoning mechanism is executed in each training iteration and specifically includes: using the collaborative policy output by the reinforcement learning model in the current iteration as a baseline policy, and based on the baseline policy, the interaction behavior data, and the updated collaborative strength weights, such as... Figure 7 As shown, perform the following operations:

[0107] S701: Identify the atomic tasks that have the highest number of negotiation rounds or the highest frequency of decision conflict events in the interaction behavior data from the currently executing task set, and use them as key atomic tasks to be modified.

[0108] S702: Virtually modify the collaboration elements related to the key atomic task to generate at least one counterfactual collaboration scheme; the modification of the collaboration elements specifically includes: replacing the agent executing the key atomic task, or changing the order of negotiation objects in the task handover process of the key atomic task.

[0109] S703: Based on the updated collaboration strength weight, deduce the virtual task completion efficiency that can be achieved by executing the counterfactual collaboration scheme;

[0110] S704: Compare the highest virtual task completion efficiency obtained from the deduction with the current overall task completion efficiency calculated based on the baseline policy and the interaction behavior data. If the highest virtual task completion efficiency is higher than the current overall task completion efficiency, generate a policy update gradient according to its corresponding counterfactual cooperation scheme, and apply the policy update gradient to the reinforcement learning model to optimize the cooperation policy.

[0111] In this embodiment, the benchmark strategy refers to the collaborative strategy output by the reinforcement learning model in the current training iteration, used for comparison, and serves as a reference standard for counterfactual reasoning. In the scenario of financial product data processing, for example, in the current iteration, the strategy output by the model, "prioritizing the processing of high-net-worth client asset data and arbitrating resource conflicts based on task submission time," is the benchmark strategy.

[0112] A critical atomic task is the one with the most prominent collaboration issues within the current task set, identified through interaction behavior data. It is the atomic task with the highest number of negotiation rounds (indicating difficulty in task handover and high collaboration costs) or the highest frequency of decision-making conflicts (indicating intense competition for task resources). For example, in financial product recommendation data processing, the task of "high-net-worth client risk assessment data processing" has 9 negotiation rounds (significantly higher than the 3-5 rounds of other tasks), making it a critical atomic task.

[0113] Counterfactual collaboration schemes refer to hypothetical collaboration schemes generated by virtually modifying the collaboration elements of key atomic tasks. These schemes are not actually executed but are only used to simulate effects. In financial product data processing scenarios, for example, replacing the executing agent of the "high-net-worth client risk assessment data processing" task from agent A to agent B, or changing the order of task handover negotiations (from the original negotiation order of agent C→D→E to D→C→E), creates two counterfactual collaboration schemes.

[0114] Virtual task completion efficiency refers to the task completion efficiency of a counterfactual collaboration scheme calculated through simulation based on the updated collaboration strength weights, rather than the actual execution efficiency. For example, in financial data processing, when simulating the scheme of "Agent B executing high-net-worth client risk assessment data processing", the virtual efficiency value of 0.88 is calculated by combining the collaboration strength weight (0.65) of the task pair and simulating the data processing flow.

[0115] In step S701, in each training iteration, the collaborative strategy output by the current reinforcement learning model is used as the baseline strategy (e.g., "prioritize processing high-net-worth client data"). Simultaneously, the collected interaction behavior data is analyzed, and the negotiation rounds and decision conflict frequencies of all atomic tasks are statistically analyzed. The atomic task with the highest value is identified as the critical atomic task (if multiple highest values ​​exist, they are all considered critical tasks). For example, in a financial product data processing scenario, the conflict frequency of the "client asset data cleaning" and "high-net-worth client risk assessment" tasks is both 8, and both are identified as critical tasks.

[0116] In step S702, for the key atomic task, the collaborative element generation scheme is virtually modified, specifically including:

[0117] Agent replacement: From the agent pool of the collaborating unit, select another agent with similar capabilities to the original executing agent (e.g., a difference of no more than 10% in historical success rate for financial data processing) to replace the original executing agent. For example, if agent A, which originally performed the "high-net-worth client risk assessment" task, had a success rate of 92%, select agent B, with a success rate of 88%, as the replacement, and generate a solution for "agent B to perform high-net-worth client risk assessment data processing".

[0118] Adjusting the order of negotiation targets: This changes the order of negotiation targets during the handover of critical atomic tasks. The original order was based on the idle status of the agents, while the new order can be based on the negotiation success rate of the agents' financial data processing. For example, the original order was agents C (idle) → D (busy) → E (idle), and the new order is E (negotiation success rate 90%) → C (negotiation success rate 85%) → D (negotiation success rate 78%), generating a solution for "adjusting the negotiation order for the handover of high-net-worth client risk assessment data".

[0119] Number of schemes controlled: 2-3 counterfactual collaboration schemes are generated for each key atomic task to avoid excessive inference costs due to too many schemes.

[0120] In step S703, based on the updated collaboration strength weights, a simulation tool is used to simulate the execution process of the counterfactual collaboration scheme, statistically analyze the financial data processing speed and task queue clearing time, and calculate the virtual task completion efficiency. For example, in the simulation of the scheme "Agent B executes high-net-worth client risk assessment data processing", a virtual efficiency of 0.88 is obtained; in the simulation of the scheme "adjusting the negotiation order", a virtual efficiency of 0.91 is obtained. The highest virtual efficiency (0.91) is compared with the current overall task completion efficiency (0.76). If the highest virtual efficiency is higher, it indicates that the corresponding counterfactual collaboration scheme is better. Based on the direction of modification of the collaboration elements of the scheme (such as "selecting agents with high negotiation success rates as priority negotiation targets"), a strategy update gradient is generated (such as increasing the weight of "negotiation success rate" in the financial data task handover decision).

[0121] In step S704, the generated policy update gradient is applied to the reinforcement learning model to adjust the model parameters, causing the model's subsequent output cooperative policies to lean towards the optimization direction of the counterfactual solution. For example, the weight of "negotiation success rate" is increased based on the gradient, making it easier for the model to prioritize agents with high negotiation success rates when transferring financial data tasks.

[0122] The aforementioned solution utilizes a counterfactual reasoning mechanism, eliminating the need to rely on real-world execution data. It explores optimal collaboration models through virtual scenario deduction, preventing models from becoming trapped in local optima by solely relying on historical financial data (e.g., if agent A is consistently used to process high-net-worth client data in real-world scenarios, the model may fail to discover that agent B is a better choice), thus expanding the strategy search space. Furthermore, it directly generates optimized solutions for critical financial data tasks with high negotiation rounds and frequent conflicts, demonstrating strong targeting and enabling rapid identification and resolution of core collaboration issues.

[0123] In some embodiments, such as Figure 8 As shown, in step S33, the iterative optimization of the cooperative strategy is achieved through an agent behavior feedback mechanism, specifically including:

[0124] S801: Calculate the corresponding individual collaborative behavior quantification value for each agent in the collaborative unit based on the interactive behavior data; the individual collaborative behavior quantification value includes at least task contribution value, negotiation participation degree, and decision influence.

[0125] S802: Input the quantified value of the individual collaborative behavior of each agent into the reinforcement learning model. Based on the overall task completion efficiency and the quantified value of the individual collaborative behavior, the reinforcement learning model generates an individual policy optimization signal for each agent. The individual policy optimization signal is used to guide the agent to adjust its behavior pattern in subsequent task handover and conflict arbitration.

[0126] S803: During the process of iteratively optimizing the cooperation strategy by the reinforcement learning model, the individual strategy optimization signal is fused with the global optimization signal based on the interaction behavior data to jointly drive the update of the cooperation strategy, so that the optimized cooperation strategy can simultaneously reflect the optimization requirements of global task efficiency and individual cooperation behavior.

[0127] In step S801, the individual collaborative behavior quantification value is used to measure the quantitative indicators of a single intelligent agent's performance in the collaborative process, including task contribution value, negotiation participation, and decision influence. It is extracted and calculated from the interaction behavior data to reflect the intelligent agent's collaborative ability and initiative in financial product data processing.

[0128] The task contribution value is an indicator calculated based on the number of financial atomic tasks successfully completed by the agent and their collaborative complexity. The more tasks completed and the higher the collaborative complexity, the greater the contribution value. For example, if agent A successfully completes three financial data processing tasks with collaborative complexities of 3, 4, and 5 respectively (such as customer asset cleaning, risk assessment, and product recommendation data integration), the contribution value is 3 × (3 + 4 + 5) / 3 = 12.

[0129] Negotiation participation rate refers to the total number of times a statistical agent participates in financial data task handover negotiations as either the initiator or the recipient. The more times an agent participates, the higher its participation rate. For example, if agent B participates in 6 financial data task negotiations as the initiator and 4 as the recipient, its negotiation participation rate is 6 + 4 = 10.

[0130] Decision influence refers to the metric that measures the degree to which an agent's suggestions are adopted in the arbitration of financial data task conflicts. It is calculated as the ratio of "number of adopted suggestions / total number of suggestions". The higher the ratio, the greater the influence. For example, if agent C makes 12 suggestions in the arbitration of financial data resource conflicts and 9 of them are adopted, the decision influence = 9 / 12 = 0.75.

[0131] Individual policy optimization signals refer to signals generated by reinforcement learning models for individual agents to adjust their individual behavioral patterns. These signals are determined based on the overall task completion efficiency and the quantitative value of individual collaborative behavior. For example, if agent D has a low task contribution value, the model generates an optimization signal to "increase the number of financial data tasks received and prioritize low-complexity customer basic data processing tasks to accumulate experience."

[0132] Global optimization signals refer to signals generated based on interactive behavior data to optimize overall collaboration strategies. They reflect the global collaboration needs of financial data processing, such as signals to "reduce the frequency of decision-making conflicts in financial data tasks" and "reduce the average idle time of agents".

[0133] In step S801, the calculation of the quantitative value of individual collaborative behavior is divided into the calculation of task contribution value, negotiation participation degree, and decision influence. The specific calculation methods are as follows:

[0134] Task contribution value calculation: Extract the list of financial atomic tasks successfully completed by each agent from the interaction behavior data, obtain the collaborative complexity of each task, and calculate the formula as follows: Task contribution value = Number of tasks completed × Average collaborative complexity. For example, if agent E completes 5 financial data tasks with collaborative complexities of 2, 3, 3, 4, and 4 respectively, the average complexity is 3.2, and the task contribution value is 5 × 3.2 = 16.

[0135] Negotiation participation calculation: Iterate through all financial data task handover records, count the number of negotiations between each agent as the initiator (actively handing over the task) and the receiver (passively receiving the task), and sum them to obtain the negotiation participation rate. For example, if agent F initiates negotiations 7 times and receives negotiations 5 times, the participation rate = 7 + 5 = 12.

[0136] Decision Influence Calculation: Extract the number of decision suggestions made by each agent and the number of suggestions adopted from the financial data conflict arbitration records, and calculate the ratio (if the total number of suggestions is 0, the influence is set to 0). For example, if agent G makes 8 suggestions and adopts 5, the decision influence = 5 / 8 = 0.625.

[0137] In step S802, the quantified value of each agent's individual collaborative behavior is input into the reinforcement learning model, and the model is analyzed in conjunction with the overall task completion efficiency of financial data processing.

[0138] If the agent's task contribution value is low (e.g., below 50% of the average), it indicates that it is handling fewer financial data tasks, and an optimization signal is generated to "increase the frequency of receiving financial data tasks and prioritize low-complexity customer basic information processing tasks to accumulate experience".

[0139] If the participation rate in the negotiation is low (e.g., below the average of 30%), it indicates insufficient enthusiasm for collaboration, and an optimization signal is generated to "actively initiate negotiations on financial data tasks with neighboring intelligent agents and participate in collaborative discussions on high-net-worth client data processing".

[0140] If the decision-making influence is low (e.g., below 0.3), it indicates that the suggestion is not reasonable, and an optimization signal is generated to "refer to the suggestion logic of high-influence agents and improve the matching degree of financial data resources in the suggestions (e.g., give priority to customer asset size)".

[0141] In step S803, during the iterative optimization of the reinforcement learning model, the generated individual policy optimization signals and global optimization signals based on interaction behavior data (such as "reducing the frequency of financial data decision conflicts" and "reducing the average idle time") are fused according to preset weights (e.g., individual signal weight 0.4, global signal weight 0.6) to form the final policy update direction. For example, by fusing the individual signal of "agent H increases the reception of financial data tasks" with the global signal of "reducing idle time", the financial data task allocation strategy is adjusted so that agent H prioritizes receiving the pending customer data tasks of idle agents, jointly driving the collaborative strategy update.

[0142] The above scheme quantifies individual collaborative behavior, making the performance of each agent in financial data processing measurable. The model can generate optimization signals for agents with poor performance, preventing some agents from relying on others to complete core financial data tasks without contributing to the collaboration, and ensuring that all agents actively participate in the collaboration. By integrating global and individual optimization signals, it avoids focusing solely on global financial data processing efficiency, which can lead to uneven development of individual agents (e.g., some agents cannot improve their collaborative ability due to long-term assignment of simple tasks), or focusing solely on individual performance, which can lead to a decline in global efficiency. The collaborative ability of each agent is gradually improved through individual optimization signals, enabling other agents to quickly fill in when some agents fail (due to their strong collaborative abilities), reducing the impact of a single agent failure on the overall financial data processing task, improving the overall risk resistance of the system, and ensuring the stable operation of financial business.

[0143] In some embodiments, the method further includes: dynamically assigning an agent with decision-making and coordination authority at each training stage of the multi-stage collaborative training course based on an optimized collaboration strategy, such as... Figure 9 As shown, it specifically includes:

[0144] S901: In each training phase, based on the interaction behavior data and the updated collaboration strength weights, a coordination ability evaluation value is calculated for each agent in the collaboration unit. The coordination ability evaluation value is calculated by weighting the current agent's historical success rate of task processing, current resource availability, and the collaboration strength of its associated tasks.

[0145] S902: Dynamically designate the agent with the highest coordination ability evaluation value in the current training phase as the decision coordinator. The decision coordinator is given the authority to execute the task transfer adjudication and decision conflict arbitration guided by the cooperation strategy in step S33.

[0146] S903: When the collaborative unit enters the next training stage according to the order of the multi-stage collaborative training course, steps S901 and S902 are automatically triggered to update the decision coordinator.

[0147] In this embodiment, the coordination ability evaluation value is a quantitative indicator used to measure the ability of an agent to assume the role of decision coordination in the current training phase. It is calculated based on the agent's historical performance, current resource status and financial task collaboration relationship. The higher the evaluation value, the more suitable the agent is as a decision coordinator.

[0148] Historical success rate refers to the ratio of the number of times an agent successfully completed a financial atomic task during the past training phase to the total number of task processing attempts, reflecting the reliability of the agent's task execution. For example, if agent I has completed 25 financial data processing tasks in the past, with 23 successful, the historical success rate is 23 / 25 = 0.92.

[0149] The current resource availability status refers to the ratio of the number of unused resources to the total resource capacity of an agent, reflecting the resource idleness of the agent. In the scenario of financial product data processing, it mainly refers to computing resources, data storage resources, etc. For example, if agent J has a total computing resource capacity of 10 (which can process 10 batches of financial data simultaneously), and currently occupies 4, the resource availability status = (10-4) / 10 = 0.6.

[0150] The collaboration strength of related tasks refers to the average collaboration strength weight between the financial atomic task currently being executed or to be executed by the agent and other tasks, reflecting the degree of close collaboration involved by the agent. For example, if agent K is associated with "high-net-worth client asset data cleaning" (collaboration weight 0.6) and "product recommendation data integration" (collaboration weight 0.55), the collaboration strength of the related tasks = (0.6 + 0.55) / 2 = 0.575.

[0151] The decision coordinator refers to the intelligent agent that has been given decision coordination authority in the current training phase. It is responsible for adjudicating the transfer of financial data tasks guided by the collaboration strategy (such as judging whether "the transfer of customer risk assessment data to asset data processing tasks" meets the strategy requirements) and arbitrating decision conflicts (such as adjudicating the priority right to use resources when multiple intelligent agents compete for high-net-worth customer data processing resources). It is the core of collaboration in the current phase.

[0152] In step S901, the calculation process of the coordination ability evaluation value is as follows: Before the start of each training phase, the coordination ability evaluation value of each agent is calculated based on the following three dimensions. The calculation formula is: Coordination ability evaluation value = ω1 × historical success rate + ω2 × current resource availability status + ω3 × cooperation intensity of associated tasks, where ω1, ω2, and ω3 are preset weights (e.g., ω1 = 0.4, ω2 = 0.3, ω3 = 0.3), with values ​​ranging from 0 to 1, and the sum is 1.

[0153] The calculation process for the historical success rate of task processing is as follows: Extract the financial data task completion data of the agent in all past training stages from the training log, calculate the ratio of the number of successful tasks to the total number of tasks, and if it is a newly added agent, the historical success rate is set to the default value of 0.75.

[0154] The calculation process for the current resource availability status is as follows: Real-time query of the agent's resource usage (such as CPU utilization, financial data storage utilization, and task queue length) to calculate the available resource percentage. For example, if agent L currently has a CPU utilization of 25%, a financial data storage utilization of 30%, and 3 tasks remaining in the task queue (out of a total capacity of 10), then the resource availability status = (1-0.25)×0.3 + (1-0.3)×0.3 + (10-3) / 10×0.4 = 0.225 + 0.21 + 0.28 = 0.715 (CPU, storage, and task queue have weights of 0.3, 0.3, and 0.4 respectively).

[0155] The calculation process for the collaboration strength of associated tasks is as follows: Extract all financial atomic tasks currently associated with the agent, obtain the collaboration strength weight of each task with other tasks, and calculate the average value. For example, if agent M is associated with "Customer Risk Assessment Data Processing" (collaboration weight 0.58) and "High-Net-Worth Customer Product Recommendation Data Processing" (collaboration weight 0.62), the average value is (0.58 + 0.62) / 2 = 0.6.

[0156] For example, if agent N has a historical success rate of 0.9, a current resource availability of 0.715, and a collaborative strength of associated tasks of 0.6, then the coordination ability evaluation value = 0.4×0.9+0.3×0.715+0.3×0.6=0.36+0.2145+0.18=0.7545.

[0157] In step S902, the coordination ability evaluation values ​​of all agents in the current training phase are sorted, and the agent with the highest evaluation value is selected as the decision coordinator. If multiple agents have the same highest evaluation value (e.g., agents O and P both have an evaluation value of 0.78), one of them is randomly selected as the coordinator. After designation, the agent is assigned decision coordination permissions, enabling it to access the adjudication rules of the collaboration strategy (e.g., the success rate threshold for financial data task transfer, the resource allocation priority for conflict arbitration, such as "high-net-worth client data tasks take precedence over ordinary client tasks"), and to perform adjudication and arbitration operations during the collaboration process.

[0158] In step S903, when the collaborative unit completes all financial data tasks in the current training phase and enters the next training phase, processes S901 and S902 are automatically triggered. This involves recalculating the coordination ability evaluation values ​​of all agents in the new phase (as the historical success rate, resource status, and associated tasks of agents may change, such as an agent increasing its success rate after completing a large number of challenging financial tasks). Based on the new evaluation values, a new decision-making coordinator is assigned. For example, if the coordinator in the first phase is agent O (evaluation value 0.78), and in the second phase agent P's historical success rate increases to 0.93, and its evaluation value becomes 0.81, then the coordinator in the second phase is updated to agent P.

[0159] The aforementioned solution assigns decision-making coordination authority to the most capable agent (with a high historical success rate, available resources, and close collaborative relationships). This allows the agent to quickly and accurately execute task transfer adjudication and conflict arbitration based on its reliable financial data processing experience, sufficient resources (time to handle decision-making matters), and in-depth understanding of collaborative relationships. Compared to fixed or randomly assigned coordinators, this approach effectively reduces decision-making time and improves accuracy. For example, in financial data processing scenarios, it ensures the timeliness and accuracy of high-net-worth client data processing.

[0160] Each training phase reassigns a coordinator to adapt to changes in agent capabilities (e.g., an agent performs poorly in the early stages but improves its ability to process financial data in the later stages) and changes in task requirements (e.g., in the later stages, there are more high-net-worth client data tasks, requiring an agent with high collaboration intensity to act as the coordinator), thus avoiding decision lags or errors caused by mismatched coordinator capabilities.

[0161] The decision-making and coordination roles are dynamically shifted as the stage progresses, avoiding excessive load caused by a single agent bearing the coordination task for a long time (such as having to handle its own financial data tasks while also dealing with a large number of adjudication requests). At the same time, it allows more agents to participate in the coordination work, improves the overall coordination capabilities of the agents, and reserves more qualified coordinators for the financial data processing system to meet the needs of peak business periods.

[0162] In a second aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-agent cooperative system construction method as described in the first aspect of the present invention.

[0163] The computer-readable storage medium may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0164] The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD ROM); the magnetic surface memory may be a disk storage device or a magnetic tape storage device.

[0165] The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synclink dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The computer-readable storage media described in the embodiments of the present invention are intended to include these and any other suitable types of memory.

[0166] like Figure 10 As shown, in a third aspect, the present invention provides an electronic device 10, including a processor 101 and a storage medium 102, wherein a computer program is stored on the storage medium, and the computer program, when executed by the processor, implements the multi-agent cooperative system construction method as described in the first aspect of the present invention.

[0167] In some embodiments, the processor may be implemented by software, hardware, firmware, or a combination thereof, and may be a circuit, one or more of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, or a microprocessor, thereby enabling the processor to execute some or all of the steps or any combination thereof in the multi-agent cooperative system construction method described in the various embodiments of this application.

[0168] Finally, it should be noted that although the above embodiments have been described in the text and drawings of this application, this should not limit the scope of patent protection of this application. Any technical solutions that are based on the essential concept of this application and utilize the content described in the text and drawings of this application, resulting in equivalent structural or procedural substitutions or modifications, as well as the direct or indirect application of the technical solutions of the above embodiments to other related technical fields, are all included within the scope of patent protection of this application.

Claims

1. A method for constructing a multi-agent cooperation system, characterized by comprising: The method comprises: S1: task modeling is performed on a target service scenario to obtain a task set containing a plurality of atomic tasks, and a task dependency graph is constructed based on dependency relationships between the atomic tasks; S2: an initial cooperation strength weight is set for each pair of atomic tasks having a cooperation relationship in the task dependency graph, and a cooperation complexity of each atomic task is calculated based on the task dependency graph, all atomic tasks are sorted in ascending order according to the cooperation complexity, and a multi-stage cooperation training course is generated; S3: a plurality of agents are controlled to execute tasks in the multi-stage cooperation training course, the plurality of agents constitute a cooperation unit, and an optimization cycle for a cooperation strategy is executed in a training process, specifically comprising: S31: interaction behavior data between the plurality of agents is collected, the interaction behavior data comprising a negotiation round number between an initiator and a receiver in a task handover process, a decision conflict event frequency caused by resource competition, and an average idle duration of the agents in a task waiting state; S32: an overall task completion efficiency of the cooperation unit is calculated, the initial cooperation strength weight is dynamically updated based on the negotiation round number and the decision conflict event frequency; S33: the interaction behavior data and the updated cooperation strength weight are input into a reinforcement learning model, the overall task completion efficiency is taken as a reward signal, the cooperation strategy is iteratively optimized, and the optimized cooperation strategy is taken as an initial cooperation strategy for executing tasks in a next stage, the cooperation strategy being used to guide the agents to perform task handover and conflict arbitration; specifically comprising: S331: the interaction behavior data and the updated cooperation strength weight are normalized, a normalized cooperation strength weight vector and a normalized interaction behavior data vector are spliced to obtain a comprehensive feature vector, and the comprehensive feature vector is taken as a state space S of the reinforcement learning model; S332: a composite reward function R combining positive efficiency rewards and negative behavior penalties is constructed to generate the reward signal, and a calculation formula is as follows: R = w e × E - (w n × N n + w c × N c + w i × T i ); wherein E is the normalized overall task completion efficiency, N n denotes the normalized round of task handover negotiation, N c denotes the normalized frequency of decision conflict events caused by resource competition, T i denotes the normalized average idle duration of the agent in the task waiting state; w e ,w n ,w c ,w i respectively denote preset weight coefficients for balancing the contribution of each index. S333: the reinforcement learning model is iteratively optimized based on the state space S and the composite reward function R, and when a change amount of the overall task completion efficiency in a plurality of continuous training iterations is lower than a preset threshold, it is determined that the cooperation strategy converges; S4: after the multi-stage cooperation training course is completely executed and the optimization cycle of the cooperation strategy converges, a final output cooperation strategy is solidified into a completed multi-agent cooperation system.

2. The multi-agent collaboration system construction method of claim 1, wherein, Step S1 comprises: S11: historical work order data and a business process specification document of the target service scenario are acquired, and the business process specification document records work procedure rules used to define task execution logic and relationships; S12: the historical work order data is decomposed into a plurality of atomic tasks capable of being independently executed by a single agent based on the work procedure rules, and the task set is obtained, and each atomic task is associated with at least the following attributes: a task type, an estimated execution time, and a required resource type set. S13: parse the process rules in the business process specification document to obtain process order and data flow direction rules, construct a time sequence dependency relationship between a plurality of atomic tasks based on the process order, and construct a data dependency relationship between the plurality of atomic tasks based on the data flow direction rules; S14: obtain the task dependency graph based on the time sequence dependency relationship and the data dependency relationship.

3. The multi-agent collaboration system construction method of claim 2, wherein, In step S2, for each pair of atomic tasks with a cooperation relationship in the task dependency graph, an initial cooperation strength weight is set, including: S21: traverse all dependency edges in the task dependency graph, and determine two atomic tasks directly connected by a dependency edge as a pair of atomic tasks with a cooperation relationship; S22: for each of the atomic task pairs, calculate an initial cooperation strength weight based on the attributes of the atomic tasks, and the calculation formula is as follows: W initial = α x F type + β x F complexity + γ x F coupling ; wherein W initial represents the initial cooperation strength weight; F type denotes a dependency type factor, the value of which is determined based on the type of dependency relationship between the pair of atomic tasks, wherein a data dependency relationship is assigned a first weight value and a timing dependency relationship is assigned a second weight value; F complexity denotes a task complexity factor, the value of which is determined based on an average of the estimated execution time of the two atomic tasks of the pair of atomic tasks; F coupling denotes a task coupling factor, whose value is determined based on the intersection size of the resource type sets required by the two atomic tasks in the atomic task pair; α, β, and γ are preset weighting coefficients.

4. The multi-agent collaboration system construction method of claim 1, wherein, In step S333, based on the state space S and the composite reward function R, the reinforcement learning model is iteratively optimized, including the following steps: S3331: set a centralized evaluation network, the input of the centralized evaluation network is the state space S, which is used to evaluate the long-term value of the global cooperation strategy; for each agent in the cooperation unit, a distributed policy network is configured, each distributed policy network generates individual action decisions for the agent based on the local observation information of the corresponding agent and the global cooperation relationship represented by the state space S; S3332: based on the state space S and the composite reward function R, the centralized evaluation network and all distributed policy networks are jointly iteratively trained, specifically including: using the reward signal generated by the composite reward function R, updating the parameters of the centralized evaluation network through time difference learning, based on the long-term value estimate output by the centralized evaluation network, generating a policy update gradient through a policy gradient method, and synchronously optimizing the parameters of all distributed policy networks accordingly; S3333: repeat step S3332 until the reinforcement learning model converges, and finally deploy the distributed policy network to each agent in the cooperation unit.

5. The multi-agent collaboration system construction method of claim 1, wherein, In step S33, the counterfactual reasoning mechanism is used as an auxiliary optimization means during iterative optimization, which is executed in each training iteration, specifically including: taking the cooperation strategy output by the reinforcement learning model in the current iteration as a baseline strategy, based on the baseline strategy, the interaction behavior data and the updated cooperation strength weight, performing the following operations: From the task set currently being executed, identify the atomic task that performs the highest negotiation round or the highest decision conflict event frequency in the interaction behavior data as the key atomic task to be modified; virtually modify the cooperation elements related to the key atomic task to generate at least one counterfactual cooperation scheme; the modification of the cooperation elements is specifically: replacing the agent executing the key atomic task, or changing the negotiation object order of the key atomic task in the task handover process; deduce a virtual task completion efficiency that can be achieved by executing the counterfactual collaboration scheme based on the updated collaboration intensity weight; compare the highest virtual task completion efficiency deduced with a current overall task completion efficiency calculated based on the baseline strategy and the interaction behavior data, and if the highest virtual task completion efficiency is higher than the current overall task completion efficiency, generate a strategy update gradient according to the corresponding counterfactual collaboration scheme, and apply the strategy update gradient to the reinforcement learning model to optimize the collaboration strategy.

6. The multi-agent collaboration system construction method of claim 1, wherein, In step S33, the iterative optimization of the collaboration strategy is achieved through an agent behavior feedback mechanism, specifically including: calculating a corresponding individual collaboration behavior quantitative value for each agent in the collaboration unit based on the interaction behavior data; the individual collaboration behavior quantitative value at least includes a task contribution value, a negotiation participation degree, and a decision-making influence; inputting the individual collaboration behavior quantitative value of each agent into the reinforcement learning model, and generating an individual strategy optimization signal for each agent based on the overall task completion efficiency and the individual collaboration behavior quantitative value by the reinforcement learning model, the individual strategy optimization signal being used to guide the agent to adjust its behavior mode in subsequent task handover and conflict arbitration; In the process of iterative optimization of the collaboration strategy by the reinforcement learning model, the individual strategy optimization signal and a global optimization signal based on the interaction behavior data are fused to jointly drive the update of the collaboration strategy, so that the optimized collaboration strategy can reflect the optimization needs of global task efficiency and individual collaboration behavior at the same time.

7. The multi-agent collaboration system construction method of claim 1, wherein, The method further includes: dynamically assigning an agent with decision-making coordination authority in each training phase of the multi-phase collaboration training course based on the optimized collaboration strategy, specifically including: S901: In each training phase, calculate a coordination ability evaluation value for each agent in the collaboration unit based on the interaction behavior data and the updated collaboration intensity weight, the coordination ability evaluation value being calculated according to the current agent's task processing historical success rate, the current resource available state, and the collaboration intensity of its associated tasks; S902: dynamically assign the agent with the highest coordination ability evaluation value in the current training phase as the decision-making coordinator, the decision-making coordinator being authorized to perform task handover arbitration and decision conflict arbitration guided by the collaboration strategy in step S33; S903: when the collaboration unit enters the next training phase according to the order of the multi-phase collaboration training course, automatically trigger steps S901 and S902 to update the decision-making coordinator.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the multi-agent collaboration system construction method of any one of claims 1 to 7.

9. An electronic device on which a computer program is stored, characterized by The program is executed by the processor to implement the multi-agent collaboration system construction method of any one of claims 1 to 7. The program is executed by the processor to implement the multi-agent collaboration system construction method of any one of claims 1 to 7.

Images