Task cooperation method, apparatus, device, storage medium, and computer program product

Abstract

The application discloses a multi-agent task cooperation method and device, equipment, storage medium and computer program product, to solve the problem that the prior art is difficult to efficiently adjust the cooperation strategy and maintain the overall performance stability of the system in the cooperation scene of dynamic change of the number of agents. The method comprises the following steps: determining the similarity distance between each agent in a multi-agent system according to the current running state of each agent, and dynamically constructing a proximity relationship graph according to the similarity distance; determining at least one agent group corresponding to each agent according to the proximity relationship graph; configuring a strategy network corresponding to each agent group, and determining a candidate task action set corresponding to each agent according to the strategy network of at least one agent group corresponding to each agent; weighting and merging each candidate task action in the candidate execution action set to obtain a final task action corresponding to each agent, and performing task processing according to the final task action.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, storage medium, and computer program product for multi-agent task collaboration. Background Technology

[0002] Multi-Agent Deep Reinforcement Learning (MADRL) constructs a distributed collaborative framework that enables multiple agents to learn autonomously and collaborate in dynamic environments. It can be widely applied in vertical fields such as intelligent manufacturing, traffic control, and the Internet of Things. Within the MADRL framework, each agent utilizes deep neural networks to process complex states and action spaces, and continuously optimizes its own strategies through reinforcement learning mechanisms to improve the overall system's decision-making efficiency and environmental adaptability.

[0003] However, in practical deployments, especially in scenarios where the number of agents changes dynamically, existing MADRL methods suffer from the following technical problems: As the number of agents increases, the system's state space and action space expand exponentially, making it difficult to directly extend and reuse existing cooperation strategies, requiring complex and time-consuming adjustments. Furthermore, agents need the ability to quickly and adaptively adjust their strategies when the number changes to maintain the stability of the overall system performance. Existing centralized or global coordination methods often have high computational overhead, poor scalability, and difficulty in effectively capturing local dependencies between agents, thus limiting their practicality and generalization ability in dynamically scaled multi-agent systems.

[0004] Therefore, there is an urgent need for a multi-agent deep reinforcement learning method that can effectively handle changes in the number of agents while taking into account both local cooperation and global coordination. Summary of the Invention

[0005] This application provides a multi-agent task collaboration method to solve the problem that existing technologies struggle to efficiently adjust collaboration strategies and maintain overall system performance stability in collaboration scenarios where the number of agents changes dynamically.

[0006] This application also provides a multi-agent task collaboration device to solve the problem that in existing technologies, it is difficult to efficiently adjust the collaboration strategy and maintain the overall performance stability of the system in collaboration scenarios where the number of agents changes dynamically.

[0007] This application also provides a multi-agent task collaboration device to solve the problem that existing technologies struggle to efficiently adjust collaboration strategies and maintain overall system performance stability in collaboration scenarios where the number of agents changes dynamically.

[0008] This application also provides a computer-readable storage medium to address the problem that in collaborative scenarios where the number of agents changes dynamically, it is difficult to efficiently adjust the collaboration strategy and maintain the overall performance stability of the system.

[0009] A computer program product designed to address the problem that existing technologies struggle to efficiently adjust collaboration strategies and maintain overall system performance stability in collaborative scenarios where the number of agents changes dynamically.

[0010] The embodiments of this application adopt the following technical solutions: A multi-agent task collaboration method includes: determining the similarity distance between each agent in a multi-agent system based on the current running state of each agent, and dynamically constructing a proximity graph based on the similarity distance; determining at least one agent group corresponding to each agent based on the proximity graph, wherein the agent group includes at least two agents with a similarity distance less than a preset threshold; configuring a policy network corresponding to each agent group, and determining a set of candidate task actions corresponding to each agent based on the policy network of at least one agent group corresponding to each agent; weighting and merging the candidate task actions in the set of candidate action actions to obtain the final task action corresponding to each agent, and performing task processing according to the final task action.

[0011] A multi-agent task collaboration device includes: a proximity graph construction unit, configured to determine the similarity distance between each agent in a multi-agent system based on the current running state of each agent, and dynamically construct a proximity graph based on the similarity distance; an agent group determination unit, configured to determine at least one agent group corresponding to each agent based on the proximity graph, wherein the agent group includes at least two agents with a similarity distance less than a preset threshold; a candidate action set determination unit, configured to configure a policy network corresponding to each agent group, and determine a candidate task action set corresponding to each agent based on the policy network of at least one agent group corresponding to each agent; and an execution unit, configured to perform weighted merging of each candidate task action in the candidate task action set to obtain the final task action corresponding to each agent, and perform task processing according to the final task action.

[0012] A multi-agent task collaboration device, comprising: The system includes a processor and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the following operations: determining the similarity distance between the agents in a multi-agent system based on their current operating states, and dynamically constructing a proximity graph based on the similarity distance; determining at least one agent group corresponding to each agent based on the proximity graph, wherein each agent group includes at least two agents with a similarity distance less than a preset threshold; configuring policy networks corresponding to each agent group, and determining a set of candidate task actions corresponding to each agent based on the policy networks of at least one agent group corresponding to each agent; weighting and merging the candidate task actions in the set of candidate task actions to obtain the final task action corresponding to each agent, and performing task processing according to the final task action.

[0013] A computer-readable storage medium stores one or more programs that, when executed by an electronic device including multiple applications, cause the electronic device to perform the following operations: determining a similarity distance between agents in a multi-agent system based on their current operating states, and dynamically constructing a proximity graph based on the similarity distance; determining at least one agent group corresponding to each agent based on the proximity graph, wherein each agent group includes at least two agents with a similarity distance less than a preset threshold; configuring a policy network corresponding to each agent group, and determining a set of candidate task actions corresponding to each agent based on the policy network of at least one agent group corresponding to each agent; weighting and merging the candidate task actions in the set of candidate task actions to obtain a final task action corresponding to each agent, and performing task processing according to the final task action.

[0014] A computer program product includes a computer program that, when executed by a processor, implements the following: determining the similarity distance between each agent in a multi-agent system based on the current operating state of each agent, and dynamically constructing a proximity graph based on the similarity distance; determining at least one agent group corresponding to each agent based on the proximity graph, wherein each agent group includes at least two agents with a similarity distance less than a preset threshold; configuring a policy network corresponding to each agent group, and determining a set of candidate task actions corresponding to each agent based on the policy network of at least one agent group corresponding to each agent; weighting and merging the candidate task actions in the set of candidate task actions to obtain the final task action corresponding to each agent, and performing task processing according to the final task action.

[0015] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: The multi-agent task collaboration method provided in this application, when performing multi-agent task collaboration scheduling, firstly, determines the similarity distance between each agent based on the current running state of each agent in the multi-agent system, and dynamically constructs a proximity graph based on the similarity distance. Then, based on the proximity graph, at least one agent group corresponding to each agent is determined. By configuring the policy network corresponding to each agent group, and based on the policy network of at least one agent group corresponding to each agent, a set of candidate task actions corresponding to each agent is determined. By weighted merging of each candidate task action in the set of candidate action actions, the final task action corresponding to each agent is obtained, and task processing is performed according to the final task action. The multi-agent task collaboration method provided in this application has two advantages. First, by dynamically constructing a proximity graph based on real-time running status, the similarity distance between agents can be determined, and agents can be divided into multiple local collaboration groups based on the similarity distance. This allows each agent to coordinate only with members within its neighboring group, avoiding the increased computational complexity caused by the growth in the number of agents in existing global coordination mechanisms. This significantly reduces the state and action space in the decision-making process, enabling the system to efficiently support large-scale collaborative scenarios with dynamically changing agent numbers. Second, the multi-agent task collaboration method provided in this application can dynamically adjust the number of agents based on their performance. The system dynamically generates the operational status of each agent in each agent group, enabling the system to respond in real time to environmental changes or sudden states of agents. Based on the current operational status, targeted policy network configurations are performed for each agent group, thereby improving the adaptability of the multi-agent system at both local and global levels. Finally, by obtaining different candidate task actions from multiple agent groups to which each agent belongs, and by merging the candidate task actions, the multi-agent system maintains consistency in intra-group coordination during task decision-making while taking into account the roles and influences of agents in different groups, thus improving the overall coordination of task execution and the rationality of the final action. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This application provides a schematic flowchart of a multi-agent task collaboration method according to an embodiment of the present application. Figure 2 A schematic diagram of the specific structure of a multi-agent task collaboration device provided in this application embodiment; Figure 3 This is a schematic diagram of the specific structure of a multi-agent task collaboration device provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] This application provides a multi-agent task collaboration method to address the problem that existing technologies struggle to efficiently adjust collaboration strategies and maintain overall system performance stability in collaborative scenarios where the number of agents changes dynamically.

[0019] The executing entity of the multi-agent task collaboration method provided in this application embodiment may be, but is not limited to, at least one of a task management server, a task scheduling server, and a multi-agent scheduling server; in addition, the executing entity of the method may also be the system or application (APP) itself running on these servers.

[0020] For ease of description, the following description uses a multi-agent task collaboration system as the execution subject to illustrate the implementation of this method. It should be understood that using a multi-agent task collaboration system as the execution subject is merely an illustrative example and should not be construed as a limitation of the method.

[0021] In this embodiment, a specific multi-agent task collaboration method is described using a multi-agent task collaboration approach in a smart manufacturing scenario as an example. In a material handling scenario, a variable number of Automated Guided Vehicles (AGVs) work collaboratively in a warehouse, responsible for transporting materials from the storage area to multiple assembly stations. The AGVs are the agents in the system, and their number may dynamically change due to task requirements, equipment malfunctions, or charging schedules. In this embodiment, the multi-agent task collaboration system can be deployed on a central scheduling server to coordinate and schedule tasks for all AGVs in real time.

[0022] Based on the above business scenarios, the specific implementation flowchart of the multi-agent task collaboration method provided in this application is shown in the figure below. Figure 1 As shown, the main steps include the following: Step 11: Based on the current operating state of each agent in the multi-agent system, determine the similarity distance between each agent, and dynamically construct a proximity graph based on the similarity distance; In this embodiment of the application, the multi-agent task collaboration system can acquire the current operating status of each AGV according to a preset acquisition cycle, such as one minute.

[0023] In this embodiment of the application, the multi-agent task collaboration system can uniformly convert the acquired current running state into a vector format, for example, acquiring the AGV's current running state. i The current running state is s i = [x i y i v i θ i , l i ], where (x i y i ) for AGV i In the warehouse, the two-dimensional location coordinates, v i Indicates AGV i Current operating speed, θ i Indicates AGV i Current driving direction, l i Indicates AGV i The current load status, for example, 0 indicates no load and 1 indicates full load.

[0024] In the embodiments of this application, the similarity distance between each AGV can be determined based on the actual physical distance between each AGV, such as Euclidean distance; it can also be determined based on the similarity measure of each AGV in the state space, such as cosine similarity; it can also be determined based on the distance of specific features, such as the distance in the embedding space learned by a neural network; or it can be determined based on the sum of the above distances.

[0025] The following description uses the actual physical distance between each AGV to determine the similarity distance between each AGV as an example. It should be understood that using Euclidean distance to determine the similarity distance between each AGV is only an exemplary illustration. This application does not specifically limit the method used to determine the similarity distance between each AGV.

[0026] In this embodiment of the application, any two intelligent agents, AGV, can be determined according to the following formula [1]. i and AGV j The similarity distance d between them {ij} : [1] Where α, β, and γ are preset weight coefficients used to adjust the importance of position, speed, and load status in distance calculation; I(·) is an indicator function, which is 1 when the condition is met and 0 otherwise. The smaller the similarity distance determined by the above formula [1], the more similar the states of the two AGVs are, and the stronger the interaction between the two AGVs in the task processing process.

[0027] In this embodiment of the application, the similarity distance between all pairs of AGVs is determined according to the above formula [1], and a proximity graph G = (V, E) is dynamically constructed based on the similarity distance. In this proximity graph, the vertex set V corresponds to all AGVs. A dynamic threshold ε is set according to the median distance between all pairs of AGVs at the current time. t If d {ij} <ε t Then, an edge is established between vertices i and j, forming an edge set E.

[0028] Step 12: Based on the proximity graph determined by executing Step 11, determine at least one agent group corresponding to each agent AGV, wherein the agent group includes at least two agents with a similarity distance of less than a preset threshold. In this embodiment of the application, the multi-agent task collaboration system can determine at least one agent group corresponding to each agent AGV according to the following sub-steps, including: Sub-step 1201: For each intelligent agent AGV, select a second intelligent agent whose similarity distance to the AGV is less than a preset threshold from the proximity relationship graph; Specifically, for each AGV i You can select k AGVs from other AGVs in the system whose similarity distance is less than a preset threshold as its "k-nearest neighbors". Here, k can be a preset parameter, for example, k=5.

[0029] Sub-step 1202: Based on the AGV and its k nearest neighbor AGVs, determine at least one intelligent agent group corresponding to the intelligent agent AGV.

[0030] Each AGV is grouped with its k nearest neighbors to form an initial agent group, which contains k+1 AGVs. The system initially generates n initial agent groups, where n is the total number of AGVs.

[0031] Iterate through all initial agent groups and check if there are multiple groups consisting of identical AGV sets. If so, perform a deduplication and merging operation, for example, keeping only one of the AGV groups. After deduplication and merging, m non-redundant agent groups are finally obtained.

[0032] Step 13: Configure the policy network corresponding to each agent group respectively, and determine the candidate task action set corresponding to each agent AGV based on the policy network of at least one agent group corresponding to each agent AGV. The policy network is configured to determine the initial task actions of each AGV in the agent group based on the operating status of the agent group.

[0033] In this embodiment, an independent policy network π can be configured for each non-redundant agent group. p In one embodiment, the policy network π can be constructed using a Soft Actor-Critic (SAC) architecture. p Specifically, the vectors of the current operating states of all AGVs in the intelligent agent group can be concatenated to obtain the joint state S of the intelligent agent group. {Gp} and the joint state S {Gp} As the strategy network π p The input of this policy network π p The output dimension is |G p |×|A|, where |G p | represents the number of AGVs within the intelligent agent group, and |A| represents the dimension of the action space of a single AGV, such as forward, backward, left turn, right turn, and stop.

[0034] In this embodiment of the application, in order to train the policy network π p A total reward function needs to be defined for each agent group, where the total reward function can be the sum or average of the individual reward values ​​corresponding to all AGVs within the agent group. By optimizing this total reward function, the policy network π can be improved. p Generate actions that maximize overall benefits.

[0035] For example, a total reward value can be defined for each group of agents, as shown in the following formula [2]: [2] Where, r i It is a single intelligent agent AGV i Individual rewards, the training objective is to adjust the policy network π p The parameter θ p , making the policy network π p Generate actions that maximize overall benefits.

[0036] It should be noted that, in this embodiment of the application, an off-policy deep reinforcement learning algorithm can be used to train the policy network π with the objective of maximizing the total reward value. p .

[0037] In this embodiment of the application, the policy network π can be performed according to the following sub-steps. p The training includes: Sub-step 1301: The obtained operating parameters of the intelligent agent group in the collaborative task processing process are stored as training data in the experience replay buffer. The operating parameters may include, but are not limited to, the status information, action information, and total reward value of the agent group during the collaborative task processing.

[0038] Specifically, during system operation, the multi-agent task collaboration system can collect the running parameters of each agent group at time step t and store the collected running parameters in a common experience playback buffer.

[0039] Sub-step 1302: Sample training data from the experience replay buffer, and update the parameters of the policy network based on the training data using a deep reinforcement learning algorithm, so that the task action output by the policy network can maximize the total reward value.

[0040] Specifically, multi-agent task collaboration systems can randomly sample small batches of data from a shared experience replay buffer. A quality-network (Q-network) is used to estimate state-action values. The policy network π is updated via policy gradients. p The parameters.

[0041] Based on the policy network π determined through the above steps p The candidate task action set corresponding to each agent AGV in the agent group is determined. Specifically, during the inference phase, for the current running state S of each agent group at the current time... {Gp} The current running state S {Gp} Input the trained policy network π p In this process, the initial task actions of each AGV within the intelligent agent group are obtained.

[0042] It's important to note that an AGV may belong to multiple different agent groups. Therefore, each AGV will be processed from the policy network π of all agent groups that include it. p Multiple initial actions are obtained, and the set of these actions constitutes its candidate task action set.

[0043] Step 14: Weighted merge of each candidate task action in the candidate action set obtained by executing step 13 to obtain the final task action corresponding to each intelligent agent AGV, and perform task processing according to the final task action.

[0044] In this embodiment of the application, the focus is on intelligent agent AGVs. iGiven a candidate action, assuming this candidate action is determined based on the policy network of agent group a, we can first determine the total similarity distance between the AGVs of each agent group a. This total similarity distance reflects the degree of cooperation among the AGVs of each agent group a. Then, based on this total distance, we determine the weight corresponding to the candidate action.

[0045] The AGV intelligent agent is determined sequentially using the methods described above. i The weights of all candidate actions are determined, and the action is assigned to the AGV agent based on these weights. i All candidate actions are weighted and summed to determine the agent AGV. i The corresponding final task action.

[0046] Finally, the multi-agent task collaboration system can issue the determined final task actions to the corresponding AGVs for execution. After all agents have executed the actions, the environment enters a new state, generating new rewards. The system then returns to step 11 to begin a new round of perception, grouping, decision-making, and fusion, achieving continuous dynamic collaborative optimization.

[0047] The multi-agent task collaboration method provided in this application, when performing multi-agent task collaboration scheduling, firstly, determines the similarity distance between each agent based on the current running state of each agent in the multi-agent system, and dynamically constructs a proximity graph based on the similarity distance. Then, based on the proximity graph, at least one agent group corresponding to each agent is determined. By configuring the policy network corresponding to each agent group, and based on the policy network of at least one agent group corresponding to each agent, a set of candidate task actions corresponding to each agent is determined. By weighted merging of each candidate task action in the set of candidate action actions, the final task action corresponding to each agent is obtained, and task processing is performed according to the final task action. The multi-agent task collaboration method provided in this application has two advantages. First, by dynamically constructing a proximity graph based on real-time running status, the similarity distance between agents can be determined, and agents can be divided into multiple local collaboration groups based on the similarity distance. This allows each agent to coordinate only with members within its neighboring group, avoiding the increased computational complexity caused by the growth in the number of agents in existing global coordination mechanisms. This significantly reduces the state and action space in the decision-making process, enabling the system to efficiently support large-scale collaborative scenarios with dynamically changing agent numbers. Second, the multi-agent task collaboration method provided in this application can dynamically adjust the number of agents based on their performance. The system dynamically generates the operational status of each agent in each agent group, enabling the system to respond in real time to environmental changes or sudden states of agents. Based on the current operational status, targeted policy network configurations are performed for each agent group, thereby improving the adaptability of the multi-agent system at both local and global levels. Finally, by obtaining different candidate task actions from multiple agent groups to which each agent belongs, and by merging the candidate task actions, the multi-agent system maintains consistency in intra-group coordination during task decision-making while taking into account the roles and influences of agents in different groups, thus improving the overall coordination of task execution and the rationality of the final action.

[0048] In one embodiment, this application also provides a multi-agent task collaboration device to address the problem in existing technologies where it is difficult to efficiently adjust collaboration strategies and maintain overall system performance stability in collaboration scenarios with dynamically changing numbers of agents. A schematic diagram of the specific structure of this multi-agent task collaboration device is shown below. Figure 2 As shown, it includes: a proximity graph construction unit 21, an agent group determination unit 22, a candidate action set determination unit 23, and an execution unit 24.

[0049] The proximity graph construction unit 21 is used to determine the similarity distance between each agent in the multi-agent system based on the current running state of each agent, and to dynamically construct a proximity graph based on the similarity distance. The agent group determination unit 22 is used to determine at least one agent group corresponding to each agent according to the proximity relationship graph, wherein the agent group includes at least two agents with a similarity distance of less than a preset threshold. The candidate action set determination unit 23 configures the policy network corresponding to each of the intelligent agent groups respectively, and determines the candidate task action set corresponding to each intelligent agent according to the policy network of at least one intelligent agent group corresponding to each intelligent agent. The execution unit 24 is used to perform weighted merging of each candidate task action in the candidate task action set to obtain the final task action corresponding to each intelligent agent, and to perform task processing according to the final task action.

[0050] In one embodiment, the agent group determination unit 22 is specifically configured to: select a second agent from the proximity graph that has a similarity distance of less than a preset threshold with respect to each agent; and determine at least one agent group corresponding to the agent based on the agent and the second agent.

[0051] In one embodiment, a merging unit is further included, specifically configured to: determine a redundant group of agents consisting of the same set of agents; and merge the redundant group of agents.

[0052] In one implementation, the candidate action set determination unit 23 is specifically used to: determine the total reward value corresponding to each of the agent groups, wherein the total reward value is the sum of the individual reward values ​​corresponding to all agents in the agent group; and train the policy network according to a deep reinforcement learning algorithm with the goal of maximizing the total reward value.

[0053] In one implementation, the candidate action set determination unit 23 is specifically configured to: store the acquired operating parameters of the agent group during the collaborative task processing as training data into an experience replay buffer, wherein the operating parameters include at least one of the agent group's state information, execution action information, and total reward value during the collaborative task processing; sample training data from the experience replay buffer; and update the parameters of the policy network based on the training data using a deep reinforcement learning algorithm, so that the task action output by the policy network can maximize the total reward value.

[0054] In one implementation, the candidate action set determination unit 23 is specifically used to: determine the policy gradient between the task action probability distribution output by the policy network and the actual task actions of the agent group based on the sampled training data; and update the parameters of the policy network based on the policy gradient.

[0055] The multi-agent task collaboration method provided in this application, when performing multi-agent task collaboration scheduling, firstly, determines the similarity distance between each agent based on the current running state of each agent in the multi-agent system, and dynamically constructs a proximity graph based on the similarity distance. Then, based on the proximity graph, at least one agent group corresponding to each agent is determined. By configuring the policy network corresponding to each agent group, and based on the policy network of at least one agent group corresponding to each agent, a set of candidate task actions corresponding to each agent is determined. By weighted merging of each candidate task action in the set of candidate action actions, the final task action corresponding to each agent is obtained, and task processing is performed according to the final task action. The multi-agent task collaboration method provided in this application has two advantages. First, by dynamically constructing a proximity graph based on real-time running status, the similarity distance between agents can be determined, and agents can be divided into multiple local collaboration groups based on the similarity distance. This allows each agent to coordinate only with members within its neighboring group, avoiding the increased computational complexity caused by the growth in the number of agents in existing global coordination mechanisms. This significantly reduces the state and action space in the decision-making process, enabling the system to efficiently support large-scale collaborative scenarios with dynamically changing agent numbers. Second, the multi-agent task collaboration method provided in this application can dynamically adjust the number of agents based on their performance. The system dynamically generates the operational status of each agent in each agent group, enabling the system to respond in real time to environmental changes or sudden states of agents. Based on the current operational status, targeted policy network configurations are performed for each agent group, thereby improving the adaptability of the multi-agent system at both local and global levels. Finally, by obtaining different candidate task actions from multiple agent groups to which each agent belongs, and by merging the candidate task actions, the multi-agent system maintains consistency in intra-group coordination during task decision-making while taking into account the roles and influences of agents in different groups, thus improving the overall coordination of task execution and the rationality of the final action.

[0056] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0057] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0058] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0059] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a multi-agent task cooperation device at the logical level. The processor executes the program stored in memory and specifically performs the following operations: Based on the current operating state of each agent in the multi-agent system, the similarity distance between each agent is determined, and a proximity graph is dynamically constructed based on the similarity distance. Based on the proximity graph, at least one agent group corresponding to each agent is determined, wherein each agent group includes at least two agents with a similarity distance less than a preset threshold. A policy network corresponding to each agent group is configured, and a candidate task action set corresponding to each agent is determined based on the policy network of at least one agent group corresponding to each agent. The candidate task actions in the candidate action set are weighted and merged to obtain the final task action corresponding to each agent, and task processing is performed according to the final task action.

[0060] The above is as stated in this application. Figure 3The method for multi-agent task collaboration electronic devices disclosed in the illustrated embodiments can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0061] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0062] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The multi-agent task collaboration method shown in the embodiment is specifically used to perform the following operations: Based on the current operating state of each agent in the multi-agent system, the similarity distance between each agent is determined, and a proximity graph is dynamically constructed based on the similarity distance. Based on the proximity graph, at least one agent group corresponding to each agent is determined, wherein each agent group includes at least two agents with a similarity distance less than a preset threshold. A policy network corresponding to each agent group is configured, and a candidate task action set corresponding to each agent is determined based on the policy network of at least one agent group corresponding to each agent. The candidate task actions in the candidate action set are weighted and merged to obtain the final task action corresponding to each agent, and task processing is performed according to the final task action.

[0063] It should be understood that the training and prediction processes of the AI ​​models involved in the various embodiments of this specification all adhere to multiple legal and compliant principles, including legal data sources, compliant data content, compliant data governance, compliant training objectives and schemes, compliant training processes, compliant training environments and tools, and compliant ethical verification of training results, and comply with the requirements of Article 5 of the Patent Law. Among them: Data source legitimacy: All datasets used for AI model training were obtained through legal means, covering three categories: publicly authorized data, data authorized by partners, and self-collected compliant data. Publicly authorized data comes from compliant data sources following open-source licenses such as Apache 2.0, with complete copyright attribution and authorization scope clearly marked, and no unauthorized open-source code or data reuse. Data authorized by partners has been subject to formal data usage agreements, clearly defining the scope, duration, and confidentiality obligations, and possessing a complete authorization chain. For self-collected data involving personal information, strict informed consent procedures have been followed, and anonymization processes (including but not limited to field masking, feature anonymization, and differential privacy technology applications) have been implemented to remove personally identifiable information, fully complying with the requirements of relevant laws and regulations such as the "Interim Measures for the Administration of Generative Artificial Intelligence Services" and the "Personal Information Protection Law."

[0064] Data content compliance: The AI ​​model's dataset undergoes multiple screenings and cleaning processes to remove all content that may violate social morality or harm public interests. It contains no obscene, pornographic, violent, discriminatory, or information that endangers national or public safety, nor does it involve the illegal acquisition or use of genetic resources. For data in sensitive fields (such as healthcare and finance), an additional privacy-preserving computation module (including federated learning and secure multi-party computation technologies) ensures that the data is "usable but not visible," avoiding compliance risks during the original data transmission process and ensuring that the data application scenarios and uses comply with public order and good morals and industry regulatory requirements.

[0065] Data governance norms: A complete data traceability system is established during the AI ​​model training process to automatically record the source, collection time, annotation process, cleaning rules, and permission allocation of training data, generating traceable compliance reports to ensure that the data is verifiable throughout its entire lifecycle. The dataset annotation process for AI models is completed by a professional human R&D team, clearly defining the proportion of human creative contributions and avoiding reliance on AI-generated data that has not undergone substantial human modification, thus meeting the examination requirements for "human main contributions" in AI patent applications.

[0066] Training objectives and plans are compliant: The training objective of the AI ​​model is to focus on the task collaboration of intelligent agents. The training scheme and the final output results do not violate any mandatory provisions of laws and administrative regulations, do not harm the public interest or the legitimate rights and interests of others, and do not pose any potential risks of being used for illegal activities, infringing on privacy, or undermining public safety. The model strictly adheres to the ethical principle of "intelligent for good".

[0067] Training process compliance: A closed-loop training framework is adopted to ensure compliance and controllability of the training process. The specific process is as follows: First, training samples are obtained through compliant data sources. After the aforementioned data cleaning and desensitization, they are input into the neural network model to generate preliminary training results. Second, an expert system is introduced to verify the preliminary results. Based on preset rules and human expert experience, the feasibility of the results is evaluated, and outputs that may pose ethical risks or compliance hazards are corrected (such as removing decision-making logic that violates public order and good morals, and adjusting model parameters that do not comply with safety regulations). Finally, the loss function weights are dynamically optimized based on expert system feedback to strengthen the model's learning of compliant results, avoid overfitting errors or non-compliant labels, and form a closed-loop control of "data input - model training - expert verification - parameter optimization - result feedback" to ensure that the entire training process complies with A5 ethical review requirements.

[0068] Training environment and tool compliance: AI model training is implemented using nationally licensed chips and a compliant training platform. All open-source frameworks and components used in the training process have obtained their corresponding licenses, and copyright statements and patent citation information are fully retained, with no instances of infringement or reuse. The training environment is built using virtual devices (containers / virtual machines) with fixed random seeds and initial parameter configurations to ensure the reproducibility of the training process. Furthermore, through access control and operation log recording, risks such as data leakage and parameter tampering during training are prevented, ensuring the security and compliance of the training process.

[0069] Training results ethical verification compliance: After the model is trained, it undergoes additional third-party ethical compliance assessment and algorithm filing review to verify that the model output does not violate social morality or harm public interests. For potentially sensitive scenarios (such as public services and intelligent decision-making), a special result verification mechanism is established to ensure that the model always complies with Article 5 of the Patent Law and relevant laws and regulations in practical applications.

[0070] In summary, the data and training process used in the AI ​​model of this specification strictly comply with the relevant provisions of Article 5 of the Patent Law and the Patent Examination Guidelines (2023 Edition), and there are no violations of laws, social ethics, public interests, or illegal use of genetic resources. It fully meets the compliance requirements for patent authorization.

[0071] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0076] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0077] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0078] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0079] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A multi-agent task collaboration method, characterized in that, include: Based on the current operating state of each agent in the multi-agent system, the similarity distance between each agent is determined, and a proximity graph is dynamically constructed based on the similarity distance. Based on the proximity graph, at least one agent group corresponding to each agent is determined, wherein the agent group includes at least two agents with a similarity distance of less than a preset threshold; Configure the policy network corresponding to each of the intelligent agent groups respectively, and determine the candidate task action set corresponding to each intelligent agent based on the policy network of at least one intelligent agent group corresponding to each intelligent agent; The candidate task actions in the candidate action set are weighted and merged to obtain the final task action corresponding to each agent, and the task is processed according to the final task action.

2. The method according to claim 1, characterized in that, The step of determining at least one agent group corresponding to each agent based on the proximity graph specifically includes: For each of the aforementioned agents, a second agent whose similarity distance to the aforementioned agent is less than a preset threshold is selected from the proximity graph; Based on the first agent and the second agent, at least one group of agents corresponding to the first agent is determined.

3. The method according to claim 1, characterized in that, After determining at least one group of intelligent agents corresponding to each of the aforementioned intelligent agents, the method further includes: Identify redundant agent groups consisting of identical sets of agents; The redundant intelligent agent groups are merged.

4. The method according to claim 1, characterized in that, The configuration of the policy network corresponding to each of the aforementioned agent groups specifically includes: Determine the total reward value corresponding to each of the aforementioned agent groups, wherein the total reward value is the sum of the individual reward values ​​corresponding to all agents within the agent group; The policy network is trained using a deep reinforcement learning algorithm with the goal of maximizing the total reward value.

5. The method according to claim 4, characterized in that, The process of training the policy network using a deep reinforcement learning algorithm specifically includes: The obtained operating parameters of the intelligent agent group during the collaborative task processing are stored as training data in the experience replay buffer. The operating parameters include at least one of the state information, execution action information and total reward value of the intelligent agent group during the collaborative task processing. Training data is sampled from the experience replay buffer, and the parameters of the policy network are updated based on the training data using a deep reinforcement learning algorithm, so that the task action output by the policy network can maximize the total reward value.

6. The method according to claim 5, characterized in that, The updating of the policy network parameters based on the deep reinforcement learning algorithm specifically includes: Based on the sampled training data, determine the policy gradient between the task action probability distribution output by the policy network and the actual task actions of the agent group; The parameters of the policy network are updated based on the policy gradient.

7. A multi-agent task collaboration device, characterized in that, include: The proximity graph construction unit is used to determine the similarity distance between each agent in the multi-agent system based on the current running state of each agent, and dynamically construct a proximity graph based on the similarity distance. The agent group determination unit is used to determine at least one agent group corresponding to each agent according to the proximity graph, wherein the agent group includes at least two agents with a similarity distance of less than a preset threshold. The candidate action set determination unit configures the policy network corresponding to each of the agent groups respectively, and determines the candidate task action set corresponding to each agent based on the policy network of at least one agent group corresponding to each agent. An execution unit is used to perform weighted merging of each candidate task action in the candidate task action set to obtain the final task action corresponding to each intelligent agent, and to perform task processing according to the final task action.

8. A multi-agent task collaboration device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the following operations: Based on the current operating state of each agent in the multi-agent system, the similarity distance between each agent is determined, and a proximity graph is dynamically constructed based on the similarity distance. Based on the proximity graph, at least one agent group corresponding to each agent is determined, wherein the agent group includes at least two agents with a similarity distance of less than a preset threshold; Configure the policy network corresponding to each of the intelligent agent groups respectively, and determine the candidate task action set corresponding to each intelligent agent based on the policy network of at least one intelligent agent group corresponding to each intelligent agent; The candidate task actions in the candidate action set are weighted and merged to obtain the final task action corresponding to each agent, and the task is processed according to the final task action.

9. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the multi-agent task cooperation method as described in any one of claims 1-6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the multi-agent task cooperation method as described in any one of claims 1-6.

Images