Multi-agent task allocation method based on network coverage dynamic matching control

Abstract

This invention discloses a multi-agent task allocation method based on network coverage dynamic matching control, comprising the following steps: Multiple agents self-organize into a network and divide it into N sub-networks. The sub-networks adopt a leader-follower model, with the leader managing and making decisions for each sub-network. Each sub-network acquires its current environmental state and uses high-level decision-making from a hierarchical deep reinforcement learning model to choose to explore a task domain or find other sub-networks. The agents move using low-level decision-making from the hierarchical deep reinforcement learning model until they find a task domain or other sub-networks. The sub-networks that have found a task domain or other sub-networks are segmented or merged through network topology control to form a covering sub-network or a new sub-network. Agents in the covering sub-network complete task execution using a task allocation optimization model, and after task completion, they transition to the new sub-network. This method can utilize a limited number of multi-agent agents to explore task distribution and complete task allocation in environments with missing network infrastructure and unknown task distribution, thereby reducing task response time and improving task completion rate.

Description

Technical Field

[0001] This invention relates to a multi-agent task allocation method under network infrastructure constraints, and more particularly to a multi-agent task allocation method that uses agent-autonomous network coverage dynamic matching control in a state where network infrastructure is constrained and tasks are unknown. Background Technology

[0002] Multi-agent systems, due to their strong environmental adaptability and intelligent decision-making capabilities, have been widely applied in scenarios such as environmental monitoring, intelligent transportation, and intelligent manufacturing. In these complex task applications, multi-agent collaboration is an effective solution to enhance the limited capabilities of individual agents in environmental analysis and intelligent computing.

[0003] To improve the efficiency and reduce the cost of multi-agent collaboration, task allocation has become an important research problem, which can be divided into two categories: centralized methods and distributed methods. Centralized methods require all individual agents to communicate with a central controller, and therefore are only suitable for systems with stable network connectivity. In contrast, distributed methods do not rely on a complete network connection, allowing each agent to detect nearby tasks and communicate with neighboring agents, thereby promoting task conflict avoidance and collaboration. Furthermore, to improve the adaptability of multi-agent systems to complex task environments, reinforcement learning has been integrated into task allocation methods.

[0004] However, most existing task allocation methods, whether based on traditional optimization or reinforcement learning, typically assume that the agent obtains task information through a task management platform rather than relying on its own task exploration capabilities. Therefore, these methods may not be suitable for environments with unknown task distributions, such as disaster relief scenarios where base stations are damaged, requiring autonomous aerial vehicles (UAVs) to both autonomously explore tasks and perform task allocation.

[0005] Existing technology discloses a method for pre-clustering UAV swarm task allocation based on probabilistic inference using the EM algorithm. The steps are as follows: First, initialize parameters such as the number of UAVs in the swarm (U), the number of targets (N), a mountain terrain map, model parameters, clustering probabilities, and a distance matrix. Then, calculate the posterior probability using the E-step of the EM algorithm, and estimate new model parameters based on the current posterior probability in the M-step to obtain the maximum likelihood probability. If the model parameters have converged, output the final model parameters and posterior probability; otherwise, continue iterative updates. Based on the final model parameters, define the range of epsilon values ​​and their decay rate. Within this range, initialize the epsilon values ​​of the Markov process, and initialize a data structure for each epsilon value, recording the UAV position, its path, and total distance traveled. Then, set the search starting point for each UAV as its current position, and perform path search and distance statistics. During the search process, if no target is found, and the generated random number is less than the current epsilon value, the UAV performs an exploration behavior, which involves calculating the distance between its current position and all remaining targets, prioritizing the nearest target within a strict range; if no target is found, the range is expanded, and a target is randomly selected as the next moving target, updating the path, position, movement distance, and epsilon value; if the random number is not less than the current epsilon value or all targets have been found, the exploration behavior is skipped, and the paths and total movement distances of all UAVs under the current epsilon value are directly recorded. This method effectively combines EM clustering with Markov exploration strategies, improving the task allocation efficiency of UAV swarms in complex terrain.

[0006] In general, most existing multi-agent task allocation technologies rely on platforms or central controllers for decision-making and require stable communication networks to function properly; in environments lacking network infrastructure, these methods are often unusable. On the other hand, since task distribution is often unknown in real-world scenarios, although it's possible to pre-obtain distribution information through auxiliary means, this information acquisition mechanism struggles to efficiently integrate with existing task allocation algorithms. For example, using a task search system to explore the entire unknown environment and then feeding back the results to the task execution system for allocation. This results in a lack of flexibility and robustness in the overall allocation process, manifesting as slow task response times and low task completion rates.

[0007] Considering the high efficiency of task exploration in unknown environments, task exploration and task allocation techniques can be combined. However, task exploration is continuous, while task allocation requires waiting for task completion. To seamlessly integrate task exploration and allocation, multi-agent systems must reduce task response time during exploration by rapidly matching global agent distribution with local task distribution to achieve local network coverage. Furthermore, during task allocation, multi-agent systems need to improve task completion rates, requiring dynamic matching between local network connectivity and local task collaboration needs. Effectively coordinating these two aspects to achieve rapid task response and allocation remains a challenging problem to solve. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to explore the task distribution and complete the task allocation in an environment where network infrastructure is lacking and the task distribution is unknown, so as to reduce the task response time and improve the task completion rate.

[0009] To address the aforementioned technical problems, the multi-agent task allocation method based on network coverage dynamic matching control of the present invention includes the following steps:

[0010] A multi-agent network is formed and divided into N subnetworks. Each subnetwork adopts a navigator-follower model, with the navigator managing and making decisions. Each subnetwork acquires the current environmental state and uses high-level decisions from a hierarchical deep reinforcement learning model to choose to explore the task domain or find other subnetworks. The agents move using low-level decisions from the hierarchical deep reinforcement learning model until they find the task domain or other subnetworks. Through network topology control, the subnetworks that have found the task domain or other subnetworks are segmented or merged to form a covered subnetwork or a new subnetwork. Agents in the covered subnetwork use a task allocation optimization model to complete task execution, and after completing the task, they are transferred to the new subnetwork.

[0011] In the above method, in the navigator and follower pattern, the navigator is the agent with the highest intermediary centrality in the subnet, responsible for collecting and processing the task and location information of the followers.

[0012] In the above method, the hierarchical deep reinforcement learning model uses sub-network load utilization (SLR) to measure the amount of task information obtained after high-level decision-making.

[0013]

[0014] Where T(N) f ) indicates that in subnetwork N f The task distribution set obtained in LR j Represents task Tj The load requirements, LC max Represents agent A i Maximum load capacity.

[0015] In the above method, when exploring the task domain, low-level decision-making is achieved through exploring utility (EU). t The decision on the mobile reward for the sub-network, the

[0016]

[0017] Among them, TD t Let TU be the task distribution density at time step t. t The urgency of the task is defined as time step t.

[0018] In the above method, the task distribution density

[0019]

[0020] Where N f,t dt represents the subnet at time step t. i,t TN(dt) represents the set of tasks within the detection range of the i-th agent at time step t. i,t ) represents the number of tasks in the set, PI represents pi, and R d,i The detection radius of the i-th agent; the task urgency

[0021]

[0022] Among them DT t TN(DT) represents the set of tasks detected by the subnet at time step t. t ) represents the number of tasks in the set, ET j BT represents the termination time of task j. j This indicates the time when task j was generated.

[0023] In the above method, the optimal task domain is selected based on task domain priority, where the priority...

[0024] DL k =-d×p

[0025] The calculation rules for d and p are as follows:

[0026] d=||P Leader -P j* ||

[0027] Where P Leader This indicates the position of the navigator in the subnet, while P... j* Indicates the geometric center of the task domain;

[0028]

[0029] Among them ET j This represents the termination time of task j, while TN(D) represents the termination time of task j. k ) indicates the number of tasks in the task domain.

[0030] In the above method, when searching for other sub-networks, the low-level decision-making is achieved through consensus utility CU. t The decision on the mobile reward for the sub-network, the

[0031]

[0032] Among them, AD t Let UI be the agent distribution density at time step t. t The information update interval is defined as time step t.

[0033] In the above method, the agent distribution density

[0034]

[0035] Where N f,t Denotes the subnetwork at time step t, da i,t AN(da) represents the set of agents within the detection range of the i-th agent at time step t. i,t ) represents the number of agents in the set, PI represents pi, and R d,i The information update interval represents the detection radius of the i-th agent.

[0036] UI t =time-t

[0037] Where time represents the time step in which the last information exchange was completed, and t represents the current time step.

[0038] In the above method, the selection priority of the agent is used. The selection of segmentation agents is based on the load capacity requirements of the task domain, and the selection priority is...

[0039]

[0040] in For the task domain geometric center The line connecting to the target position O, Represented as a task domain geometric center With agent position P i f The connection, It represents the cosine of the angle formed by the two lines.

[0041] After determining the selection priority, based on the task domain Number of agents required

[0042]

[0043] in Represents the task domain Tasks in LC max Represents intelligent agents Maximum load capacity.

[0044] In the above method, the task allocation optimization model is:

[0045]

[0046] Where m represents the total number of agents participating in task allocation, and n represents the total number of tasks in the task domain; x i,j Represents task T j Is it assigned to agent A? i x i,j =1 indicates successful allocation, x i,j =0 indicates that the allocation was unsuccessful; d i,j Represents agent A i To Task T j The distance; in formula (a), the task T involved in the assignment j The total workload of the intelligent agent must meet the workload requirements of the task, where y j Indicates the task allocation status, y j =1 indicates that it has been allocated, y j =0 indicates no allocation; in formula (b), agent A i The actual load it undertakes cannot exceed its maximum load capacity; in formula (c), the loads assigned to the same task T... j The network enables connectivity between intelligent agents.

[0047] The method of the present invention has the following advantages:

[0048] 1. It can achieve adaptive fusion of task exploration and task allocation, enabling multi-agent systems to detect local task distribution in unknown task distribution environments and adaptively switch between task exploration mode and task allocation mode. It supports simultaneous task exploration and task allocation, with the two alternating, which is beneficial to improving task response speed and global task completion rate.

[0049] 2. It can achieve multi-agent network topology control that balances the flexibility of task exploration and the optimization of local task allocation. It allows the system to split and merge subnets according to the needs of task exploration and allocation, and deploy a leader-follower pattern in the subnet to maintain the stability of the subnet, thereby optimizing the adaptability of the subnet to the local task distribution and ultimately improving the efficiency of local task collaboration.

[0050] 3. A task allocation model based on subnet collaboration optimization is proposed to optimize the task collaboration performance from the perspective of multi-agent network connectivity, ensure the stability of network connectivity quality between agents performing tasks, reduce agent movement costs, and improve the task completion rate of local subnets. Attached Figure Description

[0051] Figure 1 This is a schematic diagram illustrating the implementation process of the method of the present invention;

[0052] Figure 2 This is a flowchart illustrating the overall steps of the method of the present invention. Detailed Implementation

[0053] To visualize the entire plan, such as Figure 1 As shown in -a, a subnetwork N0 searches for the optimal task distribution area in the environment, and the navigator leads the subnetwork in exploration. From Figure 1 -a to Figure 1 -b, subnetwork N0 finds the optimal task domain, where four agents detach and become part of the overlapping subnetwork N1 to perform task assignments. Subnetwork N0 then continues moving to find other unfamiliar subnetworks N2. From Figure 1 -b to Figure 1 -c, subnetwork N0 and unfamiliar subnetwork N2 are merged into a new subnetwork N3 to find the optimal task distribution area.

[0054] Specifically, in environments with unknown and dynamically changing task distributions, the multi-agent system self-organizes into a network. It utilizes a hierarchical deep reinforcement learning model to explore local task distributions and further employs network topology control methods to achieve network coverage of these local task distributions, thus parallelizing task exploration and task allocation. Within the coverage network, the multi-agent system uses a task allocation optimization model to minimize coverage movement costs and improve load capacity utilization, ultimately determining the optimal task allocation scheme.

[0055] See Figure 2 The specific implementation steps are as follows.

[0056] S1. Multiple agents form a self-organizing network and are allowed to be divided into multiple self-organizing subnetworks. A leader-follower pattern is deployed in each subnetwork, with the leader performing subnetwork management and decision-making.

[0057] In an environment with an unknown dynamic task distribution, a self-organizing multi-agent system is deployed to continuously explore and allocate tasks. To learn the patterns of task distribution changes and exploration decisions in the dynamic environment, our environment is set up as shown in the diagram. The size of the environment is set to l×l, and the task set is defined as T={T1,T2,…,T…}. j}, where task T j The attributes are represented as TS j ={P j BT j ,ET j ,LR j CT j}, P j For the task location coordinates, BT j For the time when the task is generated, ET j For the task termination time, LR j To meet the workload requirements of the task, CT j This refers to the execution time of the task.

[0058] The set of intelligent agents is defined as A = {A1, A2, ..., A...} i}, Agent A i The attribute is represented as AS i ={P i ,LC i ,R d ,R c ,v,TL i}, where P i For the agent's position coordinates, LC i R is the load capacity of the agent. d R is the detection radius of the agent. c Let TL be the communication radius of the agent, v be the movement speed of the agent, and TL be the speed of movement of the agent. i This is a list of historical task information for the intelligent agent.

[0059] In addition, agent A i It possesses self-localization capabilities and is equipped with high-precision radar equipment, such as the 2GHz band, and wireless communication chips, such as the 5.8GHz band. All intelligent agents possess the same R... c and R d To achieve wide-area detection and high-frequency wireless communication, R d Set to greater than R c Agent A i Within its detection range, it acquires distribution information of other agents and tasks; within its communication range, it maintains continuous network communication with other agents, enabling efficient task collaboration.

[0060] Before entering the environment, each agent is informed of its size but not of the task distribution within it. Therefore, agents need to autonomously explore the environment to acquire task T during the exploration process. j Attributes of TS j And save it in the information list TL i In the middle, the attribute TS was obtained for the first time. j At that time, its BT j Initialized as an agent detecting task T j The time step, and ET j Initialize to the flag Auto.

[0061] To extend the agent's task exploration capabilities and acquire more environmental information to aid in task exploration and movement decisions, we require the agent to construct a connected network during task exploration, allowing it to form multiple isolated subnets N. f (f>0), where f represents the network number of the subnet.

[0062] To control the stability of individual subnets and provide unified guidance, we will deploy a navigator-follower model in each subnet. Specifically, in each subnet, the agent with the highest network betweenness centrality is selected as the navigator, and the other agents act as followers. The navigator will collect distribution information from the followers at each step, then synthesize and analyze this distribution information, and feed it back to all followers to achieve parameter sharing and consensus on movement decisions.

[0063] Furthermore, in order to enable the multi-agent subnet N f Based on the exploration results, we can better explore the local task distribution, subnet N f The discovered task distributions are aggregated according to distance to form task domains. Where k represents subnet N f The task domain number. Specifically, the location distance is less than R. c Tasks whose communication radius (of the agent) and whose termination time difference is less than the threshold λ belong to the same task domain.

[0064] S2. Each sub-network obtains the current environmental state, and the higher layers of the hierarchical deep reinforcement learning model determine the action S3 or S4 within consecutive time steps.

[0065] A hierarchical reinforcement learning model is designed to guide multi-agent agents in determining the optimal task domain or unfamiliar subnet. This model divides the complex problem of multi-agent task exploration into two decision-making levels: higher-level decisions guide policy selection for the multi-agent subnet, including exploration movement strategies for finding the optimal task domain and consensus movement strategies for finding the optimal unfamiliar subnet; lower-level decisions guide the movement directions of agents within the subnet. The model's state, action, policy, and reward are defined as follows:

[0066] State space: Network N f The state space is defined as S = {S T ,S X ,S Y}, including task distribution information S T The distribution of intelligent agents in the network itself, S X and the distribution of unfamiliar intelligent agents within the detection range S Y Among them, S T =TL Leader , representing network N f The historical task information list, whose dimension α is the product of the maximum list length and the task attribute length; S X ={i∈N f |P i1 ,P i2 ,…} represents the position coordinates of each agent in the subnet; S Y ={i ′ ∈A / N f |P i′ 1,P i′ 2,P i′ 3,…} represents the position coordinates of other agents (from other networks). Coordinates within the detection range are represented by their actual coordinates, while coordinates outside the detection range are filled with 0. Then S X With S Y The dimensionality of the merged network is the product of the total number of agents and the number of coordinate bits, β. Therefore, the network N... f The state dimension is α+β, and the state s at time step t is... t ∈S.

[0067] Action space: Network N f The action space is discrete and defined as Action. Any agent A... i If the action space is the integer angle values ​​from 0 to 359 on the horizontal plane, then the action space of the subnet is Action={i∈N f |a i1 ,a i2 ,a i3 ,…},a i ∈[0,360) and Subnet N represents the set of actions of all agents in the network. f In the lower-level action a at time step t t ∈Action. Additionally, if network N f Actions of agents that do not exist in the system are filled with -1.

[0068] Option space: Network N f The option space is discrete and defined as Options. During network movement, there are two basic options, where option1 represents the current network N. f Choose the exploration movement strategy, which is to explore the optimal task domain by moving through subnets. Option 2 represents the current network N. f Choosing a consensus-based movement strategy, i.e., finding the best unfamiliar subnet through subnet movement, means the option space is represented as Option = {option1, option2}, therefore, subnet N... f In time step t, the option o t ∈Option.

[0069] Strategy: In the model presented in this paper, subnet N f First, options need to be selected through a higher-level strategy ω, defined as ω(o t |s t ). And in option o t The network will then execute the corresponding in-option policy π, which determines the network's movement direction. This policy π is defined as π(a t |s t ,o t Therefore, the hierarchical strategy at time step t is defined as Π(a t |s t )=∑ o∈O ω(o t |s t )π(a t |s t ,o t ), and guide the subnet to move a distance v in the target direction.

[0070] Rewards: To better guide action selection, this paper designs rewards for both high-level and low-level decisions, with rewards for high-level decisions... The reward will be obtained at time step t when each option terminates, while the reward for lower-level decisions... This is obtained at each time step t.

[0071] For any subnet N f It is necessary to weigh the trade-offs between exploration and consensus-based mobility strategies to obtain sufficient information on task and agent distribution in order to find the optimal task domain.

[0072] 1) High-level reward design

[0073] To achieve the optimal strategy selection, we define subnet N. f The load utilization rate (SLR) is used to measure the amount of task information obtained after choosing between an exploration move strategy or a consensus move strategy.

[0074]

[0075] Where T(N) f ) indicates that in subnet N f The task distribution set obtained in LR j Represents task T j The load requirements, LC max Represents agent A i Maximum load capacity.

[0076] Specifically, after making a strategy choice at time step t, the process enters a lower-level decision-making phase. After n time steps, once the termination condition for the lower-level decision-making is met, the next strategy choice will occur at time step t+n. Therefore, the reward for the higher-level decision-making at time step t is...

[0077]

[0078] SLR t+n This indicates the subnet load utilization after implementing the policy, while SLR... t This indicates the subnet load utilization rate before the policy was implemented.

[0079] 2) Termination conditions for high-level decision-making

[0080] After making a high-level decision, the multi-agent subnet will move to a lower-level decision-making stage, guided by the lower-level decisions to select movement directions, complete task allocation or information exchange, and then enter the next round of high-level decision-making, thus repeating the cycle. Since the task distribution is continuously and dynamically changing, its final termination time is uncertain; therefore, the termination condition for high-level decision-making is set to a fixed termination time T. f .

[0081] S3. Find the optimal task distribution region. The sub-network then determines the overall movement direction of the agent in the sub-network based on the lower layer 1 of the hierarchical deep reinforcement learning model and completes the movement. If the optimal task distribution region is not found, continue to S3; if it is found, proceed to S5.

[0082] When the higher-level decision-making adopts an exploratory movement strategy, the goal of the lower-level decision-making is to find the optimal task domain to be assigned as quickly as possible. An exploration utility (EU) is defined to determine the movement reward for a subnet, and it is primarily influenced by two key factors: task distribution density (TD) and task urgency (TU). At time step t, the exploration utility EU... t Calculated as,

[0083]

[0084] Specifically, at time step t, the task distribution density TD t Calculated as,

[0085]

[0086] Where N f,t dt represents the subnet at time step t. i,t TN(dt) represents the set of tasks within the detection range of the i-th agent at time step t. i,t ) represents the number of tasks in the set, PI represents pi, and R d,i This represents the detection radius of the i-th agent.

[0087] At time step t, the mission urgency TU t Calculated as,

[0088]

[0089] Among them DT t TN(DT) represents the set of tasks detected by the subnet at time step t. t ) represents the number of tasks in the set, ET j BT represents the termination time of task j. j This indicates the time when task j was generated.

[0090] Therefore, at any time step t, the movement reward for choosing the exploration movement strategy is... for,

[0091]

[0092] EU t The exploration utility represents the time step t, while EU t-1 The exploration utility represents the time step t-1.

[0093] Termination condition: For any subnet N f Its exploration and movement strategy termination condition is defined as follows: (1) For any task domain k, where the distance between any task and at least one agent in the subnet is less than R d / 2; (2) within the subnet detection range, the distances to all tasks in task domain k are less than R d Strangers whose distance to the nearest task point is less than / 2 are considered competitors, and the sum of the distances from competitors to their respective nearest task points is calculated as a; select the same number of agents from the subnet and the sum of the distances from them to their respective nearest task points is the minimum b; if b < a, then this task domain is obtained by N f (3) Select the task domain with the highest priority DL from all task domains.

[0094] Specifically, (1) and (2) are necessary conditions, and the purpose of setting is to explore whether there are other subnets that may select the same task domain, and confirm which subnet decision these task domains belong to to avoid selection conflicts. (3) is a sufficient condition, select the best task domain from multiple task domains to maximize the overall efficiency. Further, the overall efficiency achieved by condition (3) is mainly reflected in the distance d between the subnet and the task domain and the urgency p of the task domain. Therefore, the priority DL of task domain k k is calculated as follows,

[0095] DL k = -d × p

[0096] where the calculation rules for d and p are as follows,

[0097] d = ||P Leader - P j* ||

[0098] where P Leader represents the position of the Leader in the subnet, and j* represents the geometric center of the task domain.

[0099]

[0100] where ET j represents the termination time of task j, and TN(D k ) represents the number of tasks in the task domain.

[0101] S4. Search for other self-organizing subnets. The subnets then determine the overall movement direction of the agents in the subnets based on the lower layer 2 of the hierarchical deep reinforcement learning model and complete the movement. If no other self-organizing subnets are found, continue with S4; if found, switch to S6.

[0102] When selecting the consensus movement strategy, the subnet will approach other agent subnets and communicate to exchange task information

[0103] When the higher-level decision-making selects a consensus movement strategy, the lower-level decision-making aims to find a connectable unfamiliar subnet. A consensus utility CU is defined as the movement reward used to determine the subnet, primarily influenced by two key factors: agent distribution density (AD) and information update interval (UI). At time step t, the consensus utility CU... t Calculated as,

[0104]

[0105] Specifically, at time step t, the agent distribution density AD t Calculated as,

[0106]

[0107] Where N f,t da represents the subnet at time step t. i,t AN(da) represents the set of agents within the detection range of the i-th agent at time step t. i,t ) represents the number of agents in the set, PI represents pi, and R d,i Let represent the detection radius of the i-th agent. At time step t, the information update interval is UI. t Calculated as,

[0108] UI t =time-t

[0109] Where time represents the time step in which the last information exchange was completed, and t represents the current time step.

[0110] Therefore, at any time step t, the movement reward for choosing indirect exchange is... for,

[0111]

[0112] Among them CU t CU represents the exchange utility at time step t. t-1 The exchange utility represents the time step t-1.

[0113] Termination condition: For any subnet N f Its indirect exchange strategy terminates when there exists an agent in the subnet whose distance to any unfamiliar agent within the detection range is less than the agent's communication distance R. c At this point, the subnet and the unfamiliar subnet are connected and can exchange information.

[0114] S5. Use network topology control methods to segment the network. Based on the requirements of the optimal task distribution area, select some agents to leave the sub-network and proceed to S7. If the sub-network exists, continue to S2; otherwise, proceed to S8.

[0115] During the task exploration process, the multi-agent subnet N f After determining the optimal task domain, a temporary subnet will be constructed for task allocation through network splitting, but this will disrupt the original network structure of the subnet. To better adapt to the environment for task exploration and allocation, we need to determine which agents will participate in task allocation and thus leave subnet N. f .

[0116] To determine network N f Which agents will be selected to perform the task requires evaluating each agent. Selection priority

[0117]

[0118] in For the task domain geometric center The line connecting the target location O (the geometric center of the secondary priority task domain), Represented as a task domain geometric center With agent location The connection, This represents the cosine of the angle formed by the two lines. If the agent moves towards the next moving target O, then... It is a positive value and the closer it is to the value, the larger it is. Represents intelligent agents With the task domain The negative value of the distance between them. Therefore, the further away from the next direction of movement and the closer to the task domain, the better. intelligent agents, The larger.

[0119] After determining the selection priority, based on the task domain Determine the load capacity requirements. Number of agents required

[0120]

[0121] in Represents the task domain Tasks in LC max Represents intelligent agents Maximum load capacity.

[0122] If the number of agents does not exceed the required number, all agents participate in task allocation; otherwise, priority is selected based on the agents. and the number of agents required By priority Select agents from largest to smallest to form a task collaboration subnet. Participate in task allocation (this subnet will inherit the model parameters of the original network), while subnet N f The remaining agents re-elect a new leader within the subnet to continue the mission exploration.

[0123] S6. Use network topology control methods to perform network fusion, share historical task distribution information, form new sub-networks, and continue with S2.

[0124] In order to expand the network's detection range, acquire more information on task and agent distribution, and enable the network to have greater load capacity to complete task collaboration in the task domain, fragmented networks need to be merged to give full play to the advantages of multi-agent systems.

[0125] During the task exploration process, subnet N f Find the unfamiliar subnet N connected to it. f′ Subsequently, there is a communication connection between the two agents, subnet N f With subnet N f′ Network fusion will occur between the two networks. At this point, the two networks are connected, and the agent with the highest betweenness centrality is selected as the fusion network N. m The leader is the navigator, and other intelligent agents move toward the leader and maintain connectivity with it.

[0126] To achieve efficient updates of task distribution information, we need to agree on consensus rules for information updates. Specifically, in each subnet N... f In the middle, if the intelligent agent Detect or acquire new task T j Then the new task point attribute TS j Add to historical task information list TL i In the middle. If the intelligent agent Detect or acquire mission T again j So in the list TL i Attributes of TS j In the middle, when the new BT j If the value is larger than that in the table, then update; when the new ET... j If (when not marked with the Auto flag) the value is smaller than the value in the table, then an update is performed. Specifically, when ET... j When the Auto flag is set, if a new ET is obtained. j If the value is not found, it is replaced with the new value; otherwise, its value is automatically updated at each step, i.e., the BT value at each step. j Add list TL i The summation of the durations of other tasks (ET-BT).

[0127] After the network fusion and information consensus are completed (model parameter synchronization), the network will enter the next round of task exploration.

[0128] S7. The agent detached from the subnetwork will use the task allocation optimization model to move to the task region and form a covering network, select the assigned task, and complete the task execution. It will then transition from the covering network to a new subnetwork and continue with S2.

[0129] Once the agents of the temporary subnet are determined in the multi-agent subnet, the task allocation phase begins. To further reduce task response time and improve task completion rate, a task allocation optimization model is proposed, aiming to minimize movement costs and achieve the optimal task allocation scheme. The task allocation optimization model is formulated as follows:

[0130]

[0131]

[0132] Where m represents the total number of agents participating in task allocation, and n represents the total number of tasks in the task domain; x i,j Represents task T j Is it assigned to agent A? i x i,j =1 indicates successful allocation, x i,j =0 indicates that the allocation was unsuccessful; d i,j Represents agent A i To Task T j The distance. In formula (a), the task T involved in the assignment... j The total workload of the intelligent agent must meet the workload requirements of the task, where y j Indicates the task allocation status, y j =1 indicates that it has been allocated, y j =0 indicates no allocation. In formula (b), agent A... i The actual load it undertakes cannot exceed its maximum load capacity. In formula (c), the loads assigned to the same task T... j The network enables connectivity between intelligent agents.

[0133] By solving the task allocation optimization model, the optimal task allocation scheme is obtained, and all agents move to their assigned task locations. If, during the movement, the distance between agents exceeds the minimum safe distance for the first time... The agent that is closer to the remaining agent will move one time step to avoid a collision.

[0134] On the Internet After the task is completed, the network formed by idle agents (without load consumption) will generate a new network number to continue exploring the task. At this time, the agent with the highest network betweenness centrality will be selected as the leader.

[0135] Training of the S8 subnetwork has ended.

[0136] Since the subnetwork no longer exists after being split, specifically, the network represented by that subnetwork number no longer exists, the training of that subnetwork ends, and its model will continue to be used and learned in a new subnetwork.

[0137] S9. When the specified training time is reached, terminate the model training of all sub-networks in the environment and perform model parameter synchronization.

[0138] Repeating the above steps until the specified time (the termination condition of the high-level policy) is reached will end the training process of all sub-networks in the environment and synchronize the model parameters of different sub-networks.

Claims

1. A multi-agent task allocation method based on network coverage dynamic matching control, characterized in that, Includes the following steps: Multi-agent self-organizes to form a network and is divided into N sub-networks. Each sub-network adopts a leader-follower model, and the leader performs sub-network management and decision-making. Each subnetwork acquires the current environmental state and uses the high-level decision-making of the hierarchical deep reinforcement learning model to choose to explore the task domain or find other subnetworks. The agent moves by making low-level decisions using a hierarchical deep reinforcement learning model until it finds the task domain or other subnetworks. Network topology control is used to segment or merge subnetworks of the identified task domain or other subnetworks to form overlay subnetworks or new subnetworks. Agents in the overlay subnetwork use a task allocation optimization model to complete task execution, and after the task is completed, they are transferred to a new subnetwork. The task allocation optimization model is as follows: in This represents the total number of agents involved in task allocation. J represents the total number of tasks in the task domain, and J represents the objective function / total system cost. Indicates task Whether to be assigned to an intelligent agent , This indicates a successful allocation. This indicates that the allocation was unsuccessful. LCmax represents the load requirement of task j; LCmax represents the maximum load capacity of agent i. Represents intelligent agents To the mission The distance; This represents the connectivity parameter between agent i and task j; The parameter represents the connectivity between agent i and task j; in the formula In the process, participate in the allocation of tasks. The total workload of the intelligent agent must meet the workload requirements of the task, among which Indicates the task allocation status. Indicates that it has been allocated. Indicates unallocated; in the formula In the middle, intelligent agents The actual load it bears cannot exceed its maximum load capacity; in the formula In the same task The network enables connectivity between intelligent agents.

2. The multi-agent task allocation method based on network coverage dynamic matching control according to claim 1, characterized in that: In the navigator and follower model, the navigator is the agent with the highest centrality of the subnet's mediation, responsible for collecting and processing the task and location information of the followers.

3. The multi-agent task allocation method based on network coverage dynamic matching control according to claim 1, characterized in that: Through sub-network load utilization in hierarchical deep reinforcement learning models To measure the amount of task information obtained after high-level decision-making, the aforementioned in Indicates in subnetwork The task distribution set obtained in the process, Indicates task The load requirements, Represents intelligent agents Maximum load capacity.

4. The multi-agent task allocation method based on network coverage dynamic matching control according to claim 1, characterized in that: When exploring the task domain, lower-level decision-making is achieved through exploration utility. The decision on the mobile reward for the sub-network, the in, Let be the task distribution density at time step t. The urgency of the task is defined as time step t.

5. The multi-agent task allocation method based on network coverage dynamic matching control according to claim 4, Its features The task distribution density in Indicates at time step Subnet of time, Indicates at time step No. A set of tasks within the detection range of an intelligent agent. This indicates the number of tasks in the set. Represents pi (π). Indicates the first The detection radius of each agent; the task urgency in Indicates at time step The set of tasks detected by the subnet. This indicates the number of tasks in the set. Indicates task The end time, Indicates task The time of its creation.

6. The multi-agent task allocation method based on network coverage dynamic matching control according to claim 1, Its features are: The optimal task domain is selected based on task domain priority. in and The calculation rules are as follows: in Indicating subnet The position, and Indicates the geometric center of the task domain; in Indicates task The end time, and This indicates the number of tasks in the task domain. t represents the task domain and t represents the current time step.

7. The multi-agent task allocation method based on network coverage dynamic matching control according to claim 1, characterized in that: When searching for other subnetworks, lower-level decisions are made through consensus mechanisms. The decision on the mobile reward for the sub-network, the in, Let be the agent distribution density at time step t. The information update interval is defined as time step t.

8. The multi-agent task allocation method based on network coverage dynamic matching control according to claim 7, characterized in that: The distribution density of the intelligent agents in Indicates at time step Subnetwork of time, Indicates at time step No. A set of agents within the detection range of an agent. This indicates the number of agents in the set. Represents pi (π). Indicates the first The detection radius of each agent, the information update interval in This indicates the time step in which the information exchange was last completed. Indicates the current time step.

9. The multi-agent task allocation method based on network coverage dynamic matching control according to claim 1, characterized in that: By prioritizing the selection of agents The selection of segmentation agents is based on the load capacity requirements of the task domain, and the selection priority is... in For the task domain geometric center With the target location The connection, Represented as a task domain geometric center With agent location The connection, It represents the cosine of the angle formed by the two lines. After determining the selection priority, based on the task domain Number of agents required , in Represents the task domain Tasks in , Represents intelligent agents Maximum load capacity The load requirement for task j.

Images