A multi-agent distributed path search method, system and computer device

Abstract

This invention discloses a multi-agent distributed path-finding method, system, and computer device. First, sensors are used to acquire information about the small-scale surrounding environment of each agent. Path planning is achieved using heuristic search, calculating static costs (based on the grid map topology) and dynamic costs (based on the updateable observations of individual agents) in a grid map, and selecting low-cost paths. Neural networks are used to generate strategies, and effective training can be performed without setting reward parameters or additional planning, effectively improving the method's applicability. A distributed approach is used to control individual agents, meaning that each agent's action selection plan is unique. Finally, after action decoding, the action distribution of each agent is output, thereby enabling the generation of obstacle avoidance maneuvers. This method overcomes the inefficiency of traditional methods for finding a set of conflict-free paths and exhibits improved generalization and robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-agent path planning, and in particular relates to a multi-agent distributed path search method, system and computer equipment. Background Art

[0002] Multi-agent path planning (MAPF) is a research problem that has attracted much attention in the field of artificial intelligence in recent years and is widely used in practical scenarios such as robot navigation, warehouse automation, and traffic control. In the MAPF problem, multiple agents are constrained to a graph structure, and each agent can only move to adjacent vertices or remain in place at each time step. The traditional MAPF setting usually relies on a central controller that has global information about the graph structure and the starting and goal positions of all agents, and is responsible for generating a set of conflict-free paths for all agents. This centralized and fully observable assumption makes the problem more controllable in theory. However, in many real-world application scenarios, it is impractical to rely on a central controller and obtain complete information about the real-time status and environment of all agents.

[0003] For example, in dynamic and complex human-machine coexistence environments such as restaurants and hospitals, when service robots perform food and medicine delivery tasks, although they may have access to a global map (such as a building's floor plan), the central controller may not always maintain a stable connection with all robots, and therefore cannot obtain the precise position of each robot in real time. In addition, obstacles such as pedestrians are dynamic and the environment is complex, which makes centralized path planning solutions ineffective. In these cases, distributed MAPF methods are particularly important. Each robot needs to make autonomous decisions based on its local perception rather than relying on global control. This allows the MAPF problem to be redefined as a sequential decision problem: each agent must choose an action at each time step, both to move towards the goal and to ensure that other agents can reach their goals smoothly and avoid conflicts.

[0004] When solving distributed MAPF problems, learnable methods such as reinforcement learning (RL) are widely used due to their adaptability in environments with incomplete information. However, existing RL methods perform poorly on long-duration tasks with complex causal structures, especially in the presence of hard combinatorial constraints, and they often struggle to compete with search-based planning methods.

[0005] To address this problem, the present invention proposes a hybrid method that combines a heuristic search algorithm and a learning strategy to avoid solving the MAPF problem directly through reinforcement learning. Summary of the Invention

[0006] In response to the above technical problems, the present invention provides a multi-agent distributed path search method, system and computer device.

[0007] The technical solution adopted by the present invention to solve the technical problem is:

[0008] A multi-agent distributed path search method, the method comprising the following steps:

[0009] S100: Acquire environmental information within a preset range around each intelligent agent through sensors;

[0010] S200: Obtain the starting point and the target point, plan the path of each agent using a heuristic path planning algorithm, and calculate the static cost from the current grid point to the next grid point in the grid map based on the grid map topology;

[0011] S300: Calculate the updateable observation value of each agent as a data set based on the observation function, calculate the dynamic cost from the current grid point to the next grid point based on the updateable observation value of each agent, obtain the cost from the current grid point to the next grid point based on the static cost and the dynamic cost, and select the path with the lowest cost;

[0012] S400: Build a neural network based on the critic-actor model. Input the updateable observations of the agents in the training set into the neural network for shared feature extraction and evaluation. The actor path generates an action distribution, and the critic path generates a value distribution. The shared strategy is continuously optimized through the proximal strategy optimization algorithm to complete the training of the neural network and achieve strategy generation.

[0013] S500: Input the updateable observation values ​​of the agents in the test set into the trained neural network, and finally output the action distribution of a single agent after action decoding, thereby realizing the generation of obstacle avoidance actions.

[0014] Preferably, S200 includes:

[0015] S210: performing heuristic path planning based on the A* algorithm, that is, calculating the sum of the actual cost of moving from the starting point along the generated path to the specified grid and the estimated cost from the current grid point to the target point in the grid map, and searching for the minimum value to obtain the optimal path;

[0016] S220: The cost of a single agent moving from one cell to another is composed of the sum of static cost and dynamic cost. The static cost depends entirely on the topology of the grid map. The breadth-first algorithm is used to calculate the average cost of the path from the current cell to other free cells, and the static cost is obtained based on the average cost.

[0017] S230: Calculate the observation value of the agent based on the observation function, calculate the dynamic cost of the cell based on the observation value of the agent, and update it in real time. Whenever the agent reaches the target, the dynamic cost of each cell of the path corresponding to the agent will be cleared and recalculated.

[0018] Preferably, S210 specifically includes:

[0019] ;

[0020] in, represents the cost of cell transfer, Indicates the grid points passed through, and Represent the starting point and the target point respectively. Indicates from arrive The total cost, Indicates the starting point To the passing grid point The actual cost, Represents the estimated cost from the current grid point to the target point.

[0021] Preferably, the cost of a single agent transferring from one cell to another in S220 is composed of the sum of a static cost and a dynamic cost, specifically:

[0022] ;

[0023] in, represents the static cost, represents dynamic cost, and t represents time.

[0024] Preferably, in S220, a breadth-first algorithm is used to calculate the average cost of the path from the current cell to other free cells, and a static cost is obtained based on the average cost, specifically:

[0025] ;

[0026] in, represents the average cost, Indicates from The vertex that can be reached by starting;

[0027] .

[0028] Preferably, in S230, the dynamic cost of the cell is calculated based on the observation value of the agent, specifically:

[0029] ;

[0030] in, is an observation function, i.e., at the time step Within the observation range of the current agent, each time another agent is observed, 1 is returned; otherwise, 0 is returned.

[0031] Preferably, S300 includes:

[0032] S310: Inputting the local observation value of the agent into the neural network, wherein the local observation value is composed of a 2*n*n tensor, where n is the local observation range;

[0033] S320: The spatial encoder extracts shared features from the input and generates high-dimensional feature representations. These features are divided into two paths: the actor path and the critic path. In the actor path, the bidirectional gated recurrent unit (BiGRU) is used to memorize the observation history and capture long-term dependencies in the time series. After optimizing the extracted shared features, the action decoder outputs the agent's action probability distribution for decision-making. The critic path uses the value function network and state generator to evaluate the value of the current state, that is, the expected future reward, for training.

[0034] S330: Each agent uses the same strategy and continuously iterates and optimizes the shared strategy through the proximal policy optimization algorithm. The goal of the strategy is to maximize the cumulative reward. When the agent reaches the target point, the system will give a reward. During the movement, if the agent deviates from the target point or approaches the target point, the system will restart the heuristic path planner and replan the path. In order to avoid congestion with other agents, the agent can accept a detour to ensure that the most agents move towards the target point, thereby maximizing the cumulative reward.

[0035] Preferably, in S330, the sharing strategy is continuously optimized iteratively through a proximal strategy optimization algorithm, specifically:

[0036] The state value of the current strategy is sampled, and the corresponding advantage function is calculated using the differential method or the generalized advantage estimation method. The update ratio of the new and old strategies is calculated, and the strategy is updated by minimizing the expected value in the clipping objective function while ensuring that the strategy update ratio is within a limited range. The sampling and strategy update are repeated until the strategy converges or the set number of iterations is reached.

[0037] A multi-agent distributed path search system includes an environment information acquisition module, a path planning module, a path selection module, a neural network training module, and an agent action generation module.

[0038] The environmental information acquisition module is used to obtain environmental information within a preset range around each intelligent agent through sensors;

[0039] The path planning module is used to obtain the starting point and the target point, plan the path of each agent using the heuristic path planning algorithm, and calculate the static cost from the current grid point to the next grid point in the grid map based on the grid map topology;

[0040] The path selection module is used to calculate the updateable observation value of each agent as a data set based on the observation function, calculate the dynamic cost from the current grid point to the next grid point based on the updateable observation value of each agent, obtain the cost from the current grid point to the next grid point based on the static cost and dynamic cost, and select the path with the lowest cost;

[0041] The neural network training module is used to build a neural network based on the critic-actor model. The updateable observation values ​​of the agents in the training set are input into the neural network for shared feature extraction and evaluation. The actor path generates action distribution, and the critic path generates value distribution. The shared strategy is continuously optimized through the proximal strategy optimization algorithm to complete the training of the neural network and realize strategy generation.

[0042] The agent action generation module is used to input the updateable observation values ​​of the agents in the test set into the trained neural network, and finally output the action distribution of a single agent after action decoding, thereby realizing the generation of obstacle avoidance actions.

[0043] A computer device includes a memory and a processor, wherein the memory stores a computer program and the processor implements the steps of a multi-agent distributed path search method when executing the computer program.

[0044] The aforementioned multi-agent distributed path-finding method, system, and computer device are designed to solve the multi-agent path planning (MAPF) problem. By first using a heuristic search to generate a target path for each agent and simultaneously introducing deadlock avoidance techniques to penalize paths that could potentially conflict, this effectively improves the rationality and distribution of path planning. Each agent then autonomously follows the predetermined path using a decentralized learning strategy, flexibly detouring when necessary to avoid conflicts and complete the task. This approach not only improves robustness in dynamic and incomplete information environments but also overcomes the shortcomings of pure reinforcement learning in complex tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Flowchart of a multi-agent distributed path search method according to one embodiment of the present invention;

[0046] Figure 2 This is a technical roadmap for the specific implementation process of a multi-agent distributed path search method in one embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below with reference to the accompanying drawings.

[0048] In one embodiment, Figure 1 As shown, a multi-agent distributed path search method comprises the following steps:

[0049] S100: Acquire environmental information within a preset range around each intelligent agent through sensors;

[0050] S200: Obtain the starting point and the target point, plan the path of each agent using a heuristic path planning algorithm, and calculate the static cost from the current grid point to the next grid point in the grid map based on the grid map topology;

[0051] S300: Calculate the updateable observation value of each agent as a data set based on the observation function, calculate the dynamic cost from the current grid point to the next grid point based on the updateable observation value of each agent, obtain the cost from the current grid point to the next grid point based on the static cost and the dynamic cost, and select the path with the lowest cost;

[0052] S400: Build a neural network based on the critic-actor model. Input the updateable observations of the agents in the training set into the neural network for shared feature extraction and evaluation. The actor path generates an action distribution, and the critic path generates a value distribution. The shared strategy is continuously optimized through the proximal strategy optimization algorithm to complete the training of the neural network and achieve strategy generation.

[0053] S500: Input the updateable observation values ​​of the agents in the test set into the trained neural network, and finally output the action distribution of a single agent after action decoding, thereby realizing the generation of obstacle avoidance actions.

[0054] Specifically, when obtaining the environmental information of each agent through sensors in S100, it is not necessary to obtain the global map, but only the small-scale environmental information around the single agent is required. Figure 2 As shown in the example in the figure, the current position of the agent is (7, 3) and the target position is (2, 6). In the small figure of environmental obstacles and planned paths, the + on the left represents the planned path and the - represents static obstacles. The + in the small figure of local agent retrieval and update represents other agents (agents) around. Figure 2 The agent observes dynamic and static obstacles through sensors and provides cost information. The heuristic path planner plans the path, receives observation values ​​and performs training in the reinforcement learning network, and finally generates a more reasonable strategy to guide the agent's actions.

[0055] In one embodiment, S200 includes:

[0056] S210: performing heuristic path planning based on the A* algorithm, that is, calculating the sum of the actual cost of moving from the starting point along the generated path to the specified grid and the estimated cost from the current grid point to the target point in the grid map, and searching for the minimum value to obtain the optimal path;

[0057] S220: The cost of a single agent moving from one cell to another is composed of the sum of static cost and dynamic cost. The static cost depends entirely on the topology of the grid map. The breadth-first algorithm is used to calculate the average cost of the path from the current cell to other free cells, and the static cost is obtained based on the average cost.

[0058] S230: Calculate the observation value of the agent based on the observation function, calculate the dynamic cost of the cell based on the observation value of the agent, and update it in real time. Whenever the agent reaches the target, the dynamic cost of each cell of the path corresponding to the agent will be cleared and recalculated.

[0059] In one embodiment, S210 specifically includes:

[0060] ;

[0061] in, represents the cost of cell transfer, Indicates the grid points passed through, and Represent the starting point and the target point respectively. Indicates from arrive The total cost, Indicates the starting point To the passing grid point The actual cost, Represents the estimated cost from the current grid point to the target point.

[0062] In one embodiment, the cost of a single agent transferring from one cell to another in S220 is composed of the sum of a static cost and a dynamic cost, specifically:

[0063] ;

[0064] in, represents the static cost, represents dynamic cost, and t represents time.

[0065] Specifically, the static cost depends entirely on the topology of the grid map and does not change with the positions of other agents or the Euclidean distance between them.

[0066] In one embodiment, in S220, a breadth-first algorithm is used to calculate the average cost of the path from the current cell to other free cells, and a static cost is obtained based on the average cost, specifically:

[0067] ;

[0068] in, represents the average cost, Indicates from The vertex that can be reached by starting;

[0069] Specifically, The lower the value, the more paths there are. , which will cause local congestion, so Continuous transition should bear higher costs, which can be expressed as follows:

[0070] .

[0071] Specifically, from the above formula, we can see that only when the usage frequency of a cell is low enough will it have a more ideal transition cost.

[0072] In one embodiment, the dynamic cost of the cell is calculated based on the observation value of the agent in S230, specifically:

[0073] ;

[0074] in, is an observation function, i.e., at the time step Within the observation range of the current agent, each time another agent is observed, 1 is returned; otherwise, 0 is returned.

[0075] Specifically, the dynamic cost is based on the agent's observations and is updated in real time. This dynamic cost prevents the agent from transitioning to cells that are frequently visited by other agents. To prevent excessive dynamic cost accumulation, the dynamic cost of each cell is cleared and recalculated each time the agent reaches its destination.

[0076] In one embodiment, S300 includes:

[0077] S310: Inputting the local observation value of the agent into the neural network, wherein the local observation value is composed of a 2*n*n tensor, where n is the local observation range;

[0078] S320: The spatial encoder extracts shared features from the input and generates high-dimensional feature representations. These features are divided into two paths: the actor path and the critic path. In the actor path, the bidirectional gated recurrent unit (BiGRU) is used to memorize the observation history and capture long-term dependencies in the time series. After optimizing the extracted shared features, the action decoder outputs the agent's action probability distribution for decision-making. The critic path uses the value function network and state generator to evaluate the value of the current state, that is, the expected future reward, for training.

[0079] S330: Each agent uses the same strategy and continuously iterates and optimizes the shared strategy through the proximal policy optimization algorithm. The goal of the strategy is to maximize the cumulative reward. When the agent reaches the target point, the system will give a reward. During the movement, if the agent deviates from the target point or approaches the target point, the system will restart the heuristic path planner and replan the path. In order to avoid congestion with other agents, the agent can accept a detour to ensure that the most agents move towards the target point, thereby maximizing the cumulative reward.

[0080] Specifically, this method maximizes reward by maximizing the number of global goal points reached. This takes into account the long-term accumulation of rewards, allowing other agents to pass without hindering each other's movement. Furthermore, the goal of training is to optimize shared strategies, with each agent using the same strategy to maximize cumulative reward.

[0081] In one embodiment, in S330, the sharing strategy is continuously optimized iteratively through a proximal strategy optimization algorithm, specifically:

[0082] The state value of the current strategy is sampled, and the corresponding advantage function is calculated using the differential method or the generalized advantage estimation method. The update ratio of the new and old strategies is calculated, and the strategy is updated by minimizing the expected value in the clipping objective function while ensuring that the strategy update ratio is within a limited range. The sampling and strategy update are repeated until the strategy converges or the set number of iterations is reached.

[0083] Specifically, the strategy is generated by the critic, and rewards are provided to motivate the actor to continuously learn towards a more ideal state. The strategy generation process can be regarded as a partially observable Markov decision process. The action is generated by the actor, and after action decoding, the mobile platform generates and executes the corresponding action instructions, while being continuously guided by the critic.

[0084] The above-mentioned multi-agent distributed path search method combines a heuristic search algorithm and a learning strategy to avoid directly solving the MAPF problem through reinforcement learning. Specifically, each agent first plans a path to the target through a heuristic search algorithm, without considering the existence of other agents. At the same time, a technology specifically used to avoid deadlock is introduced to penalize paths that may cause conflicts, so that the agents are better distributed in the workspace. After the path is generated, the planned path is followed through a decentralized learning strategy, and appropriate detours are made when necessary to avoid conflicts with other agents and ensure that they successfully complete the task. The present invention overcomes the problem of inefficiency in finding a set of conflict-free paths in traditional multi-agent distributed path search, and has better generalization ability and better robustness.

[0085] A multi-agent distributed path search system includes an environment information acquisition module, a path planning module, a path selection module, a neural network training module, and an agent action generation module.

[0086] The environmental information acquisition module is used to obtain environmental information within a preset range around each intelligent agent through sensors;

[0087] The path planning module is used to obtain the starting point and the target point, plan the path of each agent using the heuristic path planning algorithm, and calculate the static cost from the current grid point to the next grid point in the grid map based on the grid map topology;

[0088] The path selection module is used to calculate the updateable observation value of each agent as a data set based on the observation function, calculate the dynamic cost from the current grid point to the next grid point based on the updateable observation value of each agent, obtain the cost from the current grid point to the next grid point based on the static cost and dynamic cost, and select the path with the lowest cost;

[0089] The neural network training module is used to build a neural network based on the critic-actor model. The updateable observation values ​​of the agents in the training set are input into the neural network for shared feature extraction and evaluation. The actor path generates action distribution, and the critic path generates value distribution. The shared strategy is continuously optimized through the proximal strategy optimization algorithm to complete the training of the neural network and realize strategy generation.

[0090] The agent action generation module is used to input the updateable observation values ​​of the agents in the test set into the trained neural network, and finally output the action distribution of a single agent after action decoding, thereby realizing the generation of obstacle avoidance actions.

[0091] The specific limitations of the multi-agent distributed path-finding system can be found in the limitations of the multi-agent distributed path-finding method described above and will not be further elaborated here. Each module in the multi-agent distributed path-finding system described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the modules described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so that the processor can call and execute the operations corresponding to each of the modules described above.

[0092] A computer device includes a memory and a processor, wherein the memory stores a computer program and the processor implements the steps of a multi-agent distributed path search method when executing the computer program.

[0093] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0094] The above is a detailed introduction to the multi-agent distributed path search method, system and computer device provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A multi-agent distributed path search method, characterized in that: The method comprises the following steps: S100: Acquire environmental information within a preset range around each intelligent agent through sensors; S200: Obtain the starting point and the target point, plan the path of each agent using a heuristic path planning algorithm, and calculate the static cost from the current grid point to the next grid point in the grid map based on the grid map topology; S300: Calculate the updateable observation value of each agent as a data set based on the observation function, calculate the dynamic cost from the current grid point to the next grid point based on the updateable observation value of each agent, obtain the cost from the current grid point to the next grid point based on the static cost and the dynamic cost, and select the path with the lowest cost; S400: Build a neural network based on the critic-actor model. Input the updateable observations of the agents in the training set into the neural network for shared feature extraction and evaluation. The actor path generates an action distribution, and the critic path generates a value distribution. The shared strategy is continuously optimized through the proximal strategy optimization algorithm to complete the training of the neural network and achieve strategy generation. S500: Input the updateable observation values ​​of the agents in the test set into the trained neural network, and finally output the action distribution of a single agent after action decoding, thereby realizing the generation of obstacle avoidance actions.

2. The method according to claim 1, characterized in that S200 includes: S210: performing heuristic path planning based on the A* algorithm, that is, calculating the sum of the actual cost of moving from the starting point along the generated path to the specified grid and the estimated cost from the current grid point to the target point in the grid map, and searching for the minimum value to obtain the optimal path; S220: The cost of a single agent moving from one cell to another is composed of the sum of static cost and dynamic cost. The static cost depends entirely on the topology of the grid map. The breadth-first algorithm is used to calculate the average cost of the path from the current cell to other free cells, and the static cost is obtained based on the average cost. S230: Calculate the observation value of the agent based on the observation function, calculate the dynamic cost of the cell based on the observation value of the agent, and update it in real time. Whenever the agent reaches the target, the dynamic cost of each cell of the path corresponding to the agent will be cleared and recalculated.

3. The method according to claim 2, characterized in that S210 is specifically: ； in, represents the cost of cell transfer, Indicates the grid points passed through, and Represent the starting point and the target point respectively. Indicates from arrive The total cost, Indicates the starting point To the passing grid point The actual cost, Represents the estimated cost from the current grid point to the target point.

4. The method according to claim 3, characterized in that The cost of a single agent moving from one cell to another in S220 is composed of the sum of static cost and dynamic cost, specifically: ； in, represents the static cost, represents dynamic cost, and t represents time.

5. The method according to claim 4, characterized in that In S220, the breadth-first algorithm is used to calculate the average cost of the path from the current cell to other free cells, and the static cost is obtained based on the average cost, which is specifically: ； in, represents the average cost, Indicates from The vertex that can be reached by starting; 。 6. The method according to claim 5, characterized in that In S230, the dynamic cost of the cell is calculated based on the observation value of the agent, specifically: ； in, is an observation function, i.e., at the time step Within the observation range of the current agent, each time another agent is observed, 1 is returned; otherwise, 0 is returned.

7. The method according to claim 6, characterized in that S300 includes: S310: Inputting the local observation value of the agent into the neural network, wherein the local observation value is composed of a 2*n*n tensor, where n is the local observation range; S320: The spatial encoder extracts shared features from the input and generates high-dimensional feature representations. These features are divided into two paths: the actor path and the critic path. In the actor path, the bidirectional gated recurrent unit (BiGRU) is used to memorize the observation history and capture long-term dependencies in the time series. After optimizing the extracted shared features, the action decoder outputs the agent's action probability distribution for decision-making. The critic path uses the value function network and state generator to evaluate the value of the current state, that is, the expected future reward, for training. S330: Each agent uses the same strategy and continuously iterates and optimizes the shared strategy through the proximal policy optimization algorithm. The goal of the strategy is to maximize the cumulative reward. When the agent reaches the target point, the system will give a reward. During the movement, if the agent deviates from the target point or approaches the target point, the system will restart the heuristic path planner and replan the path. In order to avoid congestion with other agents, the agent can accept a detour to ensure that the most agents move towards the target point, thereby maximizing the cumulative reward.

8. The method according to claim 7, characterized in that S330 continuously iterates and optimizes the sharing strategy through the proximal strategy optimization algorithm, specifically: The state value of the current strategy is sampled, and the corresponding advantage function is calculated using the differential method or the generalized advantage estimation method. The update ratio of the new and old strategies is calculated, and the strategy is updated by minimizing the expected value in the clipping objective function while ensuring that the strategy update ratio is within a limited range. The sampling and strategy update are repeated until the strategy converges or the set number of iterations is reached.

9. A multi-agent distributed path search system, characterized in that: It includes environment information acquisition module, path planning module, path selection module, neural network training module and intelligent action generation module. The environmental information acquisition module is used to obtain environmental information within a preset range around each intelligent agent through sensors; The path planning module is used to obtain the starting point and the target point, plan the path of each agent using the heuristic path planning algorithm, and calculate the static cost from the current grid point to the next grid point in the grid map based on the grid map topology; The path selection module is used to calculate the updateable observation value of each agent as a data set based on the observation function, calculate the dynamic cost from the current grid point to the next grid point based on the updateable observation value of each agent, obtain the cost from the current grid point to the next grid point based on the static cost and dynamic cost, and select the path with the lowest cost; The neural network training module is used to build a neural network based on the critic-actor model. The updateable observation values ​​of the agents in the training set are input into the neural network for shared feature extraction and evaluation. The actor path generates action distribution, and the critic path generates value distribution. The shared strategy is continuously optimized through the proximal strategy optimization algorithm to complete the training of the neural network and realize strategy generation. The agent action generation module is used to input the updateable observation values ​​of the agents in the test set into the trained neural network, and finally output the action distribution of a single agent after action decoding, thereby realizing the generation of obstacle avoidance actions.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images