A multi-agent system integrating graph neural network and DH-Bug algorithm

Abstract

The present application relates to the technical field of multi-agent collaborative planning, and in particular to a multi-agent that integrates a graph neural network and a DH-Bug algorithm, wherein the method comprises: obtaining a first perception image of a target agent and a second perception image of at least one other agent around the target agent; fusing the first perception image with the second perception image of at least one other agent to obtain a fused image; inputting the fused image into a preset multi-layer perceptron, outputting the predicted direction of the next movement of the target agent, and generating a planned path based on the current position, target point position and predicted direction of the target agent, and controlling the target agent to perform the target action along the planned path. Thus, the problem of how to realize multi-agent collaborative path planning in a real application scenario taking into account physical constraints and when the information is partially known is solved.

Description

Technical Field

[0001] The present application relates to the technical field of multi-agent collaborative planning, and in particular to a multi-agent system that integrates graph neural networks and DH-Bug algorithms. Background Art

[0002] As people's desire for an intelligent world becomes stronger, traditional single-agent systems can no longer meet application needs. The multi-agent system's advantages in information sharing, comprehensive decision-making, functional coordination, etc. enable it to achieve preset goals with higher execution efficiency, wider coverage and better completion results when facing a variety of civilian applications and military tasks.

[0003] At present, multi-agent systems have been deeply applied to many fields. For example, in the military field, drone clusters cooperate in reconnaissance and attack, multiple warhead missiles jointly penetrate and guide, unmanned combat vehicles cooperate in combat and mine clearance, etc.; in the field of people's livelihood, logistics warehousing cooperative sorting, emergency rescue joint search and rescue, industrial production collaborative control, etc.

[0004] Collaborative path planning is the core support link for multi-agent systems to achieve various group capabilities such as information coverage, material transfer, and functional interaction, and has important research significance. Especially for real application scenarios where environmental information is not completely known, the information of each agent is incomplete, the communication range is limited, and collaborative planning is difficult. Summary of the invention

[0005] The present application provides a multi-agent collaborative path planning method, device, agent and storage medium to solve the problem of how to achieve multi-agent collaborative path planning in a real application scenario taking physical constraints into account when partial information is known.

[0006] The first aspect of the present application provides a collaborative path planning method for multiple intelligent agents, comprising the following steps: acquiring a first perception image of a target intelligent agent and a second perception image of at least one other intelligent agent around the target intelligent agent; fusing the first perception image and the second perception image of the at least one other intelligent agent to obtain a fused image; inputting the fused image into a preset multi-layer perceptron, outputting a predicted direction of the next movement of the target intelligent agent, generating a planned path based on the current position of the target intelligent agent, the target point position and the predicted direction, and controlling the target intelligent agent to perform the target action along the planned path.

[0007] Optionally, before generating a planned path according to the current position, target point position and predicted direction of the target intelligent body, it includes: detecting whether the area corresponding to the predicted direction is passable; if the area corresponding to the predicted direction is passable, generating a planned path according to the current position, target point position and predicted direction of the target intelligent body; if the area corresponding to the predicted direction is not passable, identifying the first perceived image to obtain the perceived information around the target intelligent body, and using a preset algorithm to calculate the actual direction of the next movement of the target intelligent body, and generating a planned path according to the current position, target point position and actual direction of the target intelligent body, and controlling the target intelligent body to perform the target action along the planned path.

[0008] Optionally, the fusing of the first perceived image and the second perceived image of the at least one other intelligent agent to obtain a fused image includes: inputting the first perceived image and the second perceived image of the at least one other intelligent agent into a preset neural network, outputting a first image feature of the first perceived image and a second image feature of each of the second perceived images; and fusing the first image feature with each of the second image features to obtain the fused image.

[0009] Optionally, before fusing the first perception image and at least one of the second perception images, the method includes: identifying the first perception image; when static obstacles and / or other intelligent entities around the target intelligent entity are identified, generating a first channel image according to the positions of the static obstacles, and generating a second channel image according to the actual states of the other intelligent entities; generating a third channel image according to the position of the target point or the projection of the target point on the boundary of the perception range of the target intelligent entity; and matrixing the first channel image, the second channel image and the third channel image to obtain a multi-channel matrixed perception image.

[0010] Optionally, the target action includes any one of a target movement action, an on-the-spot turning action, a movement action around a convex obstacle, a direction change action around an obstacle, and a dynamic obstacle avoidance action.

[0011] The second aspect of the present application provides a multi-agent collaborative path planning device, including: an acquisition module, used to acquire a first perception image of a target intelligent agent and a second perception image of at least one other intelligent agent around the target intelligent agent; a fusion module, used to fuse the first perception image and the second perception image of the at least one other intelligent agent to obtain a fused image; a planning module, used to input the fused image into a preset multi-layer perceptron, output the predicted direction of the next movement of the target intelligent agent, generate a planned path according to the current position of the target intelligent agent, the target point position and the predicted direction, and control the target intelligent agent to perform the target action along the planned path.

[0012] Optionally, the planning module is also used to, before generating a planned path based on the current position, target point position and predicted direction of the target intelligent body: detect whether the area corresponding to the predicted direction is passable; if the area corresponding to the predicted direction is passable, generate a planned path based on the current position, target point position and predicted direction of the target intelligent body; if the area corresponding to the predicted direction is not passable, identify the first perceived image to obtain the perceived information around the target intelligent body, and use a preset algorithm to calculate the actual direction of the next movement of the target intelligent body, and generate a planned path based on the current position, target point position and actual direction of the target intelligent body, and control the target intelligent body to perform the target action along the planned path.

[0013] Optionally, it also includes: a feature extraction module, used to input the first perceived image and the second perceived image of at least one other intelligent agent into a preset neural network, and output the first image feature of the first perceived image and the second image feature of each of the second perceived images; the fusion module, used to fuse the first image feature and each of the second image features to obtain the fused image after fusion.

[0014] Optionally, it also includes: an information processing module, which is used to identify the first perception image before fusing the first perception image and at least one of the second perception images; when static obstacles and / or other intelligent entities around the target intelligent entity are identified, generate a first channel image according to the position of the static obstacles, and generate a second channel image according to the actual state of the other intelligent entities; generate a third channel image according to the position of the target point or the projection of the target point on the boundary of the perception range of the target intelligent entity; matrix the first channel image, the second channel image and the third channel image to obtain a multi-channel matrixed perception image.

[0015] Optionally, the target action includes any one of a target movement action, an on-the-spot turning action, a movement action around a convex obstacle, a direction change action around an obstacle, and a dynamic obstacle avoidance action.

[0016] The third aspect of the present application provides an intelligent agent, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the collaborative path planning method for multiple agents as described in the above embodiment.

[0017] The fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the multi-agent collaborative path planning method as described in the above embodiment.

[0018] Therefore, this application has at least the following beneficial effects:

[0019] Based on the agent's perception information and the shared information of other agents, the regional direction prediction based on the fan distribution is mapped out, and the specific movement direction of the agent is calculated, so as to realize multi-agent path planning based on the fusion of data and knowledge. In this way, multi-agent path planning can be realized under the conditions of limited agent perception and communication range, no global positioning and complex physical constraint scenarios, which can not only ensure the strict convergence of the algorithm, but also fully explore the task relevance between agents, and improve the coordination and planning performance of the system. In this way, the technical problems such as how to realize multi-agent collaborative path planning in real application scenarios taking into account physical constraints and when the information is partially known are solved.

[0020] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A schematic diagram of the actual perception range of an intelligent agent provided according to an embodiment of the present application;

[0023] Figure 2 A flowchart of a schematic diagram of a multi-agent collaborative path planning method according to an embodiment of the present application;

[0024] Figure 3 A schematic diagram of an obstacle channel perception information processing process provided according to an embodiment of the present application;

[0025] Figure 4A fan diagram of the direction prediction of an intelligent agent provided according to an embodiment of the present application;

[0026] Figure 5 A framework model integrating data and knowledge provided according to an embodiment of the present application;

[0027] Figure 6 A regular maze map provided according to an embodiment of the present application;

[0028] Figure 7 An irregular environment map provided according to an embodiment of the present application;

[0029] Figure 8 This is an example diagram of a multi-agent collaborative path planning device provided according to an embodiment of the present application;

[0030] Fig. 9 A schematic diagram of the structure of an intelligent agent provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0032] The goal of multi-agent collaborative planning is to find a safe path from each starting point to each target point in a two-dimensional task space.

[0033] The simulation environment of the embodiment of the present application is a two-dimensional task space, where W and H are the width and height of the two-dimensional task space respectively. The task space contains N agents V = {v1, ..., v N} and the corresponding N target points G={G1,...,G N}, and there are several static obstacles of arbitrary shapes scattered randomly. The agent does not have a global positioning device, but only has local perception ability. The perception direction of the agent is the forward movement direction, such as Figure 1 As shown, the perception angle is 180° forward and the maximum perception distance is r ob When obstacles and other agents are sensed, only their boundary information can be detected. The communication radius of each agent is r com , when the agent v i and agent v j The relative position is less than r com When ||p i -p j ||≤r com , then the agent v iand agent v j can communicate with each other, otherwise, they cannot communicate, where p i 、p j They are respectively i and v j Location information. Definition is the communication network diagram of multiple agents at time t, where ε t A collection of edges representing agents that can communicate with each other.

[0034] To this end, the embodiment of the present application proposes an algorithm that combines a composite neural network method based on TOKF-GraphSAGE and a DH-Bug algorithm. Among them, the composite neural network based on TOKF-GraphSAGE consists of a feature extraction module CNN, an information aggregation module TOKF-GraphSAGE, and a direction prediction module MLP.

[0035] It should be noted that the principle of the prediction action generation process of N agents is the same. In the embodiment of the present application, the prediction action generation process of agent v is used as the example. i As an example, the prediction action generation process at time t is specifically introduced. The following describes the multi-agent collaborative path planning method, device, agent and storage medium of the embodiment of the present application with reference to the accompanying drawings.

[0036] Specifically, Figure 2 A flowchart of a multi-agent collaborative path planning method provided in an embodiment of the present application.

[0037] like Figure 2 As shown, the multi-agent collaborative path planning method includes the following steps:

[0038] In step S101, a first perceived image of a target agent and a second perceived image of at least one other agent around the target agent are acquired.

[0039] It can be understood that the target intelligent agent has certain perception capabilities, and at least one other intelligent agent around the target intelligent agent can be a neighboring intelligent agent with strong task relevance, and can share knowledge; therefore, the embodiment of the present application can obtain the perception image of the target intelligent agent while communicating with other intelligent agents to obtain the perception information shared by other intelligent agents.

[0040] In step S102, the first perceived image and the second perceived image of at least one other intelligent agent are fused to obtain a fused image.

[0041] In an embodiment of the present application, a first perceived image and a second perceived image of at least one other intelligent agent are fused to obtain a fused image, including: inputting the first perceived image and the second perceived image of at least one other intelligent agent into a preset neural network, outputting a first image feature of the first perceived image and a second image feature of each second perceived image; and fusing the first image feature with each second image feature to obtain a fused image.

[0042] Among them, the preset neural network can be selected according to actual conditions without specific limitation.

[0043] It can be understood that the embodiments of the present application can input the perceived image into CNN (Convolutional Neural Network), fully extract image features and generate high-dimensional knowledge vectors, and use TOKF-GraphSAGE to sample information from neighboring intelligent agents with strong task relevance. After knowledge sharing and information aggregation, the intelligent agent's understanding and cognition of the surrounding environment is enhanced, and knowledge features with a high degree of cognitive fusion are generated.

[0044] It should be noted that the embodiments of the present application can also be input into the CNN after the perception image is processed. Therefore, before fusing the first perception image and at least one second perception image, it can include: identifying the first perception image; when static obstacles and / or other agents around the target intelligent agent are identified, generating a first channel image according to the position of the static obstacles, and generating a second channel image according to the actual status of the other agents; generating a third channel image according to the position of the target point or the projection of the target point on the boundary of the perception range of the target intelligent agent; matrixing the first channel image, the second channel image and the third channel image to obtain a multi-channel matrixed perception image.

[0045] It can be understood that the embodiment of the present application can process the perceived image, and when the intelligent agent completes the processing of the perceived information, a three-channel matrixed perceived image can be generated.

[0046] Specifically, the real-time perception information of the intelligent agent is divided into three channels: "obstacle, state and goal". The obstacles in the task space are of any shape, and the intelligent agent can only detect the boundary information of obstacles and other intelligent agents within the perception range. The method of processing perception information of the obstacle channel is introduced in detail. Figure 3 As shown in the figure, there are three static obstacles of different shapes in the agent's perception field of view. Since the obstacles block the agent's perception field of view, the agent cannot reach the maximum perception distance in the obstacle area and can only perceive the boundary information of the obstacles. Therefore, the real perception image of the agent's obstacle channel is as follows: Figure 3As shown, the perception image is further matrixed to prepare for feature extraction. Then the state channel is generated according to the position information of other agents; the target channel is generated according to the position information of the target point or the projection of the target point on the boundary of the perception range, and finally a matrixed perception image of three channels is generated.

[0047] In step S103, the fused image is input into a preset multilayer perceptron, the predicted direction of the next movement of the target agent is output, and a planned path is generated according to the current position, target point position and predicted direction of the target agent, and the target agent is controlled to perform the target action along the planned path.

[0048] Among them, the target action includes any one of moving towards the target, turning on the spot, moving around a convex obstacle, changing the direction of the obstacle and avoiding the dynamic obstacle; the preset multilayer perceptron can be selected according to the actual situation without specific limitation, for example, it can be a standard MLP (Multilayer Perception).

[0049] It can be understood that the embodiments of the present application can construct a model framework composed of a data-driven module (a composite neural network composed of CNN, TOKF-GraphSAGE and MLP) and a knowledge-driven module (DH-Bug algorithm rule decision). The data-driven module maps out the regional direction prediction based on the fan distribution (region one, region two, region three, idle), and then calculates the specific movement direction of the intelligent agent through the knowledge module (DH-Bug algorithm rule decision). The final path of each intelligent agent is represented by a series of sequential actions.

[0050] Specifically, the embodiment of the present application inputs the knowledge feature into the standard MLP, and outputs the direction strategy of the agent based on the fan distribution at that moment, such as Figure 4 As shown, the sector angle of the agent toward the target point is 120°, which is equally divided into three regional sectors. The direction prediction consists of four discrete options: "region one, region two, region three, and idle", and is expressed in the form of a one-hot vector.

[0051] like Figure 5 As shown in the figure, the box below the model structure shows the workflow of the DH-Bug algorithm. For different scenarios, the DH-Bug algorithm designs 5 motion states for the agent, namely: moving toward the target (state 1), turning on the spot (state 2), moving around convex obstacles (state 3), changing the direction of the obstacle (state 4), and avoiding dynamic obstacles (state 5). Combined with the three modes in Section 5.2, the moving toward the target mode corresponds to state 1; the moving around the obstacle mode includes states 2, 3, and 4; the dynamic obstacle avoidance mode corresponds to state 5.

[0052] The DH-Bug algorithm has developed a set of rigorous and complete state transition rules based on the actual needs and possible situations of the agent path planning process. Considering the strict convergence and rule completeness of the DH-Bug algorithm, the proposed T-DH-Bug algorithm is based on the DH-Bug algorithm in the process of executing multi-agent collaborative path planning. The application time is when the agent is in a state of moving toward the target (state 1). At this time, the TOKF-GraphSAGE-based method is combined with the DH-Bug algorithm to provide an optimized prediction direction for the DH-Bug algorithm, and cooperate with the DH-Bug algorithm to navigate the agent to the target point; when the agent is in other states, the DH-Bug algorithm still uses the DH-Bug algorithm because it has a set of movement rules that have been strictly theoretically proven and can ensure strict convergence to prevent the completeness of the DH-Bug algorithm from being damaged.

[0053] In an embodiment of the present application, before generating a planned path based on the current position, target point position and predicted direction of the target intelligent body, it includes: detecting whether the area corresponding to the predicted direction is passable; if the area corresponding to the predicted direction is passable, generating a planned path based on the current position, target point position and predicted direction of the target intelligent body; if the area corresponding to the predicted direction is not passable, identifying the first perceived image to obtain the perceived information around the target intelligent body, and using a preset algorithm to calculate the actual direction of the next movement of the target intelligent body, and generating a planned path based on the current position, target point position and actual direction of the target intelligent body, and controlling the target intelligent body to perform the target action along the planned path.

[0054] Specifically, it is assumed that the agent is in a state of moving toward the target (state 1) at this time, and the composite neural network based on TOKF-GraphSAGE has mapped the predicted direction for the agent. Because it cannot be guaranteed that the predicted direction area learned by the composite neural network is passable, a feasibility judgment mechanism is added. After the direction prediction is completed, first determine whether the agent can actually pass the learned predicted direction. If it is passable, select the current predicted direction for the agent, and execute the DH-Bug algorithm for this direction. According to the movement direction calculation method under the state of moving toward the target (state 1), calculate the specific movement direction θ of the agent in the next step; if it is not passable, abandon this predicted direction, and directly calculate the specific movement direction θ of the agent in the next step according to the initial perception information of the agent through the movement direction calculation method under the state of moving toward the target (state 1) of the DH-Bug algorithm.

[0055] In summary, in order to enable multiple agents to reach their respective target points efficiently without collision, a framework model integrating data (TOKF-GraphSAGE information sharing) and knowledge (DH-Bug algorithm rule decision) is constructed based on the conditions of limited agent perception and communication range, no global positioning, and complex physical constraints. It can not only ensure the strict convergence of the algorithm, but also fully explore the task relevance between agents, thereby improving the coordination and planning performance of the system. Among them, in the data-driven (TOKF-GraphSAGE information sharing) module, CNN fully extracts the features of each agent's perception information to generate a high-dimensional feature vector, and inputs it into TOKF-GraphSAGE. TOKF-GraphSAGE aggregates the feature vectors of neighboring agents in the direction of each agent toward the target point, and inputs them into MLP, mapping out the regional direction prediction based on the fan distribution (region one, region two, region three, idle), and then calculates the specific movement direction of the agent through the knowledge module. The final path of each agent is represented by a series of sequential actions.

[0056] The following will be a simulation verification of the multi-agent collaborative planning method based on data and knowledge fusion driven by the embodiment of this application. MobileSim simulation software is selected as the experimental platform. In order to verify the performance of the T-DH-Bug algorithm on different types of maps, the embodiment of this application creates two experimental environments based on MobileSim simulation software, namely: Figure 6 The regular maze map (environment 1) shown in Figure 7 The irregular environment map shown (Environment 2).

[0057] In the experiment, for two types of maps, 600 two-dimensional space grid maps of size 20×30 were randomly generated as the task space for multi-agent collaborative path planning. Each map generated 10 sets of initial positions, totaling 6000 maps, of which 70% (4200) were used for training and 30% (1800) were used for testing. The obstacle ratio in the environment A was set to 15%, and the obstacle ratio in the environment B was set to 30%. The maximum perception distance of the agent is 5, and the maximum communication distance is 4. During the training process, the classic A* algorithm is used to generate the optimal path, and each step of the training is supervised, and the label is the generated path decision. Experimental evaluation criteria: average path length of the agent, dimensionless normalization.

[0058] Two groups of experiments were set up to verify the effectiveness and generalization of the T-DH-Bug algorithm: the first group of experiments measured the path planning execution efficiency of the T-DH-Bug algorithm by comparing the average path length of agents on two types of maps with different methods. The number of multi-agents involved in training and testing was 5 and 10 respectively, and the training cycle was 100 epochs. There were four algorithms used for comparison and verification, namely: Bug2 algorithm, DH-Bug algorithm, a method combining a composite network model based on CNN and MLP with the DH-Bug algorithm (CNN-DH-Bug algorithm), and a method combining a composite network model based on CNN, GraphSAGE, and MLP with the DH-Bug algorithm (GraphSAGE-DH-Bug algorithm); the second group of experiments tested the T-DH-Bug algorithm by changing the scale of different multi-agents (5, 10, 15, 20, 25), and compared the average path length of agents on the same map of the T-DH-Bug algorithm to measure the generalization of the T-DH-Bug algorithm to the scale of multi-agents. The training cycle was 100 epochs.

[0059] The first set of experimental results are shown in Tables 1 and 2. By comparing the average path length of agents on two types of maps with different methods, it can be found that the Bug2 algorithm has the largest average path length of agents. Compared with the Bug2 algorithm, the DH-Bug algorithm has improved system performance and slightly decreased average path length. The last three algorithms further shorten the average path length of agents on the basis of the first two methods, among which the T-DH-Bug algorithm has the best performance. The analysis shows that the first two methods are purely knowledge-driven methods with high execution efficiency, but they are prone to produce too many redundant paths; the last three methods are algorithms that combine knowledge-driven and data-driven. By learning a large number of data samples, the learning ability of the system is enhanced and the movement path of the agent is optimized. Among them, the GraphSAGEDH-Bug algorithm and the T-DH-Bug algorithm add data sampling and information aggregation mechanisms to enhance the system's synergy. The T-DH-Bug algorithm adopts a task-related directional sampling method to further improve the efficiency of the system's path planning.

[0060] Table 1 Comparison of results of different methods (n=5)

[0061]

[0062] Table 2 Comparison of results of different methods (n=10)

[0063]

[0064] The results of the second group of experiments are shown in Table 3. When the map setting and other conditions remain unchanged, when the scale of multi-agents increases to 25, the system can still achieve good experimental results. However, when the scale of multi-agents is large, it will still have a certain impact on the performance of the system. It can be found that when the number of agents exceeds 15, the average path length of the agents shows an increasing trend. This is because with the increase in the number of agents, the amount of redundant information generated will also increase, affecting the efficiency and accuracy of the agent's decision-making. At the same time, the increase in the number of agents is equivalent to the increase in the number of dynamic obstacles. The collision avoidance cost between agents will increase, which is easy to increase the probability of collision and conflict, thereby reducing the planning performance of the system. When the scale of multi-agents is small, the number is 5, the performance of the system is not better than when the number of agents is 10 or 15. This is because when the scale of multi-agents is small, the communication frequency between agents is reduced, the information aggregation channels are reduced, and the perception ability of agents to the surrounding environment is weakened, making it difficult to give play to the advantages of model collaborative planning. In summary, when the experimental map remains unchanged, the system can achieve good planning performance when the scale of multi-agents is between 5 and 15.

[0065] Table 3 Model generalization performance analysis

[0066]

[0067] The embodiment of the present application proposes a multi-agent collaborative path planning method based on the T-DH-Bug algorithm. The T-DH-Bug algorithm is an algorithm driven by the fusion of data (TOKF-GraphSAGE information sharing) and knowledge (DH-Bug algorithm rule decision), which can not only ensure the strict convergence of the algorithm, but also fully explore the task relevance between agents, thereby improving the coordination and planning performance of the system. Through MobileSim software simulation verification, the T-DH-Bug algorithm can optimize the planning path of the agent by comprehensively analyzing the related information at key points, and the algorithm has strong generalization performance.

[0068] Next, a multi-agent collaborative path planning device proposed according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0069] Figure 8 It is a block diagram of a multi-agent collaborative path planning device according to an embodiment of the present application.

[0070] like Figure 8 As shown, the multi-agent collaborative path planning device 10 includes: an acquisition module 100, a fusion module 200 and a planning module 300.

[0071] Among them, the acquisition module 100 is used to acquire a first perception image of the target intelligent agent and a second perception image of at least one other intelligent agent around the target intelligent agent; the fusion module 200 is used to fuse the first perception image and the second perception image of at least one other intelligent agent to obtain a fused image; the planning module 300 is used to input the fused image into a preset multi-layer perceptron, output the predicted direction of the next movement of the target intelligent agent, and generate a planned path according to the current position of the target intelligent agent, the target point position and the predicted direction, and control the target intelligent agent to perform the target action along the planned path.

[0072] In an embodiment of the present application, the planning module 300 is also used to, before generating a planned path based on the current position, target point position and predicted direction of the target intelligent body: detect whether the area corresponding to the predicted direction is passable; if the area corresponding to the predicted direction is passable, generate a planned path based on the current position, target point position and predicted direction of the target intelligent body; if the area corresponding to the predicted direction is not passable, identify the first perceived image to obtain the perceived information around the target intelligent body, and use a preset algorithm to calculate the actual direction of the next movement of the target intelligent body, and generate a planned path based on the current position, target point position and actual direction of the target intelligent body, and control the target intelligent body to perform the target action along the planned path.

[0073] In the embodiment of the present application, the device 10 of the embodiment of the present application further includes: a feature extraction module. The feature extraction module is used to input the first perceived image and the second perceived image of at least one other intelligent agent into a preset neural network, and output the first image feature of the first perceived image and the second image feature of each second perceived image; and a fusion module is used to fuse the first image feature and each second image feature to obtain a fused image after fusion.

[0074] In the embodiment of the present application, the device 10 of the embodiment of the present application further includes: an information processing module. The information processing module is used to identify the first perception image before fusing the first perception image and at least one second perception image; when static obstacles and / or other agents around the target intelligent agent are identified, a first channel image is generated according to the position of the static obstacles, and a second channel image is generated according to the actual state of the other agents; a third channel image is generated according to the position of the target point or the projection of the target point on the boundary of the perception range of the target intelligent agent; and the first channel image, the second channel image and the third channel image are matrixed to obtain a multi-channel matrixed perception image.

[0075] In the embodiment of the present application, the target action includes any one of a target movement action, an on-the-spot turning action, a movement action around a convex obstacle, a direction change action around an obstacle, and a dynamic obstacle avoidance action.

[0076] It should be noted that the aforementioned explanation of the embodiment of the multi-agent collaborative path planning method is also applicable to the multi-agent collaborative path planning device of this embodiment, and will not be repeated here.

[0077] According to the multi-agent collaborative path planning device proposed in the embodiment of the present application, the regional direction prediction based on the fan distribution is mapped according to the agent perception information and the shared information of other agents, and the specific movement direction of the agent is calculated, so as to realize multi-agent path planning based on the fusion of data and knowledge. Therefore, multi-agent path planning can be realized under the conditions of limited agent perception and communication range, no global positioning and complex physical constraint scenarios, which can not only ensure the strict convergence of the algorithm, but also fully explore the task correlation between agents, thereby improving the coordination and planning performance of the system.

[0078] Fig. 9 A schematic diagram of the structure of an agent provided in an embodiment of the present application. The agent may include:

[0079] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .

[0080] When the processor 902 executes the program, the multi-agent collaborative path planning method provided in the above embodiment is implemented.

[0081] Furthermore, the agent also includes:

[0082] The communication interface 903 is used for communication between the memory 901 and the processor 902 .

[0083] The memory 901 is used to store computer programs that can be executed on the processor 902 .

[0084] The memory 901 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0085] If the memory 901, the processor 902 and the communication interface 903 are implemented independently, the communication interface 903, the memory 901 and the processor 902 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0086] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.

[0087] The processor 902 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0088] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned multi-agent collaborative path planning method.

[0089] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0090] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0091] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0092] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0093] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps in the method embodiment.

Claims

1. A multi-agent collaborative path planning method, characterized in that: The following steps are involved: Acquire a first perceived image of a target intelligent agent and a second perceived image of at least one other intelligent agent around the target intelligent agent; fusing the first perceived image with the second perceived image of the at least one other intelligent agent to obtain a fused image; Input the fused image into a preset multilayer perceptron to output the predicted direction of the next movement of the target intelligent body; detect whether the area corresponding to the predicted direction is passable; If the area corresponding to the predicted direction is passable, a planned path is generated according to the current position of the target intelligent body, the target point position and the predicted direction, and the target intelligent body is controlled to perform the target action along the planned path; if the area corresponding to the predicted direction is not passable, the first perception image is identified to obtain the perception information around the target intelligent body, and the actual direction of the next movement of the target intelligent body is calculated using a preset algorithm, and a planned path is generated according to the current position of the target intelligent body, the target point position and the actual direction, and the target intelligent body is controlled to perform the target action along the planned path; wherein, planning is performed based on a model framework formed by the fusion of a data-driven module and a knowledge-driven module, and the data-driven module maps out the regional direction based on the fan distribution Prediction, and then calculate the specific movement direction of the agent through the rule decision of the knowledge-driven module. The final path of each agent is represented by a series of sequential actions. The data-driven module is a composite neural network composed of a preset neural network CNN, TOKF-GraphSAGE and a preset multi-layer perceptron MLP. The knowledge-driven module is a DH-Bug algorithm rule decision. In the process of executing multi-agent collaborative path planning, when the agent is in a state of moving toward the target, the TOKF-GraphSAGE-based method and the DH-Bug algorithm are combined. By providing an optimized prediction direction to the DH-Bug algorithm, the DH-Bug algorithm is used to navigate the agent to the target point; when the agent is in other states, the DH-Bug algorithm is still used.

2. The method according to claim 1, characterized in that The fusing the first perceived image and the second perceived image of the at least one other intelligent agent to obtain a fused image includes: Inputting the first perceived image and the second perceived image of the at least one other intelligent agent into a preset neural network, and outputting a first image feature of the first perceived image and a second image feature of each of the second perceived images; The first image feature and each of the second image features are fused to obtain the fused image.

3. The method according to claim 1, characterized in that Before fusing the first perceived image with at least one of the second perceived images, the method comprises: identifying the first perceived image; When static obstacles and / or other intelligent agents around the target intelligent agent are identified, a first channel image is generated according to the positions of the static obstacles, and a second channel image is generated according to the actual states of the other intelligent agents; Generate a third channel image according to the position of the target point or the projection of the target point on the boundary of the perception range of the target intelligent body; The first channel image, the second channel image and the third channel image are matrixed to obtain a multi-channel matrixed perception image.

4. The method according to any one of claims 1 to 3, characterized in that: The target action includes any one of a target movement action, an on-the-spot turning action, a movement action around a convex obstacle, a direction-changing action around an obstacle, and a dynamic obstacle avoidance action.

5. A multi-agent collaborative path planning device, characterized in that: include: An acquisition module, configured to acquire a first perceived image of a target intelligent agent and a second perceived image of at least one other intelligent agent around the target intelligent agent; A fusion module, used for fusing the first perception image with the second perception image of the at least one other intelligent agent to obtain a fused image; A planning module, used for inputting the fused image into a preset multi-layer perceptron and outputting a predicted direction of the next movement of the target intelligent agent; Detecting whether the area corresponding to the predicted direction is passable; If the area corresponding to the predicted direction is passable, a planned path is generated according to the current position of the target intelligent body, the target point position and the predicted direction, and the target intelligent body is controlled to perform the target action along the planned path; if the area corresponding to the predicted direction is not passable, the first perception image is identified to obtain the perception information around the target intelligent body, and the actual direction of the next movement of the target intelligent body is calculated using a preset algorithm, and a planned path is generated according to the current position of the target intelligent body, the target point position and the actual direction, and the target intelligent body is controlled to perform the target action along the planned path; wherein, planning is performed based on a model framework formed by the fusion of a data-driven module and a knowledge-driven module, and the data-driven module maps out the regional direction based on the fan distribution Prediction, and then calculate the specific movement direction of the agent through the rule decision of the knowledge-driven module. The final path of each agent is represented by a series of sequential actions. The data-driven module is a composite neural network composed of a preset neural network CNN, TOKF-GraphSAGE and a preset multi-layer perceptron MLP. The knowledge-driven module is a DH-Bug algorithm rule decision. In the process of executing multi-agent collaborative path planning, when the agent is in a state of moving toward the target, the TOKF-GraphSAGE-based method and the DH-Bug algorithm are combined. By providing an optimized prediction direction to the DH-Bug algorithm, the DH-Bug algorithm is used to navigate the agent to the target point; when the agent is in other states, the DH-Bug algorithm is still used.

6. The device according to claim 5, characterized in that Also includes: a feature extraction module, configured to input the first perceived image and the second perceived image of the at least one other intelligent agent into a preset neural network, and output a first image feature of the first perceived image and a second image feature of each of the second perceived images; The fusion module is used to fuse the first image feature and each of the second image features to obtain the fused image.

7. The device according to claim 5, characterized in that Also includes: an information processing module, configured to identify the first perceived image before fusing the first perceived image with at least one of the second perceived images; When static obstacles and / or other intelligent agents around the target intelligent agent are identified, a first channel image is generated according to the positions of the static obstacles, and a second channel image is generated according to the actual states of the other intelligent agents; A third channel image is generated according to the position of the target point or the projection of the target point on the boundary of the perception range of the target intelligent body; the first channel image, the second channel image and the third channel image are matrixed to obtain a multi-channel matrixed perception image.

8. The device according to any one of claims 5 to 7, characterized in that: The target action includes any one of a target movement action, an on-the-spot turning action, a movement action around a convex obstacle, a direction-changing action around an obstacle, and a dynamic obstacle avoidance action.

9. An intelligent agent, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-agent collaborative path planning method as described in any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the multi-agent collaborative path planning method as described in any one of claims 1-4.

Images