Multi-target collaborative pursuit method and system based on graph convolutional network

Abstract

The invention provides a multi-target collaborative pursuit method and system based on a graph convolutional network. The method comprises the following steps: determining a pursuit environment; each paving person uses a node code as a vertex, the vertex comprises feature information, all vertex information is aggregated by using a graph convolutional network, and in the aggregation process, a weight matrix is used to perform linear transformation on the vertex information; expressing the interaction relationship among the pursuers by using edges, multiplying each edge by a corresponding weight matrix, integrating, splicing the integrated information and vertex information, and determining an optimal strategy according to the spliced information; an action is obtained by using a strategy network in an actor-commentator algorithm to pursuit an escaper, an environment and a corresponding reward return are input into a value network, an optimal strategy is executed, and multi-target collaborative pursuit is realized. According to the invention, the chasing efficiency and success rate of the multi-target chasing task are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-agent control, and specifically relates to a multi-target collaborative pursuit method and system based on a graph convolutional network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] As a research hotspot in many fields such as robotics, artificial intelligence, and control theory, multi-agent systems (MAS) are gradually becoming a key technology for solving mission challenges in complex, dynamic, and uncertain environments. In many practical application scenarios such as target search, collaborative surveillance, drone swarm combat, and intelligent traffic management, multi-agent systems have demonstrated their unique advantages and potential. Through close collaboration and information sharing between agents, these systems can overcome the limitations of a single agent in perception, decision-making, and execution capabilities, thereby efficiently completing various complex tasks.

[0004] In the specific application scenario of multi-agent collaborative pursuit, as the number of agents in the environment increases, the complexity of the pursuit environment also increases dramatically. This is mainly reflected in the following aspects: first, multiple agents need to coordinate their actions in real time to avoid collisions; second, in a restricted and obstacle-filled environment, agents need to accurately perceive and avoid obstacles while tracking and predicting the dynamics of escapees; finally, in the face of multiple escapees, agents also need to formulate complex strategies to optimize the pursuit order and path to improve the overall pursuit efficiency. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a multi-target collaborative pursuit method and system based on graph convolutional network. The present invention improves the perception, decision-making and collaboration capabilities of intelligent agents in complex and restricted environments through deep learning technology, thereby improving the pursuit efficiency and success rate of multi-target pursuit tasks. At the same time, through the efficient graph network representation method, the learning efficiency and performance are improved, and large-scale multi-agent systems can be better trained.

[0006] According to some embodiments, the present invention adopts the following technical solutions:

[0007] A multi-target collaborative hunting method based on graph convolutional network includes the following steps:

[0008] Determine the chasing environment;

[0009] Obtain the position and speed information of the pursuer and the escaper to form feature information;

[0010] Each pursuer is encoded as a vertex through a node, and the vertex contains the feature information. All vertex information is aggregated using a graph convolutional network. During the aggregation process, a weight matrix is ​​used to perform a linear transformation on the vertex information.

[0011] The interaction between the pursuers is expressed by edges, and each edge is integrated after multiplying the corresponding weight matrix. The integrated information is spliced ​​with the vertex information, and the optimal strategy is determined based on the spliced ​​information.

[0012] Based on the optimal strategy, the policy network in the actor-critic algorithm is used to obtain actions to hunt down the fugitive, and the pursuit environment gives corresponding reward returns to the value network to execute the optimal strategy and realize multi-target collaborative pursuit.

[0013] As an optional implementation, a discrete action space is used in the pursuit environment to encode the actions of each pursuer and escapee, and numerical values ​​are used to represent different speeds and speed change directions.

[0014] As an optional implementation, the characteristic information includes the position and speed of the pursuer himself, the positions of other pursuers, the relative position and relative speed of the escaper, and the relative position of obstacles.

[0015] As an optional implementation, each vertex corresponds to a chaser, and all vertices are dynamically movable;

[0016] Each edge transmits the characteristic information of vertices to each other to complete the information exchange, and all edges are movable.

[0017] As an optional implementation, during the aggregation process, a weight matrix is ​​used to perform a linear transformation on the node information, and the aggregated feature information is linearly transformed through a nonlinear activation function.

[0018] As an optional implementation, each edge is multiplied by the corresponding weight matrix and then average pooling is performed to integrate the information of multiple edges.

[0019] As an optional implementation, the optimal strategy is determined using a gated recurrent unit based on the concatenated information.

[0020] As an optional embodiment, the reward includes a positive reward for the pursuer when successfully capturing the escapee, a negative reward for colliding with other pursuers and obstacles, and a reward for the total distance between the pursuer and the escapee.

[0021] As an optional implementation, the actor-critic algorithm is trained and updated in an iterative manner. In each iteration, the policy network and the environment interact to collect certain data, and then the collected data is placed in an experience pool. A batch of specified amounts of data is collected through random sampling to train and update the policy network and the value network.

[0022] A multi-target collaborative pursuit system based on graph convolutional network, comprising:

[0023] An initialization module, configured to determine a chasing environment;

[0024] The association graph generation module is configured to obtain the position information and speed information of the pursuer and the escaper to form feature information; each pursuer is encoded as a vertex through a node, and the vertex contains the feature information, and all vertex information is aggregated using a graph convolutional network. During the aggregation process, the vertex information is linearly transformed using a weight matrix; the interaction relationship between the pursuers is expressed using edges, and each edge is multiplied by the corresponding weight matrix and then integrated, and the integrated information is spliced ​​with the vertex information, and the optimal strategy is determined based on the spliced ​​information;

[0025] The action decision module is configured to obtain actions to pursue the fugitive based on the optimal strategy using the strategy network in the actor-critic algorithm, and the pursuit environment gives corresponding reward returns to the input value network to execute the optimal strategy and realize multi-target collaborative pursuit.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The present invention makes a detailed analysis of the multi-agent pursuit task, divides the pursuit task into two stages, namely the association graph generation stage and the action decision stage, proposes a multi-target collaborative pursuit method based on graph convolutional network, and defines the condition for successful pursuit as when all escapees are caught by the pursuers, the pursuit is successful;

[0028] The present invention designs the state space, discrete action space and reward function for the multi-target pursuit task, and the reasonable design enables the pursuer to successfully capture the escapee.

[0029] The present invention introduces graph convolutional networks to model the complex relationships and interactions between multiple agents. It not only solves the problem of increased environmental complexity caused by the increase in the number of agents, but also significantly improves the efficiency and success rate of pursuing multiple evaders in complex and restricted environments by enhancing the perception, decision-making and collaboration capabilities of the agents.

[0030] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0032] Figure 1 is an MPE environment simple_tag scene graph of an embodiment;

[0033] Figure 2 is a schematic diagram of discrete action space encoding of an embodiment;

[0034] Figure 3 The present invention is a schematic diagram of a multi-target collaborative hunting method process based on a graph convolutional network according to an embodiment;

[0035] Figure 4 is a schematic diagram of a multi-target collaborative pursuit process based on a graph convolutional network according to an embodiment;

[0036] Figure 5 is a multi-target pursuit simulation diagram of an embodiment;

[0037] Figure 6 It is a schematic diagram of the multi-target pursuit experiment results of an embodiment. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0039] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0040] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0041] In the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other.

[0042] Embodiment 1

[0043] A multi-target collaborative hunting method based on graph convolutional network, such as Figure 4 As shown, the following steps are included:

[0044] Determine the chasing environment;

[0045] Obtain the position and speed information of the pursuer and the escaper to form feature information;

[0046] Each pursuer is encoded as a vertex through a node, and the vertex contains the feature information. All vertex information is aggregated using a graph convolutional network. During the aggregation process, a weight matrix is ​​used to perform a linear transformation on the vertex information.

[0047] The interaction between the pursuers is expressed by edges, and each edge is integrated after multiplying the corresponding weight matrix. The integrated information is spliced ​​with the vertex information, and the optimal strategy is determined based on the spliced ​​information.

[0048] Based on the optimal strategy, the policy network in the actor-critic algorithm is used to obtain actions to hunt down the fugitive, and the pursuit environment gives corresponding reward returns to the value network to execute the optimal strategy and realize multi-target collaborative pursuit.

[0049] A detailed description is given below.

[0050] In this embodiment, Figure 1 As shown in the figure, the multi-target hunting environment adopts the simple_tag scenario of the MPE (Multi-agent Particle Environment) simulation environment officially proposed by OpenAI, namely the predator-prey scenario. This scenario not only simulates the common predation and escape behaviors in nature, but also allows domestic and foreign researchers to deeply explore the collaboration, competition and design and execution of complex strategies among multiple agents. Therefore, this scenario is an ideal experimental environment for studying the coordination and confrontation of multiple agents in the fields of artificial intelligence, machine learning and reinforcement learning. As a classic scenario in MPE, simple_tag is regarded as a particle that can move freely in two-dimensional space. These particles move according to certain physical rules (speed limit and acceleration limit) and optimize their behavior strategies by receiving environmental information and executing desired actions. Therefore, it is particularly suitable for studying the predator-prey problem of multi-agent systems. In this scenario, agents are divided into two roles: predator and prey. The goal of predators is to hunt prey, and they need to learn how to work together to adopt better strategies to increase the capture rate. Efficient communication, information sharing and strategic coordination between predators are the key to successfully capturing prey. The goal of prey is to escape capture, so they need to disperse and escape, use obstacles for cover, etc. to escape the fate of being captured.

[0051] Deep reinforcement learning builds an agent system to operate the robot to interact with the environment continuously, and take actions that can maximize the cumulative reward according to the state information of the environment; after the agent takes an action, the environment will evaluate the quality of the action and give a corresponding reward value; in the process of interacting with the environment, the agent will continuously record the state information of the interaction with the environment and the actions taken, and continuously optimize its own behavior rules based on the recorded historical information. Therefore, in reinforcement learning, the main components are agent, environment, state, action and reward.

[0052] In a multi-target cooperative pursuit task, all pursuers and fugitives move in the xy two-dimensional plane, and the observation space of a single pursuer is o s Including its own position p i and speed v i , the positions of other team members others , the relative position of the escaper p evader and relative velocity v e and the relative position p of the obstacle obs In particular, the observation space is shown in formula (1):

[0053]

[0054] Among them, p obs =[[x o1 ,y o1 ],...,[x ok ,y ok ]] represents the position coordinates of k obstacles.

[0055] In deep reinforcement learning, the set of valid actions is called the action space. Therefore, the kinematic model of the pursuer is described by the following formula (2):

[0056]

[0057] Where p(t)=(p x (t),p y (t)) represents the position of the pursuer at time t, v(t) = (v x (t),v y (t)) represents the speed of the pursuer at time t, u(t) = (u x (t),u y (t)) represents the action taken by the pursuer under observation at time t, the change is the acceleration of the pursuer at the corresponding time, and Δt is the unit time.

[0058] Therefore, in the environment of this embodiment, a discrete action space design is adopted, such as Figure 2 As shown, action 0 indicates that the agent's speed remains unchanged, and the remaining actions represent different speed change directions.

[0059] In this embodiment, the pursuer's initial speed is 0 cm / s, the acceleration is 3.5 cm / s, and the maximum speed is 3.5 cm / s, while the escaper's initial speed is 0 cm / s, the acceleration is 4 cm / s, and the maximum speed is 4 cm / s.

[0060] In reinforcement learning, designing a reasonable reward function can effectively improve the learning efficiency of the agent and the success rate of the pursuit task. Therefore, in the algorithm proposed in this paper, the reward obtained by a single pursuer at each time step is shown in formula (3):

[0061] r i =r success +r collision +r potential (3)

[0062] The pursuer receives a positive reward when successfully capturing the escapee, which is the pursuer's goal.

[0063] Among them, r success Defined as the pursuer successfully capturing the escapee:

[0064]

[0065] When the escapee is successfully captured success is 10 if yes, otherwise it is 0.

[0066] In addition, the pursuer should avoid collisions with other team members and obstacles, so there needs to be a negative reward for collision, namely r collision is the negative reward for collisions with other pursuers and obstacles, defined as:

[0067]

[0068] -2 if the pursuer hits an obstacle, 0 otherwise.

[0069] To avoid too sparse rewards in the decision-making process, the total distance between the pursuer and the escaper is regarded as a negative process reward, i.e., r potential , defined as:

[0070]

[0071] Among them, k is the proportional coefficient, here we take the value as 0.1, (x p ,y p) is the position coordinate of the pursuer, (x e ,y e ) are the location coordinates of the escapee.

[0072] It is worth noting that when the pursuer approaches the escaper, the pursuer only gets r potential The pursuer will also receive r when it collides with the pursuer. collision The pursuer will only receive a positive reward r if he successfully captures the escapee. success The overall reward function in the environment is the sum of the rewards R of the n predators, that is:

[0073]

[0074] The Actor-Critic Algorithm is a classic algorithm in reinforcement learning. It combines policy gradient and temporal difference learning. The Actor-Critic Algorithm consists of two parts: the Actor and the Critic. The Actor refers to the policy function, that is, the agent learns a policy to obtain higher rewards, which is usually represented by a neural network, so it is also called a policy network; the Critic refers to the value function, which estimates the current policy, that is, evaluates the quality of the actor network. Similarly, the Critic network is also represented by a neural network, so it is also called a value network. The actor network is used to interact with the environment and adopts the policy gradient update method, while the Critic is used to evaluate the actions output by the actor network to distinguish the quality of the actions. The Actor-Critic Algorithm is trained and updated in an iterative way. In each iteration, the actor network is first allowed to interact with the environment to collect certain data, and then the data is placed in the experience pool. Then, a batch of specified data is collected by random sampling to train and update the two networks.

[0075] Graph Convolutional Network (GCN) is a type of graph neural network. It is a deep learning model specifically used to process graph structured data. Its core idea is to perform convolution operations on graph structured data. The working principle of graph convolutional network is based on the message passing mechanism, that is, the adjacency matrix of the graph is used to represent the connection relationship between nodes on the way. For a given node, the graph convolutional network aggregates the features of its neighboring nodes (including the features of the node itself); the aggregated features are linearly transformed to obtain each node representation, thereby capturing the complex relationship between nodes and the deep structural features of the graph. Therefore, the implementation of graph convolutional network includes two parts: graph convolution layer and activation function. Graph convolutional network is responsible for performing graph convolution operations, that is, updating the feature representation of each node by aggregating the feature information of neighboring nodes; activation function enables graph convolutional network to learn and simulate complex function mapping.

[0076] In summary, this embodiment integrates the actor-critic algorithm with the graph convolutional network and applies it in a multi-target pursuit environment to form a multi-target collaborative pursuit method based on graph convolutional network (MTPGCNet), which aims to handle complex data and pursuit performance in the multi-target pursuit process.

[0077] The specific analysis is as follows:

[0078] In multi-target hunting, the entire hunting process is divided into the association graph generation stage and the action decision stage. In this structure, the association graph generation stage mainly represents the environmental information in the form of a graph to improve the learning efficiency of the agent. The policy information obtained through the association graph stage is passed to the policy network to select the appropriate action to hunt the mobile target. Therefore, the following points are mainly defined in this method:

[0079] (1) Vertex generation: In the association graph, each vertex represents a pursuer and is dynamically movable; each vertex includes a series of feature information, such as speed information and position information.

[0080] (2) Edge establishment: There is a corresponding edge between every two vertices (pursuers). The edge is used to represent the interaction relationship between vertices. Since all pursuers are homogeneous agents, the interaction relationship is a cooperative relationship, that is, cooperative pursuit. Since this environment is a dynamic environment, the position of the edge will change with the movement of the pursuer. Each edge has a corresponding weight to represent the strength and distance of the relationship.

[0081] In the association graph generation phase, each pursuer in the environment is embedded into the network as a vertex through code embedding, and all vertices are dynamically movable, and each vertex contains a variety of feature information, namely position vector, velocity vector and other information; the feature information of these vertices is input into the graph convolution network respectively, and all node information is aggregated in the graph convolution network. During the aggregation process, the node information is linearly transformed using the weight matrix; the aggregated feature information is linearly transformed through a nonlinear activation function (ReLU function) to increase the nonlinear expression ability of the model, thereby helping the neural network capture complex feature relationships; after being processed by the graph convolution layer, a weight matrix is ​​output, which includes the information of itself and neighboring nodes, and the output weight matrix is ​​multiplied with a specific edge. For the edge establishment phase, each pursuer has a relationship used to represent interaction, namely the edge. First, each edge is encoded and embedded into the network. All edges are movable, and each edge can transmit the feature information of the vertex to each other to complete the information exchange; the edge embedded in the network is multiplied by the corresponding weight matrix, and the information of multiple edges is integrated after average pooling (Average pool); the integrated information is spliced ​​and integrated with the vertex information, and finally the gated recurrent unit (GRU) enhances its decision-making ability and outputs the optimal strategy.

[0082] The optimal strategy obtained in the association graph generation phase is passed to the policy network (Actor network) in the actor-critic algorithm, which outputs appropriate actions to hunt down the escapee, and the environment gives corresponding rewards to input into the value network. Through the mutual cooperation of the association graph generation and the policy decision phase, the multi-target pursuit task can be better trained, which can effectively solve the problems of low learning efficiency and poor pursuit success rate of reinforcement learning.

[0083] In order to verify the effectiveness of the proposed algorithm, this embodiment verifies the algorithm in the simple_tag scene of the MPE (Multi-agent ParticleEnvironment) simulation environment. Starting from the round reward value and the indicators of the pursuit success rate of the training process, multiple groups of experiments with different numbers of pursuers and escapees are set up for verification. The sign of success of the multi-target pursuit task is: during the execution of the task, if the distance between any pursuer and the escapee it is tracking is less than 2cm, it is considered that the pursuer has successfully pursued the escapee, which marks the success of the entire multi-target pursuit task. First of all, in the training process of this paper, due to the large number of intelligent agents, the complexity of the pursuit environment becomes larger, and a higher hardware configuration is required to better serve this experiment. Therefore, the hardware equipment of this experiment is represented as follows:

[0084] Table 1 Experimental hardware equipment

[0085]

[0086]

[0087] During the training process, the maximum number of training rounds is set to 5000 rounds (episode), and the maximum number of execution steps per round (episode) is 100. The remaining parameter settings are shown in the following table:

[0088] Table 2 Hyperparameter settings of multi-target collaborative hunting method based on graph convolutional network

[0089]

[0090] For multi-target hunting tasks, in order to verify the performance of multi-target hunting methods based on graph convolutional networks, this paper conducted experiments on long-term reward values ​​and hunting success rates for hunting tasks, and set up hunting comparison experiments based on the MAPPO algorithm. Therefore, this paper named the experimental schemes as follows:

[0091] Table 3 Experimental plan

[0092]

[0093]

[0094] Based on the above analysis, this paper sets up three groups of different numbers of pursuers and escapees in the simulation scenario to verify the scheme of this paper, namely 6 pursuers vs 2 escapees, 7 pursuers vs 3 escapees, and 10 pursuers vs 4 escapees. In the MPE environment, the above three groups of experiments are simulated, and the paths of the pursuers and escapees are drawn respectively. Figure 5 (a) is 6 pursuers chasing 2 escapers, Figure 5 (b) is 7 pursuers chasing 3 escapees, Figure 5 (c) is 10 pursuers chasing 4 escapees.

[0095] For the above three groups of experiments, the success rate of pursuit is shown in the form of graphs, as shown in the figure below: Figure 6 As shown in the figure, the multi-target collaborative pursuit method based on graph convolutional network (MTPGCNet) has significant advantages in solving complex multi-target pursuit tasks, and also deeply analyzes the learning dynamics and performance of this method in different pursuit-escape scenarios. Figure 6It can be seen that in the early stage of training, the pursuer cannot learn a good pursuit strategy, resulting in a low success rate of pursuit; however, as the number of training increases, the graph convolutional network can effectively capture the interaction between intelligent agents, thereby gradually forming an efficient collaborative pursuit strategy, and the pursuer gradually learns how to effectively pursue the escapee. This strategy optimization process reflects the potential of deep reinforcement learning in dealing with complex environment interactions, and also highlights the unique advantages of graph convolutional networks in modeling complex system structures. Figure 6 It can be clearly seen from the three experimental figures that the success rate increases with the increase of training rounds and eventually converges, and the final success rate is above 90%. Figure 6 In (a), when six pursuers are chasing two escapees, the success rate of the MTPGCNet method proposed in this paper is above 95%, while the success rate based on the MAPPO algorithm can only fluctuate around 30%; Figure 6 In (b), when seven pursuers are chasing three escapees, the success rate of the proposed method is over 90%, while the success rate of multi-target pursuit based on MAPPO algorithm is only between 40% and 50%. Figure 6 In (c), when the number of pursuers is set to 10 and the number of escapees is set to 4, the environment becomes more complex. However, the success rate of the MTPGCNet method proposed in this paper after convergence is above 98%, while the success rate of the MAPPO algorithm is only about 70%. The above results not only prove the robustness and efficiency of this method in complex environments, but also inspire us to fully consider the spatial relationship and dynamic interaction between agents when designing a multi-agent collaborative system.

[0096] In summary, the method proposed in this paper significantly improves the learning efficiency and performance of multi-target pursuit tasks by introducing graph convolutional networks to model the complex relationships and interactions between multiple agents. Its excellent performance in multiple experimental scenarios not only verifies the effectiveness of this method, but also provides new ideas and methods for future research in the field of multi-agent collaborative decision-making.

[0097] Embodiment 2

[0098] A multi-target collaborative pursuit system based on graph convolutional network, comprising:

[0099] An initialization module, configured to determine a chasing environment;

[0100] The association graph generation module is configured to obtain the position information and speed information of the pursuer and the escaper to form feature information; each pursuer is encoded as a vertex through a node, and the vertex contains the feature information, and all vertex information is aggregated using a graph convolutional network. During the aggregation process, the vertex information is linearly transformed using a weight matrix; the interaction relationship between the pursuers is expressed using edges, and each edge is multiplied by the corresponding weight matrix and then integrated, and the integrated information is spliced ​​with the vertex information, and the optimal strategy is determined based on the spliced ​​information;

[0101] The action decision module is configured to obtain actions to pursue the fugitive based on the optimal strategy using the strategy network in the actor-critic algorithm, and the pursuit environment gives corresponding reward returns to the input value network to execute the optimal strategy and realize multi-target collaborative pursuit.

[0102] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principle of the present invention without creative labor shall be included in the protection scope of the present invention.

Claims

1. A multi-target collaborative hunting method based on graph convolutional network, characterized in that: The following steps are involved: Determine the chasing environment; Obtain the position and speed information of the pursuer and the escaper to form feature information; Each pursuer is encoded as a vertex through a node, and the vertex contains the feature information. All vertex information is aggregated using a graph convolutional network. During the aggregation process, a weight matrix is ​​used to perform a linear transformation on the vertex information. The interaction between the pursuers is expressed by edges, and each edge is integrated after multiplying the corresponding weight matrix. The integrated information is spliced ​​with the vertex information, and the optimal strategy is determined based on the spliced ​​information. Based on the optimal strategy, the policy network in the actor-critic algorithm is used to obtain actions to hunt down the fugitive, and the pursuit environment gives corresponding reward returns to the value network to execute the optimal strategy and realize multi-target collaborative pursuit.

2. The multi-target collaborative hunting method based on graph convolutional network as claimed in claim 1, characterized in that: In the pursuit environment, a discrete action space is used to encode the actions of each pursuer and escapee, and numerical values ​​are used to represent different speeds and speed change directions.

3. The multi-target collaborative hunting method based on graph convolutional network as claimed in claim 1, characterized in that: The characteristic information includes the position and speed of the pursuer itself, the positions of other pursuers, the relative position and relative speed of the escaper, and the relative position of obstacles.

4. The multi-target collaborative hunting method based on graph convolutional network as claimed in claim 1, characterized in that: Each vertex corresponds to a chaser, and all vertices are dynamically movable; Each edge transmits the characteristic information of vertices to each other to complete the information exchange, and all edges are movable.

5. The multi-target collaborative hunting method based on graph convolutional network as claimed in claim 1, characterized in that: During the aggregation process, the weight matrix is ​​used to perform linear transformation on the node information, and the aggregated feature information is linearly transformed through a nonlinear activation function.

6. The multi-target collaborative hunting method based on graph convolutional network as claimed in claim 1, characterized in that: After each edge is multiplied by the corresponding weight matrix, the information of multiple edges is integrated after average pooling.

7. The multi-target collaborative hunting method based on graph convolutional network as claimed in claim 1, characterized in that: Based on the concatenated information, the gated recurrent unit is used to determine the optimal strategy.

8. The multi-target collaborative hunting method based on graph convolutional network as claimed in claim 1, characterized in that: The reward includes a positive reward for the pursuer when successfully capturing the escapee, a negative reward for colliding with other pursuers and obstacles, and a reward for the total distance between the pursuer and the escapee.

9. The multi-target collaborative hunting method based on graph convolutional network as claimed in claim 1, characterized in that: The actor-critic algorithm is trained and updated in an iterative manner. In each iteration, the policy network and the environment interact to collect certain data, which are then placed in an experience pool. A batch of specified amounts of data is collected through random sampling to train and update the policy network and the value network.

10. A multi-target collaborative pursuit system based on graph convolutional network, characterized by comprising: An initialization module, configured to determine a chasing environment; The association graph generation module is configured to obtain the position information and speed information of the pursuer and the escaper to form feature information; Each pursuer is encoded as a vertex through a node, and the vertex contains the feature information. All vertex information is aggregated using a graph convolutional network. During the aggregation process, the vertex information is linearly transformed using a weight matrix. The interaction between the pursuers is expressed using edges, and each edge is multiplied by the corresponding weight matrix and then integrated. The integrated information is spliced ​​with the vertex information, and the optimal strategy is determined based on the spliced ​​information. The action decision module is configured to obtain actions to pursue the fugitive based on the optimal strategy using the strategy network in the actor-critic algorithm, and the pursuit environment gives corresponding reward returns to the input value network to execute the optimal strategy and realize multi-target collaborative pursuit.

Images