Multi-target pursuit method and system based on clustering and graph convolutional network

Abstract

The invention belongs to the technical field of multi-agent control, and discloses a multi-target pursuit method and system based on clustering and a graph convolutional network, and the method comprises the steps: regarding an agent as a node in a graph structure, processing the node to obtain a node feature, and combining the node feature with an adjacent matrix to obtain a first weight matrix; establishing edge connections between the agents, performing product operation on the edge connections and the first weight matrix to generate edge features, and splicing and integrating the edge features and the node features to generate graph representation; clustering the graph representation to obtain a plurality of clusters, calculating the importance of different clusters to obtain a second weight matrix, and performing weighted fusion on the second weight matrix and cluster features to obtain a global state; and taking the global state as a state space, modeling multi-target pursuit as a Markov decision process, and designing a reward function to obtain an optimal pursuit action. According to the method, efficient team cooperation pursuit is realized by clustering multiple agents, the training time is shortened, and powerful support is provided for improving the pursuit efficiency.

Description

Technical Field

[0001] This invention relates to the field of multi-agent control technology, and in particular to a multi-target tracking method and system based on clustering and graph convolutional networks. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, multi-agent cooperative control has become a prominent research area. Cooperative encirclement is a typical multi-agent cooperative control problem, which mainly refers to the process by which multiple pursuing agents approach and surround one or more escaping agents, restricting their activity range and ultimately capturing them.

[0004] In collaborative pursuit missions, agents can pursue escaped targets either individually or in groups. In individual pursuit, each agent acts independently, making its own observations and decisions. This requires each agent to have a high degree of autonomy, but individual pursuit only utilizes its own observations, resulting in insufficient utilization of information from other pursuing agents and leading to prolonged pursuit times. Group pursuit effectively solves this problem. It allows multiple agents to share information and make collaborative decisions to jointly pursue the escaped target. Furthermore, by cooperating with each agent in the group, the team can complete the pursuit more efficiently, effectively avoiding duplicate pursuits and conflicts.

[0005] Grouping is a crucial strategy in cooperative pursuit, and various methods exist, including distance-based, communication mechanism-based, and speed-based grouping. Distance-based grouping allows for rapid response to changes in the escape target's location, but it may lead to overutilization of some pursuing agents while others remain relatively idle, resulting in uneven resource allocation. Communication mechanism-based grouping effectively promotes collaborative decision-making and information sharing among agents, improving overall pursuit effectiveness; however, the communication mechanism may limit the flexibility of grouping and hinder rapid adaptation to dynamically changing environments. Speed-based grouping allows faster agents to pursue faster escape targets, but differences in speed can lead to imbalances in the pursuit process.

[0006] Existing multi-agent cooperative control methods often suffer from excessively long training times when performing multi-target pursuit tasks due to the increased complexity of the environment caused by the increased number of agents. There is an urgent need for an efficient team-based pursuit method to reduce training time. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a multi-target pursuit method and system based on clustering and graph convolutional networks. By integrating the distance, speed, and communication mechanisms between the pursuing agent and the escaping target, it enables effective grouping and collaborative action among agents. Through clustering of multiple agents, it achieves efficient team-based pursuit, reduces training time, and provides strong support for improving pursuit efficiency.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a multi-target tracking method based on clustering and graph convolutional networks, comprising the following steps: The pursuit agent is regarded as a node in a graph structure. Encoding and feature extraction are performed on each node to obtain node features. Multi-layer graph convolution operation is performed on the node features and the adjacency matrix to obtain the first weight matrix. Establish edge connections between the pursuing agents, multiply the edge connections with the first weight matrix to generate edge features, and then concatenate and integrate the edge features with the node features to generate a graph representation. Cluster the graph representation to obtain multiple clusters, embed features into each cluster, use a self-attention mechanism to calculate the importance of different clusters to obtain a second weight matrix, and then weight and fuse the second weight matrix with the cluster features to obtain the global state. By using the global state as the state space, multi-target pursuit is modeled as a Markov decision process, and a reward function is designed to obtain the optimal pursuit action.

[0009] As an alternative implementation, all pursuing agents move at the same speed, and the number of pursuing agents is greater than the number of escaping targets.

[0010] As an alternative implementation, the basis for clustering the graph representation includes: the distance between the pursuing agent and the escaping target, and the action interaction information between the pursuing agents.

[0011] As an alternative implementation method, when designing the reward function, the pursuit objective should be clearly defined first, that is, the reward for successfully pursuing all targets.

[0012] As an alternative implementation, the reward function may also include a distance reward between the pursuing agent and the escaping target, a cooperative reward between clusters, and a reward for avoiding collisions between the agent and obstacles.

[0013] As an alternative implementation, the cluster division is also dynamically adjusted due to the dynamic changes between the escaping target and the pursuing agent.

[0014] Secondly, the present invention provides a multi-target tracking system based on clustering and graph convolutional networks, comprising: The first weight matrix generation module is configured to: treat the pursuing agent as a node in a graph structure, perform encoding operations and feature extraction on each node to obtain node features, and perform multi-layer graph convolution operations on the node features and the adjacency matrix to obtain the first weight matrix. The graph representation generation module is configured to: establish edge connections between the pursuing agents, multiply the edge connections with the first weight matrix to generate edge features, and concatenate and integrate the edge features with the node features to generate a graph representation. The global state generation module is configured to: cluster the graph representation to obtain multiple clusters, embed features into each cluster, calculate the importance of different clusters using a self-attention mechanism to obtain a second weight matrix, and weightedly fuse the second weight matrix with the cluster features to obtain the global state. The decision-making module is configured to: use the global state as the state space, model multi-target pursuit as a Markov decision process, design a reward function, and obtain the optimal pursuit action.

[0015] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0016] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.

[0017] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a multi-target pursuit method and system based on clustering and graph convolutional networks. It combines K-Medoids clustering analysis with graph convolutional networks and the MAPPO reinforcement learning algorithm, proposing a KM-GCN-based multi-target pursuit method for training pursuit agents' pursuit strategies. An aggregation-based agent clustering method is used to pursue escaping targets. Through cluster team cooperation in pursuing escaping targets, the pursuit efficiency is improved, and the convergence time of the algorithm is significantly shortened. Furthermore, to test the robustness of the algorithm under different complex environments, this invention designs 6 vs. 2, 7 vs. 3, and 10 vs. 4 multi-target pursuit experiments. By comparing the success rates in different scenarios, the effectiveness of the proposed algorithm is verified.

[0019] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0021] Figure 1 This is a framework diagram of the KM-GCN algorithm provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of the multi-target tracking method based on clustering and graph convolutional networks provided in Embodiment 1 of the present invention; Figure 3 The graphs for the 6vs2 chase experiment are shown, where (a) is the reward function graph and (b) is the success rate graph. Figure 4 The diagram shows the 6vs2 experimental test process, where (a) is a diagram of the start of the pursuit, (b) is a diagram of the pursuit process, and (c) is a diagram of the end of the pursuit. Figure 5 The graphs for the 7vs3 chase experiment are shown, where (a) is the reward function graph and (b) is the success rate graph. Figure 6 The diagram shows the 7vs3 test process, where (a) is a diagram of the start of the pursuit, (b) is a diagram of the pursuit process, and (c) is a diagram of the end of the pursuit. Figure 7 The graphs for the 10 vs 4 pursuit experiment are shown, where (a) is the reward function graph and (b) is the success rate graph. Figure 8 The diagram shows the 10 vs 4 experimental test process, where (a) is a diagram of the start of the pursuit, (b) is a diagram of the pursuit process, and (c) is a diagram of the end of the pursuit. Figure 9 The charts show a comparison of the test results of the two algorithms, where (a) is a comparison of the number of test execution steps and (b) is a comparison of the pursuit success rate. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] It should be noted that the following detailed description is exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but includes other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] Example 1 like Figure 1 and Figure 2 As shown, this embodiment provides a multi-target tracking method based on clustering and graph convolutional networks, including the following steps: The pursuit agent is regarded as a node in a graph structure. Encoding and feature extraction are performed on each node to obtain node features. Multi-layer graph convolution operation is performed on the node features and the adjacency matrix to obtain the first weight matrix. Establish edge connections between the pursuing agents, multiply the edge connections with the first weight matrix to generate edge features, and then concatenate and integrate the edge features with the node features to generate a graph representation. Cluster the graph representation to obtain multiple clusters, embed features into each cluster, use a self-attention mechanism to calculate the importance of different clusters to obtain a second weight matrix, and then weight and fuse the second weight matrix with the cluster features to obtain the global state. By using the global state as the state space, multi-target pursuit is modeled as a Markov decision process, and a reward function is designed to obtain the optimal pursuit action.

[0027] The following is a detailed explanation of the solution of the present invention: In multi-target pursuit tasks, the increased complexity of the environment due to the number of agents leads to excessively long training times during deep reinforcement learning training. Therefore, to shorten training time, it is necessary to optimize the collaborative behavior of the pursuit agents during training. This can be achieved by designing collaborative mechanisms between systems to encourage cooperation and joint efforts to achieve the pursuit objective. Based on this, a multi-target pursuit method based on clustering analysis is proposed, aiming to achieve efficient team-based pursuit through clustering of multiple agents and reduce training time.

[0028] This invention proposes a multi-target pursuit algorithm based on KM-GCN. The overall design of the algorithm is as follows: First, a graph state representation is established. Then, through a graph convolutional network and K-Medoids clustering, the pursuing agent is divided into K clusters based on K escapees. Each cluster is multiplied by its corresponding weight and used as the global state input to the central value network. Finally, the Q-value is output to evaluate the quality of the Actor network's actions, thereby enabling the appropriate pursuit actions to be taken.

[0029] This invention employs distance-based clustering with the K-Medoids clustering algorithm, enabling more efficient and flexible pursuit and capture of escaping targets in a pursuit environment. In this embodiment, all pursuing agents are set to the same speed, and the number of pursuing agents is greater than the number of escaping targets.

[0030] This invention models the multi-target pursuit problem as a Markov decision process, starting with the design of the state space.

[0031] In a multi-target pursuit environment, the pursuing agent moves towards the escape target after initialization. During movement, the agent aggregates into K clusters using K-Medoids, forming different small teams to pursue the escape target. During movement, the agent also needs to consider dynamic interaction, meaning the agent's relationship with the target is dynamic, leading to changes in the cluster divisions, but the total number of clusters remains constant. Furthermore, collision avoidance with obstacles must be considered. Therefore, the state space design is as follows: First, define the observation space of a single agent. The observation space of a single agent includes the agent's own state. The status of nearby escape targets Location of obstacles and cluster partitioning information The definition is as shown in formula (1): (1) in, For intelligent agents The set of target states within the perception range. Indicates the pursuit of intelligent agents and escape target The distance between them The radius of perception of the intelligent agent.

[0032] Secondly, define the observation space for the entire cluster. This includes the observation space of all agents within the cluster and the state of the escaped target being jointly pursued by the team, as defined in formula (2): (2) in, It is a cluster The set of observation spaces of all intelligent agents within the system. It is a cluster The set of assigned target states.

[0033] In summary, the global state space design is defined as shown in formula (3): (3) in, The set of states of all pursuing agents; The set of states for all clusters; The set of all escape targets.

[0034] Design of the reward function: When designing the reward function for the KM-GCN algorithm, several factors need to be considered to ensure that the agent can effectively learn and optimize its policy. Therefore, the first step in designing the reward function is to clearly define the pursuit objective, namely, successfully pursuing all targets. Thus, the reward function is designed as follows: First, the reward for successfully capturing an escaping target is defined as shown in formula (4): (4) Secondly, a distance reward is set between the pursuing agent and the escaping target, defined as shown in formula (5): (5) in, This is the weighting coefficient, used to adjust the magnitude of the distance reward; it is set here. .

[0035] In addition, the collaborative reward between clusters should also be considered, as defined in formula (6): (6) in, For intelligent agents and The distance between them This is a weighting factor used to adjust the magnitude of the collaborative reward. The collaboration factor is used to measure the degree of collaboration between the pursuing agents, and is set to 0.5 here.

[0036] To avoid collisions between the agent and obstacles, a collision penalty is set, defined as shown in formula (7): (7) In summary, the overall reward function As shown in formula (8): (8) KM-GCN algorithm design: The state space and reward function of the algorithm have been introduced previously. This section mainly introduces the design of a multi-object pursuit algorithm based on KM-GCN. This method combines K-Medoids clustering analysis with graph convolutional networks into a reinforcement learning framework, aiming to handle complex data and improve pursuit performance in multi-object pursuit. The entire pursuit process can be divided into three stages: the association graph generation stage, the cluster partitioning stage, and the decision-making stage. A detailed analysis follows: The KM-GCN algorithm structure diagram is shown below. Figure 1 As shown: First, in the association graph construction phase, each pursuit agent undergoes node encoding, representing each agent as a vertex in the graph structure. The encoded node information is then transformed into an initial embedding vector by the feature extraction module, serving as the input data for the graph. Subsequently, the system completes the graph structure construction and feature learning through the following steps: Graph convolution processing: The input graph data, combined with the feature information of the adjacency matrix, undergoes multiple layers of graph convolution operations. By stacking multiple graph convolution layers, the features of neighboring nodes are aggregated step by step, ultimately outputting the corresponding weight matrix.

[0037] Edge relationship modeling: Edge connections are established between the pursuing agents. After encoding the connections, a message passing mechanism is used to propagate feature information between adjacent nodes. Each edge connection is multiplied with the learned weight matrix to generate edge-level output features.

[0038] Feature fusion: The edge feature output is pooled and then concatenated and integrated with the node features to finally generate a graph representation containing global association information.

[0039] This processing flow transforms raw agent information into a high-level relational graph representation, providing a structured environmental representation for subsequent decision-making.

[0040] Secondly, after the association graph generation stage is completed, the system uses the K-Medoids clustering algorithm to group the feature information of the agents. The clustering is based on the following criteria: 1. Distance from the escape target (reflecting the spatial relationship between each agent).

[0041] 2. Track the action interaction information between intelligent agents (measure the degree of cooperation).

[0042] After clustering, the agents are divided into K clusters, each cluster representing a potential cooperative unit. The system then executes the following processing flow: 1. Cluster representation embedding and attention weighting; 2. Perform feature embedding on each cluster to extract its cooperation pattern; 3. The importance of different clusters is calculated using a self-attention mechanism, and a weight matrix with dimension (1, K) is output, where each element represents the contribution ratio of the corresponding cluster (the sum of the weights is normalized to 1). 4. Weighted fusion of the weight matrix and cluster features enhances the influence of key collaborative units.

[0043] Finally, the weighted features are input into the Central Critic network, which outputs the Q-value of global cooperation, used to evaluate the expected benefits of different joint actions.

[0044] By comparing the Q-values ​​of different joint actions, the agent can more clearly understand the impact of its own state on the global situation and the dynamic trends of environmental changes. Based on Q-value feedback, the agent can quickly adjust its strategy, optimize its cooperation methods, and improve the efficiency of pursuing escaped targets.

[0045] This method effectively improves the decision-making coordination of multi-agent systems through a process of clustering, attention weighting, and global collaborative optimization, enabling them to adapt to complex dynamic environments and achieve efficient pursuit.

[0046] Results and analysis of multi-target pursuit experiments: To verify the effectiveness and stability of the proposed algorithm, experiments were conducted in the MPE simulation environment. Starting with the convergence speed and pursuit success rate of the training rounds, three scenarios with different numbers of pursuing agents and escapees were set up for experimental verification. In this task, the success criterion for multi-target pursuit is defined as the successful capture of all escape targets within a single round time step.

[0047] This experiment primarily focuses on improving convergence speed and pursuit success rate. The proposed KM-GCN multi-target pursuit algorithm and the MTPGCN multi-target pursuit algorithm were trained under the same scenario. The reward function and success rate of each algorithm were obtained and compared. Three pursuit scenarios were set up in the experiment: 6 pursuing agents pursuing 2 escape targets (6 vs 2), 7 pursuing agents pursuing 3 escape targets (7 vs 3), and 10 pursuing agents pursuing 4 escape targets (10 vs 4). The experimental setup is shown in Table 2. Table 2 Experimental Design

[0048] Based on the above experimental design, a multi-target pursuit experiment was conducted, with a total of 5000 training rounds. First, a 6vs2 pursuit experiment was performed. Three groups with different random seeds were set for training. After 5000 training rounds each, the data obtained from the KM-GCN and MTPGCN algorithms were compared on the same graph to compare the performance of the two algorithms. The reward function curve for the 6vs2 experiment is shown below. Figure 3 As shown.

[0049] Through the curve Figure 3 (a) It can be seen that the reward function curve of the KM-GCN algorithm begins to rise at 300 rounds and converges at 1100 rounds, remaining stable throughout the convergence process. In contrast, the MTPGCN algorithm converges more slowly, with its curve rising at 500 rounds and stabilizing at 2000 rounds. Therefore, the KM-GCN algorithm has a greater advantage in convergence speed compared to the MTPGCN algorithm. Thus, the proposed KM-GCN algorithm demonstrates a superior convergence speed.

[0050] Similarly, during the training process, the success rates of the KM-GCN and MTPGCN algorithms in a multi-target tracking task were evaluated, with different random seeds set, and the evaluation results were plotted as success rate curves. Experimental results are as follows: Figure 3 As shown in (b). Figure 3 (b) It can be seen that the KM-GCN algorithm also improves the pursuit success rate and convergence speed compared to the MTPGCN algorithm. The execution process of the KM-GCN algorithm in the 6vs2 test environment is as follows. Figure 4 As shown.

[0051] To further verify the scalability of the KM-GCN algorithm, a 7-vs3 pursuit experiment was designed, building upon the 6vs2 pursuit experiment. By comparing the performance of the KM-GCN and MTPGCN algorithms on tasks of different scales, the performance of the KM-GCN algorithm in handling complex multi-target pursuit tasks can be more comprehensively evaluated. Similarly, training data was collected after multiple training runs, and reward function curves for the two algorithms were plotted. The reward function curves are shown below. Figure 5 As shown.

[0052] pass Figure 5 (a) It can be seen that the KM-GCN algorithm begins to converge and tends to stabilize after 1700 training rounds; in contrast, the MTPGCN algorithm only begins to converge after 2000 rounds. This demonstrates that the KM-GCN-based multi-target tracking algorithm has a significant advantage in improving convergence speed. Similarly, the success rate is recorded after 5000 training rounds. Figure 5(b) It can be seen that the KM-GCN algorithm has a higher success rate than the MTPGCN algorithm. The execution process of the KM-GCN algorithm in the 7vs3 test environment is as follows: Figure 6 As shown.

[0053] Finally, to further extend the scalability of the KM-GCN algorithm in multi-target pursuit tasks, a 10 vs 4 pursuit experiment was set up, i.e., training was conducted in a scenario with 10 pursuing agents and 4 escaping targets. After 5000 training rounds, the data was obtained and plotted as a reward function curve. The reward function curve is shown below. Figure 7 As shown.

[0054] Depend on Figure 7 (a) It can be seen that in the 10 vs 4 multi-target pursuit experiment, the KM-GCN algorithm began to converge after 1800 rounds, while the MTPGCN algorithm began to converge after 2100 rounds. Both algorithms eventually stabilized, but the former converged faster. This demonstrates that the KM-GCN algorithm is well-suited for large-scale, complex multi-agent pursuit tasks and possesses significant performance advantages. Similarly, the pursuit success rate was evaluated in the 10 vs 4 multi-target pursuit experiment, and the success rate was plotted as a reward function curve, as shown in the figure. Figure 7 As shown in (b). Figure 7 (b) As can be seen from the success rate curve, the KM-GCN algorithm still shows a certain advantage in success rate compared to the KM-GCN algorithm, and it also exhibits significant scalability and stability in different task scenarios. The execution process of the KM-GCN algorithm in a 10vs4 test environment is as follows: Figure 8 As shown.

[0055] The above three sets of experiments (6 vs. 2, 7 vs. 3, and 10 vs. 4) lead to the conclusion that in multi-target pursuit tasks, aggregating multi-agent teams into multiple clusters using the K-Medoids algorithm and then aggregating environmental feature information through graph convolutional networks can effectively model complex interaction relationships, thereby optimizing pursuit strategies. This method not only effectively handles complex dynamic scenes but also has good scalability, providing an efficient solution for multi-target pursuit tasks.

[0056] To more intuitively compare the efficiency of the KM-GCN and MTPGCN algorithms in target acquisition, this paper statistically analyzes the average number of execution steps and the success rate of the two algorithms in the same scenario. A total of 100 rounds of testing were conducted, and the data for both algorithms were recorded. The recorded results are shown in Tables 3 and 4, respectively.

[0057] Table 3 Number of Steps in Multi-Target Pursuit

[0058] The data in Table 3 shows that the KM-GCN algorithm requires fewer execution steps on average than the MTPGCN algorithm in multi-target pursuit tasks. This result indicates that the KM-GCN algorithm is more efficient than the MTPGCN algorithm in task execution, and can complete the pursuit task in fewer steps.

[0059] Table 4 Success Rate of Multi-Target Pursuit

[0060] Table 4 shows that the KM-GCN algorithm successfully captured escaping targets slightly more often than the MTPCGN algorithm during the test. This result indicates that the KM-GCN algorithm significantly improves the success rate of multi-target pursuit tasks through its cluster cooperative pursuit strategy. To more intuitively illustrate these two sets of results, the number of execution steps and success rate of the two algorithms recorded in the three sets of experiments are plotted into bar charts and line graphs, as shown below. Figure 9 As shown.

[0061] Depend on Figure 9 The superior performance of the KM-GCN algorithm in multi-target pursuit tasks is clearly evident. During multi-target pursuit, the KM-GCN algorithm executed fewer steps than the MTPGCN algorithm in all three experimental sets (6 vs. 2, 7 vs. 3, and 10 vs. 4), indicating higher efficiency in multi-target pursuit. Furthermore, its success rate was slightly higher than that of the MTPGCN algorithm. This result demonstrates that the KM-GCN algorithm can effectively reduce the agent's step consumption and improve the pursuit success rate.

[0062] in conclusion: This paper proposes a KM-GCN-based multi-target pursuit method for training pursuit agents, combining K-Medoids clustering analysis with graph convolutional networks and the MAPPO reinforcement learning algorithm. The method employs aggregation to divide agents into clusters for pursuit of escaping targets. This collaborative approach among clusters improves pursuit efficiency and significantly shortens the algorithm's convergence time. Furthermore, to test the algorithm's robustness under various complex environments, 6 vs. 2, 7 vs. 3, and 10 vs. 4 multi-target pursuit experiments were designed. By comparing the success rates in different scenarios, the effectiveness of the proposed algorithm is verified.

[0063] Example 2 This embodiment provides a multi-target tracking system based on clustering and graph convolutional networks, including: The first weight matrix generation module is configured to: treat the pursuing agent as a node in a graph structure, perform encoding operations and feature extraction on each node to obtain node features, and perform multi-layer graph convolution operations on the node features and the adjacency matrix to obtain the first weight matrix. The graph representation generation module is configured to: establish edge connections between the pursuing agents, multiply the edge connections with the first weight matrix to generate edge features, and concatenate and integrate the edge features with the node features to generate a graph representation. The global state generation module is configured to: cluster the graph representation to obtain multiple clusters, embed features into each cluster, calculate the importance of different clusters using a self-attention mechanism to obtain a second weight matrix, and weightedly fuse the second weight matrix with the cluster features to obtain the global state. The decision-making module is configured to: use the global state as the state space, model multi-target pursuit as a Markov decision process, design a reward function, and obtain the optimal pursuit action.

[0064] It should be noted that the above modules correspond to the steps in Embodiment 1, and the examples and application scenarios implemented by the above modules and their corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules can be executed in a computer system as part of the system.

[0065] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0066] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0067] A computer-readable storage medium for storing computer instructions that, when executed by a processor, perform the method of Embodiment 1.

[0068] The method in Example 1 can be directly executed by a hardware processor, or it can be executed by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0069] A computer program product includes a computer program that, when executed by a processor, implements the method in Embodiment 1.

[0070] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.

[0071] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0072] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0073] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0074] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A multi-target tracking method based on clustering and graph convolutional networks, characterized in that, Includes the following steps: The pursuit agent is regarded as a node in a graph structure. Encoding and feature extraction are performed on each node to obtain node features. Multi-layer graph convolution operation is performed on the node features and the adjacency matrix to obtain the first weight matrix. Establish edge connections between the pursuing agents, multiply the edge connections with the first weight matrix to generate edge features, and then concatenate and integrate the edge features with the node features to generate a graph representation. Cluster the graph representation to obtain multiple clusters, embed features into each cluster, use a self-attention mechanism to calculate the importance of different clusters, obtain a second weight matrix, and then weight and fuse the second weight matrix with the cluster features to obtain the global state. By using the global state as the state space, multi-target pursuit is modeled as a Markov decision process, and a reward function is designed to obtain the optimal pursuit action.

2. The multi-target tracking method based on clustering and graph convolutional networks as described in claim 1, characterized in that, All pursuing agents move at the same speed, and the number of pursuing agents is greater than the number of escaping targets.

3. The multi-target tracking method based on clustering and graph convolutional networks as described in claim 1, characterized in that, The basis for clustering graph representations includes: the distance between the pursuing agent and the escaping target, and the action interaction information between the pursuing agents.

4. The multi-target tracking method based on clustering and graph convolutional networks as described in claim 1, characterized in that, When designing the reward function, the first step is to define the target of the pursuit mission, namely the reward for successfully pursuing all targets.

5. The multi-target tracking method based on clustering and graph convolutional networks as described in claim 4, characterized in that, The reward function also includes distance rewards between the pursuing agent and the escaping target, cooperative rewards between clusters, and rewards for avoiding collisions between the agent and obstacles.

6. The multi-target tracking method based on clustering and graph convolutional networks as described in claim 1, characterized in that, Due to the dynamic changes between the escape target and the pursuing agent, the cluster division will also be dynamically adjusted accordingly.

7. A multi-target tracking system based on clustering and graph convolutional networks, characterized in that, include: The first weight matrix generation module is configured to: treat the pursuing agent as a node in a graph structure, perform encoding operations and feature extraction on each node to obtain node features, and perform multi-layer graph convolution operations on the node features and the adjacency matrix to obtain the first weight matrix. The graph representation generation module is configured to: establish edge connections between the pursuing agents, multiply the edge connections with the first weight matrix to generate edge features, and concatenate and integrate the edge features with the node features to generate a graph representation. The global state generation module is configured to: cluster the graph representation to obtain multiple clusters, embed features into each cluster, calculate the importance of different clusters using a self-attention mechanism to obtain a second weight matrix, and weightedly fuse the second weight matrix with the cluster features to obtain the global state. The decision-making module is configured to: use the global state as the state space, model multi-target pursuit as a Markov decision process, design a reward function, and obtain the optimal pursuit action.

8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-6.

Images