Multi-agent centralization strategy updating method and evaluation system based on game confrontation

Abstract

The invention provides a game confrontation-based multi-agent centralization strategy updating method and evaluation system, and the method comprises the steps: building a communication undirected graph of our nodes, distributing an ID and a communication port for each node in the communication undirected graph, selecting a central node based on the node ID, and carrying out the game confrontation-based multi-agent centralization strategy updating. According to the method and the system, the non-central node and the central node are combined, and the message sending path between the non-central node and the central node is determined, so that local information can be summarized by utilizing the central node, a more reasonable strategy can be formulated in a short time, and the decision-making intelligence capability level of a multi-agent system can be improved to a certain extent. According to the technical scheme, the technical problems that in the prior art, an existing model strategy is long in updating time and poor in updating effect are solved.

Description

Technical Field

[0001] The present invention relates to the field of multi-agent system decision-making technology, and in particular to a multi-agent centralized strategy updating method and evaluation system based on game confrontation. Background Art

[0002] Multi-agent systems have been widely used in various fields such as economy and military, and have received increasing attention and research in recent years. This field is dedicated to studying the interaction and strategic games between multiple agents in competitive and cooperative environments. In this system, agents interact with each other, each agent pursues its own goals, and its behavior and decision-making are affected by other agents. The game-adversarial nature of the multi-agent system means that there is competition and conflict between agents. By choosing different strategies, agents can try to optimize their own utility functions, while also considering the behavior of other agents. This dynamic of competition and cooperation has applications in many fields in the real world, such as financial markets, resource allocation, traffic flow, etc. In this field, researchers explore various strategy update models, game equilibrium, cooperation and competition, and other issues to promote the learning and decision-making capabilities of agents in adversarial environments.

[0003] The multi-agent system based on game confrontation is dedicated to exploring the strategic selection and interactive behavior of multiple agents in a competitive and cooperative environment. In this system, agents influence each other, and each agent pursues its own interests by choosing different strategies, while also being affected by the strategies of other agents. This adversarial environment triggers a game of strategy selection, and agents need to weigh their own interests and the behaviors of other agents to achieve the best decision. The multi-agent system based on game confrontation covers multiple research fields and methods involving competition and cooperation between agents, such as multi-agent game theory, evolutionary game theory, group behavior simulation, multi-agent social network, adversarial reinforcement learning, group decision-making and cooperation, multi-agent competition and collaboration, evolutionary game dynamics, Stackelberg safety game, etc. These fields and methods all involve game and decision-making problems between agents in multi-agent systems, thus promoting the development of multi-agent system research. The intersection and integration between different fields also provide more research and application opportunities for multi-agent systems.

[0004] The strategy updating model of multi-agent system is an important field to study how multi-agents adjust and improve their strategies to adapt to environmental changes and opponent behaviors. In the past few decades, the research on multi-agent strategy updating has been continuously deepened, and many innovative methods and theoretical models have emerged. Some common multi-agent system strategy updating models include comparative updating, imitative updating, Fermi updating, Moran process, genetic algorithms, reinforcement learning, evolutionary game, etc.

[0005] Comparative updating is a commonly used stochastic process to describe the rules for updating the strategy of an agent. In comparative updating, two objects A and B are randomly selected, one of which is the imitation object of the other. The probability that A accepts B's strategy as its own strategy is P, which is a linear function of the benefits of the two agents, and:

[0006]

[0007] Where ω is the selection strength, representing the effect of neutral drift, π f and π r are the benefits of the two agents, and Δπ is the maximum benefit difference between the agents.

[0008] Imitation updating is a strategy updating method based on imitating the behavior of others. In imitation learning, one or more agents try to learn strategies by observing the behavior of other agents or experts. The basic idea is that agents can learn from existing excellent strategies instead of obtaining strategies through trial and error and autonomous exploration. Imitation learning can be divided into the following types: Behavioral Cloning: The agent directly copies the behavior of the expert and tries to learn the mapping relationship from the observed input to the output. The challenge of this method is that it may not be able to cope with situations that have not been seen during training; Inverse Reinforcement Learning: Inverse reinforcement learning observes the behavior of experts and infers the goals and preferences pursued by experts, so as to learn more general strategies. This method can adapt to unknown environments and tasks; Cooperative Imitation Learning: Multiple agents learn cooperative strategies by imitating each other's behavior to achieve better synergy. This method is particularly useful in multi-agent systems.

[0009] The Fermi update is derived from the comparative update. The difference from the comparative update is that the update rule usually describes P as a nonlinear function of the return. The common form is:

[0010]

[0011] When the selection intensity ω<<1, P is asymptotically the difference in return π f -π r is a linear function of ω, and when ω→∞, the process is close to mimicking dynamics.

[0012] The Moran process was first introduced into game theory by Taylor et al. in 2004 and quickly attracted the attention of researchers. Scholars have studied the Moran process from various perspectives. The Moran process is divided into two steps. An individual is selected with a probability proportional to its fitness and produces an offspring, which inherits the strategy selection of the parent. In order to keep the entire population unchanged, another randomly selected individual will be removed. The Moran process shows a simple life and death process. The fitness of an individual can be defined as a function of different forms of benefits. The general form is:

[0013] f=1-ω+ωπ,

[0014] Among them, ω is the selection intensity and π is the current individual's benefit.

[0015] In addition, there are some policy update models that have been widely used in recent years, such as the policy update model based on genetic algorithms, which simulates the process of biological evolution and optimizes the strategy of the agent through operations such as selection, crossover and mutation. This method can find better solutions in a larger policy space. At the same time, reinforcement learning is also a common policy update method. The agent updates the strategy based on the feedback of the environment to obtain higher cumulative rewards. In multi-agent systems, it can be extended to multi-agent scenarios through methods such as game theory. However, as a powerful machine learning method, reinforcement learning also has some disadvantages. For example, reinforcement learning usually requires a large number of samples to train the agent, especially in complex environments. This makes it very expensive and time-consuming to collect enough samples in practical applications, especially in physical environments in the real world. And reinforcement learning algorithms usually require a long training time to achieve good performance. The length of training time depends on factors such as environmental complexity, algorithm selection and computing resources. In addition, reinforcement learning algorithms usually have many hyperparameters that need to be adjusted, such as learning rate, discount factor, exploration strategy, etc. Different hyperparameter selections may cause significant changes in algorithm performance, and parameter adjustment may be difficult. Therefore, in some specific scenarios, the strategy update model in traditional game theory can achieve better results than reinforcement learning, and does not require a lot of computing resources. To this end, we proposed a centralized strategy update algorithm that enables multi-agent systems to update strategies in a short time and achieve good evaluation indicators.

[0016] Regardless of the game model, the study of multi-agent game strategy update models and evaluation indicators of multi-agent decision intelligence is crucial. From the perspective of multi-agent game strategy update models, the agents in a multi-agent system influence each other, and their behaviors and strategies interact to produce complex results. The game strategy update model can help analyze the strategy choices between different agents, thereby revealing the stability and equilibrium of the system. At the same time, through the game strategy update model, the decision optimization problems between different agents in terms of resource allocation, task division, etc. can be studied. This helps to improve the overall performance of the system. In addition, the agents in a multi-agent system usually evolve their strategies according to the behavior of their opponents. The game strategy update model can help study the impact of different strategy update methods on the system dynamics. From the perspective of the evaluation indicators of multi-agent decision intelligence, the performance of a multi-agent system depends not only on the performance of the individual, but also on factors such as the degree of collaboration and resource utilization efficiency. Evaluation indicators can quantify the overall performance of the system and help select appropriate decision strategies. Different decision strategies have different effects on system results. Evaluation indicators can help analyze the impact of different decisions on system effects, thereby optimizing decision strategies. Through evaluation indicators, we can understand the impact of different decision-making strategies on resource allocation and utilization, so as to rationally plan resources to maximize system benefits. Therefore, the research on multi-agent game strategy update model and multi-agent decision intelligence evaluation indicators is of great significance in optimizing the performance of multi-agent systems, predicting system behavior, and realizing reasonable decision-making strategies. These studies help us deeply understand the behavior patterns of multi-agent systems and provide a scientific basis for practical applications and decision-making.

[0017] The research on intelligent evaluation criteria for decision-making in multi-agent systems has gradually attracted the attention of scholars at home and abroad. Researchers are committed to exploring issues such as decision-making mechanisms, collaboration strategies, and performance evaluation of multi-agent systems. The capability evaluation indicators of multi-agent systems involve multiple aspects to reflect the performance, efficiency, adaptability, and other characteristics of the system at different levels. Currently, the commonly used multi-agent system capability evaluation indicators include collaboration efficiency, cooperation stability, task allocation effectiveness, decision quality, conflict resolution ability, adaptability, learning ability, utility and gain, system stability, information sharing, environment adaptability, time efficiency, etc. The above evaluation indicators usually have different weights and importance in different application scenarios. In addition, in some specific application scenarios, some evaluation indicators may even lose their evaluation capabilities. Therefore, for specific application scenarios, reasonable evaluation indicators should be formulated to evaluate the intelligent capabilities of multi-agent systems. We have constructed a complete multi-agent system intelligence capability level evaluation system, which comprehensively considers and evaluates the decision-making intelligence capability level of the multi-agent system in the scenario of the present invention from multiple perspectives, thereby verifying the effectiveness of the aforementioned proposed strategy update model compared with the traditional strategy update model. Summary of the invention

[0018] The present invention provides a multi-agent centralized strategy updating method and evaluation system based on game confrontation, which can solve the technical problems in the prior art that the existing model strategy updating time is long and the updating effect is poor.

[0019] According to one aspect of the present invention, a method for updating a multi-agent centralized strategy based on game confrontation is provided, and the method comprises:

[0020] S1, establish a communication undirected graph of our nodes, and assign an ID and a corresponding communication port to each node in the communication undirected graph. The ID is a sequence number. Our nodes include search nodes, tracking nodes, and firepower strike nodes;

[0021] S2, each node exchanges IDs with all its neighboring nodes and determines the direction of the edge between each node and each neighboring node based on the size relationship of the IDs;

[0022] S3, divide the two nodes connected by each edge into incoming neighbor nodes and outgoing neighbor nodes according to the direction of the edge, and divide all nodes into source nodes, receiving nodes and internal nodes according to the direction of the edges between each node and all neighbor nodes;

[0023] S4, each source node transmits its ID to all neighboring nodes, each internal node waits to receive the IDs from all its incoming neighboring nodes, calculates the minimum value among these IDs and sends it to all outgoing neighboring nodes, each receiving node receives the IDs of all its neighboring nodes and calculates the minimum value among these IDs, the receiving node sends YES to the neighboring node that sends the minimum value to it, and sends NO to the remaining neighboring nodes. If all the messages received by an internal node are YES, then the internal node sends YES to all neighboring nodes that have sent messages to it, and sends NO to all neighboring nodes that have not sent messages to it. If an internal node receives a NO, then the internal node sends NO to all neighboring nodes ， The source node waits until all the information is received. If all the information received by a source node is YES, the source node remains a candidate, otherwise the source node is no longer a candidate;

[0024] S5, when a node sends NO to its neighboring node, the direction of the corresponding edge is reversed;

[0025] S6, repeat S3 to S5 until only one node is left as a candidate, and the node with the candidate identity is used as the central node, and the remaining nodes are used as non-central nodes;

[0026] S7, each non-central node determines its own forward node according to the central node, and the central node obtains the path for each non-central node to send a message to the central node based on the forward node;

[0027] S8, generating an empty graph based on our nodes and enemy nodes, connecting edges on the empty graph according to the information of the enemy nodes, and generating a tracking and attacking graph in combination with the communication undirected graph of our nodes;

[0028] S9, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path through which each non-central node sends messages to it, the tracking strike map, the tracking nodes and the fire strike nodes, and generates the final topology structure based on the generated strike chain.

[0029] Furthermore, determining the direction of the edge between each node and each neighboring node according to the size relationship of the ID includes:

[0030] Compare the IDs of the current node and its neighboring nodes. If the ID of the current node is smaller than the ID of the neighboring node, the direction of the corresponding edge is from the current node to the neighboring node. If the ID of the current node is larger than the ID of the neighboring node, the direction of the corresponding edge is from the neighboring node to the current node.

[0031] Furthermore, dividing the two nodes connected by each edge into incoming neighbor nodes and outgoing neighbor nodes according to the direction of the edge includes:

[0032] If the direction of the edge is from the current node to the neighboring node, the neighboring node is the outgoing node of the current node, and the current node is the incoming node of the neighboring node. If the direction of the edge is from the neighboring node to the current node, the current node is the outgoing node of the neighboring node, and the neighboring node is the incoming node of the current node.

[0033] Furthermore, all nodes are divided into source nodes, receiving nodes and internal nodes according to the directions of the edges between each node and all neighboring nodes, including:

[0034] If the direction of the edges between a node and all its neighboring nodes is from the node to the corresponding neighboring node, then the node is the source node;

[0035] If the direction of the edges between a node and all its neighboring nodes is that the corresponding neighboring nodes are facing the node, then the node is a receiving node;

[0036] If a node is neither a source node nor a receiving node, it is an internal node.

[0037] Furthermore, establishing a communication undirected graph of our node and assigning an ID and a corresponding communication port to each node in the communication undirected graph includes: dividing the communication undirected graph into multiple sub-communication undirected graphs, and assigning an ID and a corresponding communication port to each node in each sub-communication undirected graph.

[0038] Furthermore, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path of each non-central node sending a message to it, the tracking strike map, the tracking node and the fire strike node, including:

[0039] For the current enemy node that can be covered by the search node, the tracking nodes and fire strike nodes are fully arranged and traversed, and each pair of track-->firing chains that can cover the current enemy target are traversed. Then, the shortest path of track-->firing is searched in the tracking and firing graph, and it is determined whether the firing chain is feasible. If feasible, the nodes are connected in the empty graph, and the traversal continues for the next enemy node. If not feasible, the next pair of track-->firing firing chains are traversed for the current enemy node.

[0040] Further, the method includes: when it is impossible to obtain information about real enemy nodes, obtaining information about imaginary enemy nodes by sampling a given enemy target probability distribution multiple times, S8 includes:

[0041] According to the information of the imaginary enemy nodes obtained in each sampling, the edges are connected on the empty graph, and the corresponding tracking and attack graph is generated in combination with the communication undirected graph of our nodes.

[0042] Furthermore, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path of each non-central node sending a message to it, the tracking strike map, the tracking node and the fire strike node, including:

[0043] For the information of the hypothetical enemy node obtained by each sampling, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path through which each non-central node sends messages to it, the tracking and strike map corresponding to the information of the hypothetical enemy node obtained by a single sampling, the tracking node and the fire strike node.

[0044] Furthermore, generating a final topology structure based on the generated strike chain includes:

[0045] The strike chain generated based on the information of the hypothetical enemy node obtained by each sampling generates a corresponding sub-topology structure;

[0046] The number of connections of nodes in all sub-topological structures is statistically sorted, and the edges in the empty graph are connected in descending order to obtain the final topological structure.

[0047] According to another aspect of the present invention, an evaluation system for evaluating the centralized strategy update method proposed in the present invention is provided, the evaluation system comprising a graph connectivity evaluation module, a task matching evaluation module, a destruction rate evaluation module, a task coverage evaluation module and a resilience and anti-destruction evaluation module;

[0048] The graph connectivity evaluation module is used to evaluate the connectivity capability of the multi-agent topology structure updated by the centralized strategy update method; the task matching evaluation module is used to evaluate the potential ability of the multi-agent topology structure updated by the centralized strategy update method to destroy enemy targets; the destruction rate evaluation module is used to evaluate the ability of the multi-agent topology structure updated by the centralized strategy update method to perform destruction tasks; the task coverage evaluation module is used to evaluate the rationality of the multi-agent topology structure updated by the centralized strategy update method; and the elastic anti-destruction evaluation module is used to evaluate the anti-strike capability of the multi-agent topology structure updated by the centralized strategy update method.

[0049] Furthermore, the graph connectivity evaluation module is used to evaluate the connectivity of the multi-agent topology structure updated by the centralized strategy update method according to the following formula:

[0050] C=w1C1+w2C2,

[0051] In the above formula, C represents the connectivity, C1 represents the number of connected branches in the multi-agent topology structure, C2 represents the average in-degree and out-degree of nodes in the multi-agent topology structure, w1<0, represents the weight coefficient of the number of connected branches, and w2>0, represents the weight coefficient of the average in-degree and out-degree of nodes.

[0052] Furthermore, the task matching evaluation module is used to evaluate the potential ability of the multi-agent topology structure updated by the centralized strategy updating method to destroy the enemy target according to the following formula:

[0053] K=a1K1+a2K2,

[0054]

[0055] In the above formula, K represents the task matching degree, K1 represents the average number of kill chains of enemy targets, K2 represents the average length of kill chains, a1>0 represents the weight coefficient of the average number of kill chains of enemy targets, a2<0 represents the weight coefficient of the average length of kill chains, N c Represents the total number of kill chains of enemy targets, and M represents the number of enemy targets.

[0056] Furthermore, the destruction rate evaluation module is used to evaluate the ability of the multi-agent topology structure updated by the centralized strategy update method to perform the destruction task according to the following formula:

[0057]

[0058] In the above formula, D represents the destruction rate, and m represents the number of enemy targets destroyed by the multi-agent topology after the mission is completed.

[0059] Furthermore, the task coverage evaluation module is used to evaluate the rationality of the multi-agent topology structure updated by the centralized strategy update method according to the following formula:

[0060]

[0061] In the above formula, T represents the task coverage, represents the number of enemy targets that the multi-agent topology can destroy, Indicates the number of enemy targets that can be monitored.

[0062] Furthermore, the elastic anti-destruction evaluation module is used to evaluate the anti-strike capability of the multi-agent topological structure updated by the centralized strategy updating method according to the following formula:

[0063]

[0064] In the above formula, R represents resilience and indestructibility, C, K, D, and T represent the connectivity, task matching, destruction rate, and task coverage of the multi-agent topological structure before a certain proportion of nodes are damaged, respectively; C', K', D', and T' represent the connectivity, task matching, destruction rate, and task coverage of the multi-agent topological structure after a certain proportion of nodes are damaged, respectively.

[0065] By applying the technical solution of the present invention, a multi-agent centralized strategy updating method and evaluation system based on game confrontation is provided. The method establishes a communication undirected graph of our nodes, assigns an ID and a communication port to each node in the communication undirected graph, selects a central node based on the node ID, and determines the path for sending messages between non-central nodes and the central node. The central node can be used to summarize local information, thereby formulating more reasonable strategies in a short time, which can improve the decision-making intelligence ability of the multi-agent system to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The included drawings are used to provide a further understanding of the embodiments of the present invention, which constitute a part of the specification, are used to illustrate the embodiments of the present invention, and together with the text description, explain the principles of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 A schematic diagram of the process of a multi-agent centralized strategy updating method based on game confrontation provided according to a specific embodiment of the present invention is shown;

[0068] Figure 2 A comparison update schematic diagram provided according to a specific embodiment of the present invention is shown;

[0069] Figure 3 A schematic diagram of Fermi update according to a specific embodiment of the present invention is shown;

[0070] Figure 4 It shows a schematic diagram of imitation update provided according to a specific embodiment of the present invention;

[0071] Figure 5 A schematic diagram of a leader election algorithm provided according to a specific embodiment of the present invention is shown. DETAILED DESCRIPTION

[0072] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0074] Unless otherwise specifically stated, the relative arrangement of the parts and steps described in these embodiments, numerical expressions and numerical values ​​do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be regarded as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0075] like Figure 1 As shown, according to a specific embodiment of the present invention, a multi-agent centralized strategy updating method based on game confrontation is provided, and the method includes:

[0076] S1, establish a communication undirected graph of our nodes, and assign an ID and a corresponding communication port to each node in the communication undirected graph. The ID is a sequence number. Our nodes include search nodes, tracking nodes, and firepower strike nodes;

[0077] S2, each node exchanges IDs with all its neighboring nodes and determines the direction of the edge between each node and each neighboring node based on the size relationship of the IDs;

[0078] S3, divide the two nodes connected by each edge into incoming neighbor nodes and outgoing neighbor nodes according to the direction of the edge, and divide all nodes into source nodes, receiving nodes and internal nodes according to the direction of the edges between each node and all neighbor nodes;

[0079] S4, each source node transmits its ID to all neighboring nodes, each internal node waits to receive the IDs from all its incoming neighboring nodes, calculates the minimum value among these IDs and sends it to all outgoing neighboring nodes, each receiving node receives the IDs of all its neighboring nodes and calculates the minimum value among these IDs, the receiving node sends YES to the neighboring node that sends the minimum value to it, and sends NO to the remaining neighboring nodes. If all the messages received by an internal node are YES, then the internal node sends YES to all neighboring nodes that have sent messages to it, and sends NO to all neighboring nodes that have not sent messages to it. If an internal node receives a NO, then the internal node sends NO to all neighboring nodes ， The source node waits until all the information is received. If all the information received by a source node is YES, the source node remains a candidate, otherwise the source node is no longer a candidate;

[0080] S5, when a node sends NO to its neighboring node, the direction of the corresponding edge is reversed;

[0081] S6, repeat S3 to S5 until only one node is left as a candidate, and the node with the candidate identity is used as the central node, and the remaining nodes are used as non-central nodes;

[0082] S7, each non-central node determines its own forward node according to the central node, and the central node obtains the path for each non-central node to send a message to the central node based on the forward node;

[0083] S8, generating an empty graph based on our nodes and enemy nodes, connecting edges on the empty graph according to the information of the enemy nodes, and generating a tracking and attacking graph in combination with the communication undirected graph of our nodes;

[0084] S9, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path through which each non-central node sends messages to it, the tracking strike map, the tracking nodes and the fire strike nodes, and generates the final topology structure based on the generated strike chain.

[0085] By applying this configuration method, a multi-agent centralized strategy update method based on game confrontation is provided. The method establishes a communication undirected graph of our nodes, assigns an ID and a communication port to each node in the communication undirected graph, selects a central node based on the node ID, and determines the path for sending messages between non-central nodes and the central node. The central node can be used to summarize local information, thereby formulating a more reasonable strategy in a short time, and can improve the decision-making intelligence ability level of the multi-agent system to a certain extent. Compared with the prior art, the technical solution of the present invention can solve the technical problems of the long update time and poor update effect of the existing model strategy in the prior art.

[0086] Further, in an embodiment of the present invention, determining the direction of the edge between each node and each neighboring node according to the size relationship of the ID includes: comparing the size of the ID of the current node and its neighboring node, if the ID of the current node is smaller than the ID of the neighboring node, then the direction of the corresponding edge is from the current node to the neighboring node, if the ID of the current node is larger than the ID of the neighboring node, then the direction of the corresponding edge is from the neighboring node to the current node. Dividing the two nodes connected by each edge into incoming neighboring nodes and outgoing neighboring nodes according to the direction of the edge includes: if the direction of the edge is from the current node to the neighboring node, then the neighboring node is the outgoing node of the current node, and the current node is the incoming node of the neighboring node, if the direction of the edge is from the neighboring node to the current node, then the current node is the outgoing node of the neighboring node, and the neighboring node is the incoming node of the current node.

[0087] Based on the above embodiments, in an embodiment of the present invention, all nodes are divided into source nodes, receiving nodes and internal nodes according to the direction of the edges between each node and all neighboring nodes, including: if the direction of the edges between a node and all its neighboring nodes is that the node is toward the corresponding neighboring node, then the node is a source node; if the direction of the edges between a node and all its neighboring nodes is that the corresponding neighboring node is toward the node, then the node is a receiving node; if a node is neither a source node nor a receiving node, then the node is an internal node.

[0088] That is to say, in the centralized update strategy proposed by the present invention, a key issue is to select a connected branch center for each connected branch in the topological structure formed by the multi-agent system. As a specific embodiment of the present invention, the present invention adopts the classic Y o -y oThe leader election algorithm determines the central node. This algorithm has leading performance in both data transmission volume and time consumption. In distributed computing, leader election is the process of designating a process as the organizer of certain tasks distributed among multiple computers (nodes). Before the task begins, all nodes either do not know which node will act as the "leader" of the task or cannot communicate with the current leader. However, after applying the leader election algorithm, every node in the entire network will identify a specific, unique node as the task leader.

[0089] In the practical application scenario of the present invention, the main idea is to construct a spanning tree on the connected graph and select its root as the leader, such as Figure 5 As shown, it specifically includes two stages: preprocessing stage and a series of iterations. Assume that our node N = {N1, N2, ..., N n}Node N i The ID of each node N is i. i With all its neighboring nodes N c = {N i1 , N i2 , ..., N im}Exchange their IDs and determine the direction of their associated edges based on the IDs. For example, if node N i The ID of node N is smaller than j , then node N i Towards node N j , at this time node N j Node N i The outgoing neighbor node, node N i Node N j If a node N i If the ID of a node N is smaller than the IDs of all its neighboring nodes, it is called a source node. i If the ID of a node is greater than the IDs of all its neighboring nodes, it is called a sink node. All nodes that are neither source nodes nor sink nodes are called internal nodes. When all edges are directed, the second part of the iteration begins. Each iteration is an election, and some candidates will be removed. Each iteration is divided into two phases: the forward phase and the backward phase. The process is as follows:

[0090] Forward phase: 1. First, each source node transmits its ID to all neighboring nodes; 2. The internal node waits to receive IDs from all its incoming neighboring nodes, it calculates the minimum of these values ​​and sends it to all outgoing neighboring nodes; 3. The receiving node receives all IDs and calculates their minimum.

[0091] Backward phase: 1. The receiving node sends YES to the neighboring node that sends the minimum value, and sends NO to other nodes; 2. When all the messages received by the internal node are YES, the internal node sends YES to all neighboring nodes that have sent messages to it, and sends NO to other neighboring nodes. If an internal node receives a NO, it will send NO to all neighboring nodes; 3. The source node will wait until it receives all the information. If all the information it receives is YES, it will remain a candidate, otherwise it will no longer be a candidate; 4. When node N i To neighboring node N j When NO is sent, the direction of the edge is reversed.

[0092] This iterative process continues until only one candidate node remains. The iteration ends and the candidate node becomes the central node. After the central node is elected, each non-central node needs to determine its own forward node. The role of the forward node is to help the non-central node pass messages to the central node, that is, each node needs to send all its information to the forward node, and then continue to pass it until the information is passed to the central node. In this process, the central node first broadcasts the message to all neighboring nodes, and then the node that receives the message broadcasts the message to all its neighboring nodes again. Each node only broadcasts the message once after receiving it, and will no longer propagate the message after receiving it again. The node that receives the first message from each node is the forward node of the node.

[0093] After determining the forward node of each node, it is equivalent to determining the path for each node to send messages to the central node. However, the central node also needs to know the path to send messages to each node. In this process, each node sends its node name to the forward node. The forward node adds its own node name to the received node name and sends it to its forward node until it is sent to the central node. The information received by the central node is the path to each node, and the election of the central node is completed.

[0094] However, when constructing a complete multi-agent system topology, the topology does not always form a connected graph. In this case, there will be multiple connected branches in the topology. At this time, a connected branch center is selected for each connected branch. Each node of the connected branch transmits the monitored enemy unit information to the connected branch center. The connected branch center summarizes all information and formulates a strategy for each enemy target. The formulated strategy is transmitted to the nodes of each connected branch, and the nodes of each connected branch then act according to the instructions issued by the connected branch center. This strategy evolution method is called the centralized strategy evolution method.

[0095] At the beginning of the task, any node N iThey all know the location of the central node and will mess their own messages. i , such as the location of the monitored enemy node and other information is sent to the central node. At this time, the central node will have global information and start to formulate strategies. The process of strategy formulation is as follows: for each enemy target, the central node calculates a shortest attack link from the current topology map to attack the enemy target, and then the central node sends a connection command to each of our nodes on this link. After receiving the connection command, our node connects until all enemy targets are traversed. The strategy formulation is completed, and the central node then passes the specified strategy to each node.

[0096] In the existing three policy updating methods of comparison updating, Fermi updating and imitation updating, the policy of each node is determined by itself, that is, which nodes to connect is determined only by the node itself through the corresponding policy updating method. At this time, the node can only obtain all its own information, so it has limitations when updating node policies. In the centralized policy updating method proposed in the present invention, the connected branch center obtains the information of all nodes of the connected branch and then uniformly allocates policies. Instead of each node making decisions on its own, the connected branch center can obtain local or even global information, so it can be more reasonable and effective when allocating policies to each node.

[0097] Further, after the central node is selected for each connected subgraph by the leader election algorithm, if there is a connected subgraph with a particularly large number of nodes, the time complexity of the subsequent central node calculation connection strategy will be very high. Therefore, in order to be able to perform more efficient parallel computing, in an embodiment of the present invention, before the leader election algorithm is used to elect the central node, the communication graph is divided into many small subgraphs by a graph partitioning algorithm, that is, the communication undirected graph of our node is established, and each node in the communication undirected graph is assigned an ID and a corresponding communication port, including: dividing the communication undirected graph into multiple sub-communication undirected graphs, and assigning an ID and a corresponding communication port to each node in each sub-communication undirected graph. As a specific embodiment of the present invention, the number of nodes in each subgraph does not exceed 150, and the central node election algorithm is run on each subgraph after segmentation. The key part of the graph segmentation algorithm calls the part_graph function in the pymetis package in python, which can divide a given connected graph into a specific number of subgraphs required. After performing the graph partitioning algorithm, assuming that we have obtained n small connected subgraphs, we perform the leader election algorithm on these n small connected subgraphs to select the central node of each small connected subgraph. After that, we can calculate the connection strategy of each small connected subgraph in parallel on these n small connected subgraphs, which can greatly optimize the time complexity of formulating the connection strategy.

[0098] Based on the above embodiment, in the embodiment of the present invention, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path of each non-central node sending a message to it, the tracking strike graph (full_fight_graph), the tracking node and the fire strike node, including: for the current enemy node that can be covered by the search node, the tracking node and the fire strike node are fully arranged and traversed, and each pair of track-->firing chains that can cover the current enemy target are traversed, and then the shortest path of track-->firing is searched in the tracking strike graph, and it is judged whether the strike chain is feasible. If it is feasible, the node connection is performed in the empty graph (empty_graph), and the traversal is continued for the next enemy node. If it is not feasible, the next pair of track-->firing strike chains are continued to be traversed for the current enemy node until all enemy targets are traversed, and the decision algorithm of the centralized update strategy proposed by the present invention is completed. Among them, the tracking strike graph is used to search for the shortest path. When generating the tracking strike graph, all the edges that can be connected on the graph are connected, and the restrictions of slot and limit are not considered first.

[0099] Furthermore, in the above-mentioned centralized update strategy algorithm, a kill chain can only be established if all information about all enemy targets (enemy nodes) is known, which is named the traditional centralized update strategy algorithm. However, in real scenarios, we may not be able to obtain all information about enemy targets in advance. Therefore, the present invention further proposes a centralized update strategy based on probability graph distribution for generating a strike chain on an imaginary enemy target sampled based on a given probability distribution. That is, the method includes: when the information of the real enemy node cannot be obtained, the information of the imaginary enemy node is obtained by multiple sampling of the given enemy target probability distribution, and S8 includes: connecting edges on the empty graph according to the information of the imaginary enemy node obtained by each sampling, and generating a corresponding tracking and strike graph in combination with the communication undirected graph of our node.

[0100] The present invention considers two different centralized update strategy algorithms based on probabilistic graph distribution. The first is a centralized update strategy algorithm based on probabilistic graph distribution inspired by voting theory, and the second is a centralized update strategy algorithm based on probabilistic graph distribution inspired by Shapley value theory. We will introduce the implementation details of these two algorithms in detail below.

[0101] The decision algorithm process of the centralized update strategy based on the probability graph distribution inspired by the voting theory is as follows. First, generate sample_number enemy target samples. For each sampled enemy sample, a topological structure of our side can be obtained by using the decision algorithm of the aforementioned centralized update strategy. Therefore, a total of sample_number of our topological structures can be generated. The number of all connected edges in these sample_number of our topological structures is counted, and all connected edges are sorted from high to low according to the number of connections. Then, these edges are connected in our topological structure in turn until all edges are connected or no edges can be connected. Finally, our final topological structure is obtained. After obtaining our final topological structure, all track-->firing chains in the architecture need to be found and stored in the file as potential strike chains for use in subsequent kill chain generation modules and elastic anti-destruction modules.

[0102] That is to say, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path through which each non-central node sends a message, the tracking strike map, the tracking node and the fire strike node. For the information of the imaginary enemy node obtained by each sampling, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path through which each non-central node sends a message, the tracking strike map corresponding to the information of the imaginary enemy node obtained by a single sampling, the tracking node and the fire strike node. The final topological structure is generated based on the generated strike chain, including: generating sub-topological structures corresponding to the strike chains generated based on the information of the imaginary enemy node obtained by each sampling; statistically sorting the number of connections of the nodes in all sub-topological structures, and connecting the edges in the empty graph in descending order to obtain the final topological structure.

[0103] The decision algorithm process of the centralized update strategy based on probability graph distribution inspired by Shapley value theory is roughly the same as the decision algorithm process of the centralized update strategy based on probability graph distribution inspired by voting theory. The only difference is that in the decision algorithm process of the centralized update strategy based on probability graph distribution inspired by Shapley value theory, for each of our topological structures generated by the enemy sample, we count the number of strike chains participated by each edge in the structure as the marginal benefit contributed by this edge. Then count the number of strike chains participated by all connected edges in this sample_number of our topological structures, and sort all connected edges from high to low according to the number of times they participate in the strike chain. Then, connect these edges in our topological structure in turn until all edges are connected or no edges can be connected. Finally, we get our final topological structure. Among them, the generation method of tracking strike graph, the generation method of strike chain, and the generation method of topological structure in the centralized update strategy algorithm based on probability graph distribution can refer to the relevant exemplary description in the aforementioned traditional centralized update strategy algorithm, which will not be repeated here.

[0104] According to another aspect of the present invention, there is provided an evaluation system for evaluating the centralized strategy update method proposed above in the present invention, the evaluation system comprising a graph connectivity evaluation module, a task matching evaluation module, a kill rate evaluation module, a task coverage evaluation module and a resilient anti-destruction evaluation module; the graph connectivity evaluation module is used to evaluate the connectivity capability of the multi-agent topology structure updated by the centralized strategy update method, the task matching evaluation module is used to evaluate the potential capability of the multi-agent topology structure updated by the centralized strategy update method to destroy enemy targets, the kill rate evaluation module is used to evaluate the capability of the multi-agent topology structure updated by the centralized strategy update method to execute the kill mission, the task coverage evaluation module is used to evaluate the rationality of the multi-agent topology structure updated by the centralized strategy update method, and the resilient anti-destruction evaluation module is used to evaluate the anti-strike capability of the multi-agent topology structure updated by the centralized strategy update method. Specific descriptions are as follows:

[0105] 1) Connectivity

[0106] Connectivity is determined by the number of connected branches and the average in-and-out degree of nodes. Its value is the weighted average of the number of connected branches and the average in-and-out degree of nodes. In other words, the graph connectivity evaluation module is used to evaluate the connectivity of the multi-agent topology structure updated by the centralized strategy update method according to the following formula:

[0107] C=w1C1+w2C2,

[0108] In the above formula, C represents the connectivity, C1 represents the number of connected branches in the multi-agent topology structure, C2 represents the average in-degree and out-degree of nodes in the multi-agent topology structure, w1<0, represents the weight coefficient of the number of connected branches, and w2>0, represents the weight coefficient of the average in-degree and out-degree of nodes.

[0109] The larger the connectivity index C is, the better the connectivity performance of the topological structure of the multi-agent system is. In the evaluation experiment, the connectivity is calculated by the connectivity function in utils.analysers.dynamic_analyser.py, where w1=-0.1 and w2=0.5.

[0110] 2) Task matching

[0111] The task matching degree is determined by the average number of kill chains and the average length of the kill chains, and is used to measure the potential ability of the multi-agent system to destroy the enemy target. The task matching degree is the weighted average of the average number of kill chains of the enemy target and the average length of the kill chain. In other words, the task matching degree evaluation module is used to evaluate the potential ability of the multi-agent topology structure updated by the centralized strategy update method to destroy the enemy target according to the following formula:

[0112] K=a1K1+a2K2,

[0113]

[0114] In the above formula, K represents the task matching degree. The larger its value is, the stronger the potential ability of the multi-agent system to destroy the enemy target is. K1 represents the average number of kill chains of the enemy target, K2 represents the average length of the kill chain, a1>0 represents the weight coefficient of the average number of kill chains of the enemy target, a2<0 represents the weight coefficient of the average length of the kill chain, N c Represents the total number of kill chains of enemy targets, and M represents the number of enemy targets.

[0115] 3) Destruction rate

[0116] The kill rate indicates the ratio of the multi-agent system to destroy enemy targets after the mission is completed, which can effectively measure the ability of the multi-agent system to perform the kill mission. In the embodiment of the present invention, the kill rate evaluation module is used to evaluate the ability of the multi-agent topology structure updated by the centralized strategy update method to perform the kill mission according to the following formula:

[0117]

[0118] In the above formula, D represents the destruction rate, and m represents the number of enemy targets destroyed by the multi-agent topology after the mission is completed.

[0119] 4) Task coverage

[0120] Task coverage is the ratio of the number of enemy targets that the multi-agent system can destroy to the number of enemy targets that it can monitor at the current moment, which is used to measure the rationality of the topological structure formed by the multi-agent system. In other words, the task coverage evaluation module is used to evaluate the rationality of the multi-agent topological structure updated by the centralized strategy update method according to the following formula:

[0121]

[0122] In the above formula, T represents the task coverage, represents the number of enemy targets that the multi-agent topology can destroy, Represents the number of enemy targets that can be monitored. Task coverage T∈[0,1], the larger the task coverage, the more reasonable the topological architecture of the multi-agent system.

[0123] 5) Elastic anti-destruction

[0124] Resilient indestructibility indicates the anti-attack capability of the multi-agent system, that is, the ability of the multi-agent system to maintain after suffering a certain attack. It is the maintenance ratio of the above four indicators after a certain proportion of nodes are damaged (random damage or targeted maximum node damage) at the current moment. In other words, the elastic indestructibility evaluation module is used to evaluate the anti-attack capability of the multi-agent topology structure updated by the centralized strategy update method according to the following formula:

[0125]

[0126] In the above formula, R represents resilience and indestructibility, the larger the value, the better; C, K, D, and T respectively represent the connectivity, task matching, destruction rate, and task coverage of the multi-agent topological structure before a certain proportion of nodes are damaged; c′, K′, D′, and T′ respectively represent the connectivity, task matching, destruction rate, and task coverage of the multi-agent topological structure after a certain proportion of nodes are damaged.

[0127] Furthermore, in an embodiment of the present invention, in view of the problem of generating system architecture for special scenarios, a software system is developed based on the above-mentioned evaluation system to independently evaluate the decision-making intelligence capability under system games. The number of system nodes supported by the software system should be no less than 10,000 (including at least 50 clusters with no less than 100 nodes), and the scenario assumptions can be loaded through a predefined data interface, and the game strategy model that meets the interface requirements can be accessed. For the game strategy combination in the scenario, the decision-making intelligence capability evaluation index proposed by us can be efficiently calculated, and the system architecture generation process and results as well as the game confrontation process can be simply displayed in a visual way. This set of multi-agent system decision-making intelligence capability level evaluation index system can reasonably and effectively evaluate the decision-making intelligence capability level of the multi-agent system.

[0128] Based on the above evaluation system, the application scenario of the multi-agent centralized strategy updating method based on game confrontation proposed in the present invention can be modeled as follows:

[0129] Identify participants: In the game confrontation scenario of this system, the participants are various nodes of our side and enemy aircraft nodes. Among them, various nodes of our side include search nodes, tracking nodes, command and control nodes, and firepower strike nodes.

[0130] Determine the strategy: In game theory, strategy refers to the way players choose actions. i ={a1, a2, ..., a n} is the player (node) N i The set of actions that can be selected, let s i (a i ) is player N i Select action a i The probability of s i (a i )satisfy:

[0131]

[0132] If there is an action a i , so that s i (a i )=1, then s i is a pure strategy, otherwise it is called s i For all kinds of our nodes, since our goal is to build a reasonable system to resist the attack of enemy nodes, it is very important to explore how to construct the connection communication between nodes. Assume that our node N i The nodes that can be connected are {N i1 , N i2 , ..., N im}, define node N i The set of actions available is A i ={a i1 , a i2 , ..., a im}, where a ij Represents node N i Connect to node N ij , let s i (a ij ) represents our node N i Select action a ij The probability of s i (a ij )satisfy:

[0133]

[0134] Determine our node N i The strategy is essentially to determine s i Therefore, for each of our nodes, the strategy refers to how to determine the connection between nodes. For example, for a given node, it can choose whether to connect to other nodes that can be connected within its range, and how to reconnect between nodes after the relevant nodes are damaged.

[0135] For the enemy node, since the enemy's goal is to destroy our node, it is assumed that the enemy node E i The nodes that can be attacked are {N1, N2, ..., N n ), define the enemy node E i The set of actions that can be selected is B i ={b1, b2, ..., b n}, where b i Represents the enemy node E i Choose to attack our node N i , let s i (b i ) represents the enemy node E i Select action b i The probability of , then satisfies:

[0136]

[0137] Therefore, the strategy of each enemy node refers to the way to determine the nodes to be attacked, that is, the probability of each enemy node attacking each of our nodes.

[0138] Determine the utility: For our nodes, the utility is calculated using the established evaluation index system, such as connectivity, task matching, task coverage, destruction rate, and resilience. The greater the above index, the higher the utility we obtain. Let our node utility be U. For enemy nodes, the number of our nodes destroyed is used to represent the utility of the enemy node, that is, the more our nodes destroyed, the greater the utility of the enemy node. Let the enemy node utility be V.

[0139] In the game confrontation scenario of this system, its essence is a multi-round game process. In each round, the enemy node will move one unit to the destination. After that, the enemy node and our node can choose a strategy to play the game to obtain higher utility. Therefore, the game tree can be used to construct the detailed process of the game. At the beginning of the game, in the first round, our node determines the node connection strategy s i (a ij ), the way to connect nodes, let N = {N1, N2, ..., N n} represents each of our nodes, Edge = {e ij|i=1,2,...,n;j=1,2,...,n} is the connection strategy, where e ij =1 means our node N i and our node N j Connected, e ij =0 means our node N i and our node N j N and Edge constitute the topology of our system. The enemy node selects the attack strategy s i (b i ), that is, the way to attack our node, let E = {E1, E2, ..., E m} represents each enemy node, Attack = {a ij |i=1,2,...,m;j=1,2,...,n} represents the attack target of each enemy node, where a ij =1 indicates the enemy node E i Will attack our node N j , a ij =0 indicates the enemy node E i Will not attack our node N j In each subsequent round, our nodes can attack enemy nodes through the established multi-agent system, and enemy nodes can also attack our nodes within the attack range or move one unit toward the selected target node. At the same time, in each round, our nodes can calculate their own utility U according to the real-time situation and choose to update the connection strategy s i (a ij ) to establish a more reasonable and effective multi-agent system. As the confrontation game continues, our nodes will continue to attack the enemy nodes within the firepower range. Similarly, the enemy nodes will continue to attack our target nodes within their firepower range until all the nodes of one party are destroyed or all the enemy nodes reach the destination, and the system game confrontation ends. In the end, the utility U obtained by our nodes and the utility V obtained by the enemy nodes can be calculated, and the intelligence level of our multi-agent system can be evaluated based on the final results and the data during the game.

[0140] After constructing a complete multi-agent system intelligence capability level evaluation system, the present invention proposes to evaluate the multi-agent system intelligence capability under two different game paradigms, namely, cooperative game paradigm and competitive game paradigm.

[0141] In the cooperative game paradigm, our nodes are in a cooperative relationship, and each node hopes to connect to more nodes with high degrees so that it can connect with more nodes. Therefore, in the cooperative game paradigm, we can define node N iThe utility U of the node is the degree D of the node, that is: U(N i )=D(N i ). At this point, each node can connect to nodes with higher degrees based on the utility of surrounding nodes.

[0142] In the competitive game paradigm, there is a competitive relationship between our nodes. Each node hopes to connect to more nodes, but at the same time does not make itself a key node to avoid being destroyed by the enemy. Therefore, in the adversarial game paradigm, when connecting nodes, it will first determine whether its degree has exceeded the maximum degree. For example, for node N i , assuming that the nodes to be connected are {N i1 , N i2 , ..., N im ), then determine whether it wants to connect to node N ij Follow the following rules:

[0143]

[0144] Furthermore, the present invention calculates the intelligence capability levels of various game update strategy models such as comparative update, imitation update and Fermi update respectively under two different game paradigms according to the existing evaluation index system. The details are as follows:

[0145] like Figure 2 As shown, in the comparative update under the cooperative game paradigm, for node N i , assuming that the nodes that can be connected to it are {N1, N2, ..., N m}, then node N i Connect to node N j The probability of where u i For node N i The degree, u j For node N j The degree of node N j The degree c is a constant, and c=0.5 is set in the experiment. After obtaining the nodes that need to be connected, the node currently being updated needs to first cancel the connection with the nodes that are already connected but not in the nodes that need to be connected, and then connect the nodes that are not yet connected but need to be connected. The probability of connecting each node in the comparative update under the competitive game paradigm is the same as the comparative update under the cooperative game paradigm. The difference is that when connecting nodes, it will first determine whether its degree has exceeded the maximum degree of the surrounding nodes.

[0146] like Figure 3 As shown in the Fermi update under the cooperative game paradigm, for node N i , assuming that the nodes that can be connected to it are {N1, N2, ..., Nm}, then node N i Connect to node N j The probability of where u i For node N i degree, c and α are constants. In the experiment, c=1, α=1. After obtaining the nodes that need to be connected, the node currently being updated needs to first cancel the connection with the nodes that are already connected but not in the nodes that need to be connected, and then connect the nodes that are not yet connected but need to be connected. The probability of connecting each node in the Fermi update under the competitive game paradigm is the same as the comparative update under the cooperative game paradigm. The difference is that when connecting nodes, it will also first determine whether its degree has exceeded the maximum degree of the surrounding nodes.

[0147] like Figure 4 As shown, in the imitation update under the cooperative game paradigm, for node N i , assuming that the nodes that can be connected to it are {N1, N2, ..., N m}, and the utility of these nodes decreases in sequence, that is, u1≥u2≥…≥u m , then node N i Prioritizes connecting nodes with higher degrees and the nodes to which they are connected. After obtaining the order of connecting nodes, the node currently being updated needs to first disconnect the already connected nodes, and then connect the nodes to be connected in sequence until the slot is used up. The order of connecting nodes in the imitation update under the competitive game paradigm is the same as that in the cooperative game paradigm. The difference is that when connecting nodes, it will also first determine whether its degree has exceeded the maximum degree of the surrounding nodes.

[0148] During the evaluation, the kill chain generation module inputs the current topological architecture of our node, the information of the enemy target, and the update strategy used, and outputs all possible kill chains and enemy targets that can be searched and tracked for subsequent indicator calculations.

[0149] If the three traditional update strategies, namely comparison update, Fermi update and imitation update, are used, all feasible kill chains and enemy targets that can be searched are directly calculated.

[0150] If the centralized update strategy proposed in the present invention is used, when calculating all feasible kill chains for the topological structure generated by the centralized update strategy, directly searching for the shortest path to calculate the kill chain for each enemy target will result in very high time complexity. Therefore, in order to reduce the time complexity, we have optimized the kill chain generation module. The specific optimization process is as follows.

[0151] First, we have used the graph partitioning algorithm to partition our topological structure into many small connected subgraphs before the leader election algorithm. Therefore, in the kill chain generation module, we can parallelly calculate the kill chains that can be formed by each subgraph on all small connected subgraphs, which greatly reduces the time complexity of kill chain generation. Secondly, in the decision algorithm based on the centralized update strategy of probabilistic graph distribution, we have found all track-->firing chains in the generated topological architecture and stored them in the file as potential strike chains. Therefore, we do not need to search for the shortest path for each enemy target here, but directly traverse all track-->firing chains. For each track-->firing chain, check whether the enemy target can be hit. If it can be hit, save the kill chain and continue to traverse the next enemy target; if it cannot be hit, continue to traverse the subsequent track-->firing chains until a feasible kill chain is found or all track-->firing chains are traversed. Since all track-->firing chains are saved in advance, we save the shortest path search process, which also greatly reduces the time complexity of the kill chain generation module.

[0152] In addition, during the evaluation, the input of the elastic anti-destruction module is the current topological architecture of our side, the enemy target information, the probability of our node being destroyed, the update strategy adopted, and the game paradigm. The output is the kill chain that can be generated after destroying some of our nodes and the enemy targets that can be searched.

[0153] When the update strategy adopted is one of the three traditional update strategies, namely, comparison update, Fermi update and imitation update, we first randomly generate our actual destroyed nodes according to the probability of our nodes being destroyed. Then, we delete these destroyed nodes and the corresponding connected edges in our topology. Finally, we calculate the new feasible kill chain and the enemy targets that can be searched for the generated new topology through the kill chain generation module.

[0154] When the update strategy adopted is the centralized update strategy, the kill chain that can be generated after destroying some nodes is calculated first. In this module, the time complexity of directly recalculating the kill chain for the topological architecture after destroying some nodes is very high. Therefore, in order to reduce the time complexity, we optimized the elastic anti-destruction module. The specific optimization process is as follows.

[0155] In this module, for each subgraph in our topology, we first randomly generate the nodes that are destroyed in the subgraph. Then, we delete the kill chains with destroyed nodes from the previously saved potential kill chains. Next, we re-detect new kill chains for enemy targets that were previously hit but are now not hit. If we can detect the kill chain, we save it. If we cannot detect the new kill chain, we give up on hitting the enemy target. After gathering the results of all subgraphs, we get the final feasible kill chain and the enemy target that can be tracked.

[0156] The present invention establishes a heterogeneous multi-agent system. Comparative update, imitation update and Fermi update are only applicable to strategy updates between homogeneous multi-agents, but not between heterogeneous multi-agent systems. By adjusting these three game update strategy models under different game paradigms, they can adapt to the strategy updates between heterogeneous multi-agent systems.

[0157] This paper compares the effects and average running time of the centralized update strategy method proposed in this paper and the traditional strategy update method under various evaluation indicators. In this experiment, 1000 friendly nodes and 1000 enemy nodes were randomly generated for confrontation, and the results are shown in the following table.

[0158] Destruction rate Task coverage Task Matching time Compare Updates 0.756 0.551 0.756 2381 seconds Fermi Update 0.778 0.661 0.778 2311 seconds Imitation Update 0.512 -0.35 0.512 4085 seconds Centralized Updates 0.870 1.205 1.0 3.6 seconds

[0159] In the traditional strategy update method, each node makes decisions based only on its own current local information. For example, in the comparison update, each node makes connection decisions based on the degree information of the surrounding nodes, so the macro evaluation index performance cannot be guaranteed. In the centralized strategy update method, the central node uniformly collects the information detected by all other nodes, including the enemy targets that can be detected and the enemy targets that can be hit. The central node uniformly establishes a feasible strike chain based on the collected information, so it can directly establish a strike chain based on the macro goal, which can more effectively evaluate the index.

[0160] In summary, the present invention provides a multi-agent centralized strategy update method and evaluation system based on game confrontation. The method establishes a communication undirected graph of our nodes, assigns an ID and a communication port to each node in the communication undirected graph, selects a central node based on the node ID, and determines the path for sending messages between non-central nodes and the central node. The central node can be used to summarize local information, thereby formulating a more reasonable strategy in a short time, and can improve the decision-making intelligence ability level of the multi-agent system to a certain extent. Compared with the prior art, the technical solution of the present invention can solve the technical problems in the prior art that the existing model strategy update time is long and the update effect is poor.

[0161] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0162] In addition, it should be noted that the use of terms such as "first" and "second" to limit components is only for the convenience of distinguishing the corresponding components. If not otherwise stated, the above terms have no special meaning and therefore cannot be understood as limiting the scope of protection of the present invention.

[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-agent centralized strategy updating method based on game confrontation, characterized in that: The method comprises: S1, establish a communication undirected graph of our nodes, and assign an ID and a corresponding communication port to each node in the communication undirected graph, wherein the ID is a sequence number, and our nodes include search nodes, tracking nodes, and firepower strike nodes; S2, each node exchanges IDs with all its neighboring nodes and determines the direction of the edge between each node and each neighboring node based on the size relationship of the IDs; S3, divide the two nodes connected by each edge into incoming neighbor nodes and outgoing neighbor nodes according to the direction of the edge, and divide all nodes into source nodes, receiving nodes and internal nodes according to the direction of the edges between each node and all neighbor nodes; S4, each source node transmits its ID to all neighboring nodes, each internal node waits to receive the IDs from all its incoming neighboring nodes, calculates the minimum value among these IDs and sends it to all outgoing neighboring nodes, each receiving node receives the IDs of all its neighboring nodes and calculates the minimum value among these IDs, the receiving node sends YES to the neighboring node that sends the minimum value to it, and sends NO to the remaining neighboring nodes. If all the messages received by an internal node are YES, then the internal node sends YES to all neighboring nodes that have sent messages to it, and sends NO to all neighboring nodes that have not sent messages to it. If an internal node receives a NO, then the internal node sends NO to all neighboring nodes ， The source node waits until all the information is received. If all the information received by a source node is YES, the source node remains a candidate, otherwise the source node is no longer a candidate; S5, when a node sends NO to its neighboring node, the direction of the corresponding edge is reversed; S6, repeat S3 to S5 until only one node is left as a candidate, and the node with the candidate identity is used as the central node, and the remaining nodes are used as non-central nodes; S7, each non-central node determines its own forward node according to the central node, and the central node obtains a path for each non-central node to send a message to the central node based on the forward node; S8, generating an empty graph based on our node and the enemy node, connecting edges on the empty graph according to the information of the enemy node, and generating a tracking and attacking graph in combination with the communication undirected graph of our node; S9, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path through which each non-central node sends messages to it, the tracking strike map, the tracking nodes and the fire strike nodes, and generates a final topology structure based on the generated strike chain.

2. The method according to claim 1, characterized in that Determining the direction of the edge between each node and each neighboring node based on the size relationship of the ID includes: Compare the IDs of the current node and its neighboring nodes. If the ID of the current node is smaller than the ID of the neighboring node, the direction of the corresponding edge is from the current node to the neighboring node. If the ID of the current node is larger than the ID of the neighboring node, the direction of the corresponding edge is from the neighboring node to the current node.

3. The method according to claim 2, characterized in that The two nodes connected by each edge are divided into incoming neighbor nodes and outgoing neighbor nodes according to the direction of the edge, including: If the direction of the edge is from the current node to the neighboring node, the neighboring node is the outgoing node of the current node, and the current node is the incoming node of the neighboring node. If the direction of the edge is from the neighboring node to the current node, the current node is the outgoing node of the neighboring node, and the neighboring node is the incoming node of the current node.

4. The method according to claim 3, characterized in that According to the direction of the edges between each node and all neighboring nodes, all nodes are divided into source nodes, receiving nodes and internal nodes, including: If the direction of the edges between a node and all its neighboring nodes is from the node to the corresponding neighboring node, then the node is the source node; If the direction of the edges between a node and all its neighboring nodes is that the corresponding neighboring nodes are facing the node, then the node is a receiving node; If a node is neither a source node nor a receiving node, it is an internal node.

5. The method according to claim 4, characterized in that Establishing a communication undirected graph of our node and assigning an ID and a corresponding communication port to each node in the communication undirected graph includes: dividing the communication undirected graph into multiple sub-communication undirected graphs, and assigning an ID and a corresponding communication port to each node in each of the sub-communication undirected graphs.

6. The method according to claim 5, characterized in that The central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path of each non-central node sending a message to the central node, the tracking strike map, the tracking node and the fire strike node, including: For the current enemy node that can be covered by the search node, the tracking node and the fire strike node are fully arranged and traversed, and each pair of track-->firing chains that can cover the current enemy target are traversed. Then, the shortest path of track-->firing is searched in the tracking and firing graph, and it is determined whether the firing chain is feasible. If feasible, nodes are connected in the empty graph, and the traversal is continued for the next enemy node. If not feasible, the next pair of track-->firing firing chains is traversed for the current enemy node.

7. The method according to claim 5, characterized in that The method includes: when it is impossible to obtain information about real enemy nodes, obtaining information about imaginary enemy nodes by sampling a given enemy target probability distribution multiple times, S8 includes: The edges are connected on the empty graph according to the information of the imaginary enemy nodes obtained in each sampling, and the corresponding tracking and attacking graph is generated in combination with the communication undirected graph of our nodes.

8. The method according to claim 7, characterized in that The central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path of each non-central node sending a message to the central node, the tracking strike map, the tracking node and the fire strike node, including: For the information of the hypothetical enemy node obtained by each sampling, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path through which each non-central node sends messages to it, the tracking and strike map corresponding to the information of the hypothetical enemy node obtained by a single sampling, and the tracking node and the fire strike node.

9. The method according to claim 8, characterized in that The final topology generated based on the generated strike chain includes: The strike chain generated based on the information of the hypothetical enemy node obtained by each sampling generates a corresponding sub-topology structure; The connection times of the nodes in all sub-topological structures are statistically sorted, and the edges in the empty graph are connected in descending order to obtain the final topological structure.

10. An evaluation system for evaluating the centralized strategy update method according to any one of claims 1 to 9, characterized in that: The evaluation system includes a graph connectivity evaluation module, a task matching evaluation module, a kill rate evaluation module, a task coverage evaluation module and a resilience anti-destruction evaluation module; The graph connectivity evaluation module is used to evaluate the connectivity capability of the multi-agent topology structure updated by the centralized strategy update method, the task matching evaluation module is used to evaluate the potential ability of the multi-agent topology structure updated by the centralized strategy update method to destroy enemy targets, the destruction rate evaluation module is used to evaluate the ability of the multi-agent topology structure updated by the centralized strategy update method to perform destruction tasks, the task coverage evaluation module is used to evaluate the rationality of the multi-agent topology structure updated by the centralized strategy update method, and the elastic anti-destruction evaluation module is used to evaluate the anti-strike capability of the multi-agent topology structure updated by the centralized strategy update method.

11. The evaluation system according to claim 10, characterized in that: The graph connectivity evaluation module is used to evaluate the connectivity of the multi-agent topology structure updated by the centralized strategy updating method according to the following formula: C=w1C1+w2C2, In the above formula, C represents the connectivity, C1 represents the number of connected branches in the multi-agent topology structure, C2 represents the average in-degree and out-degree of nodes in the multi-agent topology structure, w1<0, represents the weight coefficient of the number of connected branches, and w2>0, represents the weight coefficient of the average in-degree and out-degree of nodes.

12. The evaluation system according to claim 10, characterized in that: The task matching evaluation module is used to evaluate the potential ability of the multi-agent topology structure updated by the centralized strategy updating method to destroy the enemy target according to the following formula: K=a1K1+a2K2, In the above formula, K represents the task matching degree, K1 represents the average number of kill chains of enemy targets, K2 represents the average length of kill chains, a1>0 represents the weight coefficient of the average number of kill chains of enemy targets, a2<0 represents the weight coefficient of the average length of kill chains, N c Represents the total number of kill chains of enemy targets, and M represents the number of enemy targets.

13. The evaluation system according to claim 10, characterized in that: The destruction rate evaluation module is used to evaluate the ability of the multi-agent topology structure updated by the centralized strategy updating method to perform the destruction task according to the following formula: In the above formula, D represents the destruction rate, and m represents the number of enemy targets destroyed by the multi-agent topology after the mission is completed.

14. The evaluation system according to claim 10, characterized in that: The task coverage evaluation module is used to evaluate the rationality of the multi-agent topology structure updated by the centralized strategy updating method according to the following formula: In the above formula, T represents the task coverage, represents the number of enemy targets that the multi-agent topology can destroy, Indicates the number of enemy targets that can be monitored.

15. The evaluation system according to claim 10, characterized in that The elastic invulnerability evaluation module is used to evaluate the anti-attack capability of the multi-agent topological structure updated by the centralized strategy updating method according to the following formula: In the above formula, R represents resilience and indestructibility, C, K, D, and T represent the connectivity, task matching, destruction rate, and task coverage of the multi-agent topological structure before a certain proportion of nodes are damaged, respectively; C', K', D', and T' represent the connectivity, task matching, destruction rate, and task coverage of the multi-agent topological structure after a certain proportion of nodes are damaged, respectively.

Images