Multi-agent centralized strategy updating method based on game confrontation and evaluation system

Abstract

The application provides a multi-agent centralized strategy updating method and evaluation system based on game confrontation, the method establishes a communication undirected graph of nodes of our side, allocates an ID and a communication port for each node in the communication undirected graph, selects a central node based on the node ID, and determines a path for sending a message between the non-central node and the central node, the central node can be used to summarize local information, so that a more reasonable strategy can be formulated in a short time, and the intelligent decision-making ability level of the multi-agent system can be improved to a certain extent. The technical scheme of the application is applied to solve the technical problems of long strategy updating time and poor updating effect of the existing model in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-agent system decision-making, and particularly relates to a multi-agent centralized strategy updating method and evaluation system based on game confrontation. BACKGROUND

[0002] Multi-agent systems have wide applications in various fields such as economy and military, and have been attracting more and more attention and research in recent years. This field is dedicated to studying the interaction and strategic game between multiple agents in competitive and cooperative environments. In this system, agents interact with each other, each agent pursues its own goal, and its behavior and decision-making are influenced by other agents. The game confrontation nature of multi-agent systems means that there is competition and conflict between agents. By choosing different strategies, agents can try to optimize their utility functions while also considering the behavior of other agents. This dynamic competition and cooperation has applications in many areas of the real world, such as financial markets, resource allocation, traffic flow, etc. In this field, researchers explore various strategy updating models, game equilibria, cooperation and competition, etc. to promote the learning and decision-making ability of agents in a confrontational environment.

[0003] Multi-agent systems based on game confrontation are dedicated to exploring the strategy selection and interactive behavior of multiple agents in competitive and cooperative environments. In this system, agents influence each other, and each agent pursues its own interests by choosing different strategies while also being influenced by the strategies of other agents. This confrontational environment leads to a game of strategy selection, and agents need to balance their own interests and the behavior of other agents to make optimal decisions. Multi-agent systems based on game confrontation cover multiple research fields and methods related to the competition and cooperation between agents, such as multi-agent game theory, evolutionary game theory, group behavior simulation, multi-agent social networks, adversarial reinforcement learning, group decision-making and cooperation, multi-agent competition and cooperation, evolutionary game dynamics, Stackelberg security game, etc. These fields and methods involve the game and decision-making problems between agents in multi-agent systems, thus promoting the development of multi-agent system research. The cross and integration between different fields also provides more research and application opportunities for multi-agent systems.

[0004] The strategy updating model of multi-agent systems is an important field of study on how agents adjust and improve their strategies to adapt to environmental changes and opponent behavior. Over the past few decades, research on multi-agent strategy updating has deepened, and many innovative methods and theoretical models have emerged. Some common multi-agent system strategy updating models include Comparative Updating, Imitative Updating, Fermi Updating, Moran Process, Genetic Algorithms, Reinforcement Learning, and Evolutionary Game.

[0005] Comparative Updating is a commonly used random process to describe the updating rules of agent strategies. In Comparative Updating, two objects A and B are randomly selected, one of which is the imitating object of the other. The probability that A accepts B's strategy as its own strategy is P, which is a linear function of the benefits of the two agents, and has:

[0006]

[0007] where ω is the selection intensity, representing the influence of neutral drift, π f and π r are the benefits of the two agents, and Δπ is the maximum benefit difference of the agents.

[0008] Imitative Updating is a strategy updating method based on imitating the behavior of others. In imitative learning, one or more agents try to learn strategies by observing the behavior of other agents or experts. The basic idea is that agents can learn from existing excellent strategies, rather than obtaining strategies through trial and error and autonomous exploration. Imitative learning can be divided into the following types: Behavioral Cloning: agents directly copy the behavior of experts, trying to learn the mapping relationship between observed inputs and outputs. The challenge of this method is that it may not be able to deal with situations that have not been seen during training; Inverse Reinforcement Learning: inverse reinforcement learning observes the behavior of experts and infers the goals and preferences pursued by experts, thereby learning more general strategies. This method can adapt to unknown environments and tasks; Cooperative Imitation Learning: multiple agents learn cooperative strategies by imitating each other's behavior to achieve better collaborative effects. This method is particularly useful in multi-agent systems.

[0009] Fermi update is derived from comparison update, but the difference is that P is usually described as a nonlinear function of payoff in the update rule, and the common form is:

[0010]

[0011] When the selection intensity ω<<1, P is asymptotically a linear function of payoff difference π f -π r When ω→∞, this process is close to imitation dynamics.

[0012] Moran process is first introduced to game theory by Taylor et al. in 2004, and quickly attracts the attention of researchers. Scholars have studied Moran process from various perspectives. Moran process is divided into two steps, an individual selects with a probability proportional to its fitness, and produces an offspring, the offspring inherits the parent's strategy selection; in order to keep the total population constant, another randomly selected individual will be removed. Moran process shows a simple birth and death process, and the fitness of an individual can be defined as a function of payoff in different forms, and the common form is:

[0013] f=1-ω+ωπ,

[0014] where ω is the selection intensity, and π is the current individual's payoff.

[0015] In addition, there are some widely used strategy updating models in recent years, such as the strategy updating model based on genetic algorithm, which simulates the process of biological evolution and optimizes the agent's strategy through operations such as selection, crossover and mutation. This method can find better solutions in a larger strategy space. At the same time, reinforcement learning is also a common strategy updating method, in which the agent updates its strategy according to the feedback from the environment to obtain higher cumulative rewards. In multi-agent systems, game theory and other methods can be extended to multi-agent scenarios. However, as a powerful machine learning method, reinforcement learning also has some drawbacks, such as the need for a large number of samples to train the agent, especially in complex environments. This makes it very expensive and time-consuming to collect enough samples in practical applications, especially in real-world physical environments. In addition, reinforcement learning algorithms usually require a long training time to achieve good performance. The length of the training time depends on factors such as environmental complexity, algorithm selection and computing resources. Furthermore, reinforcement learning algorithms usually have many hyperparameters that need to be adjusted, such as learning rate, discount factor, exploration strategy, etc. Different hyperparameter choices can cause significant changes in algorithm performance, and parameter tuning can be difficult. Therefore, in some specific scenarios, the strategy updating model in traditional game theory can achieve better results than reinforcement learning and does not require a large amount of computing resources. For this reason, we propose a centralized strategy updating algorithm that can enable multi-agent systems to update their strategies in a short time and achieve good evaluation metrics.

[0016] Regardless of the game model, the study of multi-agent game strategy updating model and evaluation index of multi-agent decision-making intelligence is crucial. From the perspective of multi-agent game strategy updating model, agents in multi-agent systems interact with each other, and their behaviors and strategies interact to produce complex results. Game strategy updating model can help analyze the strategy selection between different agents, thus revealing the stability and equilibrium of the system. At the same time, through the game strategy updating model, the decision optimization problem of different agents in resource allocation, task division, etc. can be studied. This helps to improve the overall performance of the system. Moreover, agents in multi-agent systems usually evolve their strategies based on the behavior of their opponents. Game strategy updating model can help study the impact of different strategy updating methods on system dynamics. From the perspective of evaluation index of multi-agent decision-making intelligence, the performance of multi-agent systems depends not only on the performance of individuals, but also on factors such as the degree of cooperation and resource utilization efficiency. Evaluation index can quantify the overall performance of the system, helping to choose the appropriate decision strategy. Different decision strategies have different effects on the system results. Evaluation index can help analyze the impact of different decisions on system effectiveness, thus optimizing decision strategies. Through the evaluation index, we can understand the impact of different decision strategies on resource allocation and utilization, thus planning resources reasonably to maximize system efficiency. Therefore, the study of multi-agent game strategy updating model and evaluation index of multi-agent decision-making intelligence is of great significance in optimizing the performance of multi-agent systems, predicting system behavior, and implementing reasonable decision strategies. These studies help us understand the behavior patterns of multi-agent systems in depth, providing scientific basis for practical applications and decision-making.

[0017] The research on the evaluation criteria of multi-agent system decision intelligence has gradually attracted the attention of scholars at home and abroad. Researchers are committed to exploring the decision mechanism, collaboration strategy and performance evaluation of multi-agent systems. The evaluation index of the ability of multi-agent system involves multiple aspects to reflect the performance, efficiency, adaptability and other characteristics of the system at different levels. The commonly used evaluation indexes of the ability of multi-agent system include collaboration efficiency, cooperation stability, task allocation effectiveness, decision quality, conflict resolution ability, adaptability, learning ability, utility and gain, system stability, information sharing, environment adaptability, time efficiency, etc. The above evaluation indexes usually have different weights and importance in different application scenarios. And in some specific application scenarios, some evaluation indexes may even lose the evaluation ability, so for specific application scenarios, reasonable evaluation indexes should be developed to evaluate the intelligent ability of multi-agent system. We construct a complete multi-agent system intelligent ability level evaluation system, which comprehensively considers the evaluation of the decision intelligence level of the multi-agent system in the application scenario from multiple angles, so as to verify the effectiveness of the strategy updating model compared with the traditional strategy updating model. SUMMARY

[0018] The application provides a multi-agent centralized strategy updating method and evaluation system based on game confrontation, which can solve the technical problems of long strategy updating time and poor updating effect of the existing model in the prior art.

[0019] According to an aspect of the application, a multi-agent centralized strategy updating method based on game confrontation is provided, which comprises:

[0020] S1, a communication undirected graph of our node is established, and each node in the communication undirected graph is allocated with an ID and a corresponding communication port, the ID is a serial number, and our node includes a search node, a tracking node and a firepower attack node;

[0021] S2, each node exchanges the ID with all adjacent nodes and determines the direction of the edge between each node and each adjacent node according to the size relationship of the ID.

[0022] S3, dividing two nodes connected by each edge into incoming adjacent node and outgoing adjacent node according to the direction of the edge, and dividing all nodes into source node, receiving node and internal node according to the direction of the edge between each node and all adjacent nodes;

[0023] S4, each source node transmits its ID to all adjacent nodes, each internal node waits to receive IDs from all incoming adjacent nodes, calculates the minimum value of the IDs and sends it to all outgoing adjacent nodes, each receiving node receives the IDs of all adjacent nodes and calculates the minimum value of the IDs, the receiving node sends YES to the adjacent node which sends the minimum value to it and sends NO to the remaining adjacent nodes, if an internal node receives all YES, the internal node sends YES to all adjacent nodes which send messages to it and sends NO to all adjacent nodes which do not send messages to it, if an internal node receives a NO, the internal node sends NO to all adjacent nodes ， The source node waits until it receives all information, if a source node receives all YES, the source node remains as a candidate, otherwise the source node is no longer a candidate;

[0024] S5, when a node sends NO to its adjacent node, the direction of the corresponding edge is reversed;

[0025] S6, repeating S3 to S5 until only one node remains as a candidate, taking the node with the candidate identity as the center node and the remaining nodes as non-center nodes;

[0026] S7, each non-center node determines its forward node according to the center node, and the center node obtains the path of each non-center node sending messages to the center node based on the forward node;

[0027] S8, generating an empty graph based on the friendly node and the enemy node, connecting edges on the empty graph according to the information of the enemy node, and generating a tracking attack graph combining the communication undirected graph of the friendly node;

[0028] S9, the center node generates a strike chain for the enemy node that can be covered by the search node based on the path of each non-center node sending messages to it, the tracking attack graph, the tracking node and the fire attack node, and generates a final topology based on the generated strike chain.

[0029] Further, determining the direction of the edge between each node and each adjacent node according to the size relationship of the ID comprises:

[0030] Comparing the ID size of the current node and its adjacent node, if the ID of the current node is smaller than the ID of the adjacent node, the direction of the corresponding edge is that the current node faces the adjacent node, if the ID of the current node is greater than the ID of the adjacent node, the direction of the corresponding edge is that the adjacent node faces the current node.

[0031] Further, dividing the two nodes connected by each edge into an incoming adjacent node and an outgoing adjacent node according to the direction of the edge comprises:

[0032] If the direction of the edge is that the current node faces the adjacent node, the adjacent node is the outgoing node of the current node, and the current node is the incoming node of the adjacent node, if the direction of the edge is that the adjacent node faces the current node, the current node is the outgoing node of the adjacent node, and the adjacent node is the incoming node of the current node.

[0033] Further, dividing all nodes into a source node, a receiving node and an internal node according to the direction of the edge between each node and all adjacent nodes comprises:

[0034] If the direction of the edge between a node and all adjacent nodes of the node is that the node faces the corresponding adjacent node, the node is a source node;

[0035] If the direction of the edge between a node and all adjacent nodes of the node is that the corresponding adjacent node faces the node, the node is a receiving node;

[0036] If a node is neither a source node nor a receiving node, the node is an internal node.

[0037] Further, establishing a communication undirected graph of the I side nodes and assigning an ID and a corresponding communication port to each node in the communication undirected graph comprises: dividing the communication undirected graph into a plurality of sub-communication undirected graphs, and assigning an ID and a corresponding communication port to each node in each sub-communication undirected graph.

[0038] Further, the center node generates a strike chain of enemy nodes that can be covered by the search node based on the path of each non-center node sending a message to it, the track strike graph, the track node and the fire strike node comprises:

[0039] For the current enemy node that can be covered by the search node, the track node and the fire strike node are traversed in full permutation, each pair of track->firing chains that can cover the current enemy target is traversed, then the shortest path of track->firing in the track strike graph is searched, and it is judged whether the strike chain is feasible, if feasible, node connection is performed in the empty graph, and traversal is continued for the next enemy node, if not feasible, the next pair of track->firing strike chains is traversed for the current enemy node.

[0040] Further, the method comprises: when the information of the real enemy node cannot be acquired, the information of the imaginary enemy node is obtained by sampling the given enemy target probability distribution multiple times, S8 comprises:

[0041] The edges are connected on the empty graph according to the information of the imaginary enemy node obtained each time, and a corresponding tracking strike graph is generated in combination with the communication undirected graph of the own node.

[0042] Further, the generation of the strike chain of the enemy node that can be covered by the search node by the center node based on the path of each non-center node to which the message is sent, the tracking strike graph, the tracking node and the fire strike node comprises:

[0043] For the information of the imaginary enemy node obtained each time, the center node generates a strike chain of the enemy node that can be covered by the search node based on the path of each non-center node to which the message is sent, the tracking strike graph corresponding to the information of the imaginary enemy node obtained by single sampling, the tracking node and the fire strike node.

[0044] Further, the generation of the final topology structure based on the generated strike chain comprises:

[0045] The strike chain generated based on the information of the imaginary enemy node obtained each time corresponds to the generation of a sub-topology structure;

[0046] The connection times of the nodes in all sub-topology structures are counted and sorted in descending order, and the edges in the empty graph are connected in turn to obtain the final topology structure.

[0047] According to another aspect of the present application, an evaluation system for evaluating the centralized strategy updating method proposed in the foregoing of the present application is provided, and the evaluation system comprises a graph connectivity evaluation module, a task matching degree evaluation module, a hit rate evaluation module, a task coverage degree evaluation module and a flexible anti-destroying evaluation module;

[0048] The graph connectivity evaluation module is used to evaluate the connectivity capability of the multi-agent topology structure updated by the centralized strategy updating method, the task matching degree evaluation module is used to evaluate the potential capability of the multi-agent topology structure updated by the centralized strategy updating method to hit the enemy target, the hit rate evaluation module is used to evaluate the capability of the multi-agent topology structure updated by the centralized strategy updating method to execute the hit task, the task coverage degree evaluation module is used to evaluate the rationality of the multi-agent topology structure updated by the centralized strategy updating method, and the flexible anti-destroying evaluation module is used to evaluate the anti-strike capability of the multi-agent topology structure updated by the centralized strategy updating method.

[0049] Further, the graph connectivity evaluation module is used to evaluate the connectivity capability of the multi-agent topology structure updated by the centralized strategy updating method according to the following formula:

[0050] C = w1C1 + w2C2,

[0051] In the above formula, C represents the connectivity, C1 represents the number of connected components in the multi-agent topology structure, C2 represents the average in-out degree of nodes in the multi-agent topology structure, w1 < 0 represents the weight coefficient of the number of connected components, and w2 > 0 represents the weight coefficient of the average in-out degree of nodes.

[0052] Further, the task matching degree evaluation module is configured to evaluate the potential ability of the multi-agent topology structure updated by the centralized strategy updating method to destroy enemy targets according to the following formula:

[0053] K = a1K1 + a2K2,

[0054]

[0055] In the above formula, K represents the task matching degree, K1 represents the average number of damage chains of enemy targets, K2 represents the average length of damage chains, a1 > 0 represents the weight coefficient of the average number of damage chains of enemy targets, and a2 < 0 represents the weight coefficient of the average length of damage chains, N c represents the total number of damage chains of enemy targets, and M represents the number of enemy targets.

[0056] Further, the destruction rate evaluation module is configured to evaluate the ability of the multi-agent topology structure updated by the centralized strategy updating method to perform the destruction task according to the following formula:

[0057]

[0058] In the above formula, D represents the destruction rate, and m represents the number of enemy targets destroyed by the multi-agent topology structure after the task is completed.

[0059] Further, the task coverage degree evaluation module is configured to evaluate the rationality of the multi-agent topology structure updated by the centralized strategy updating method according to the following formula:

[0060]

[0061] In the above formula, T represents the task coverage degree, represents the number of enemy targets that can be destroyed by the multi-agent topology structure, represents the number of enemy targets that can be monitored.

[0062] Further, the flexible anti-destroying evaluation module is configured to evaluate the anti-attack ability of the multi-agent topology structure updated by the centralized strategy updating method according to the following formula:

[0063]

[0064] In the formula, R represents the elastic invulnerability, C, K, D and T represent the connectivity, the task matching degree, the hit rate and the task coverage of the multi-agent topology structure before a certain proportion of node damage in turn, and C', K', D' and T' represent the connectivity, the task matching degree, the hit rate and the task coverage of the multi-agent topology structure after a certain proportion of node damage in turn.

[0065] The technical scheme of the application provides a multi-agent centralized strategy updating method and evaluation system based on game confrontation, which establishes a communication undirected graph of the nodes of the side, allocates an ID and a communication port to each node in the communication undirected graph, selects a central node based on the node ID, and determines the path for sending messages between the non-central nodes and the central node, so that the central node can be used to collect local information, so that a more reasonable strategy can be formulated in a short time, and the intelligent decision-making ability level of the multi-agent system can be improved to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0066] The accompanying drawings included to provide a further understanding of the embodiments of the application and constitute a part of the specification, serve to explain the principles of the application together with the text of the description. Obviously, the drawings described below are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0067] Figure 1 A flowchart of a multi-agent centralized strategy updating method based on game confrontation according to a specific embodiment of the application is shown;

[0068] Figure 2 A comparison updating schematic diagram according to a specific embodiment of the application is shown;

[0069] Figure 3 A Fermi updating schematic diagram according to a specific embodiment of the application is shown;

[0070] Figure 4 An imitation updating schematic diagram according to a specific embodiment of the application is shown;

[0071] Figure 5 A leader election algorithm schematic diagram according to a specific embodiment of the application is shown. DETAILED DESCRIPTION

[0072] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other in the case of no conflict. The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The description of the at least one example embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0073] It should be noted that the terms used herein are only intended to describe specific embodiments, and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form, unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a reference to the presence of a feature, step, operation, device, component and / or combinations thereof.

[0074] Unless specifically stated otherwise, the relative arrangement of components and steps, numerical expressions, and numerical values set forth in the various embodiments described herein are not limiting. It should be understood that the various parts shown in the drawings are not necessarily drawn to scale. Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered as part of the description of the present application. In all examples shown and discussed herein, any specific value should be interpreted as merely an example, and not as a limitation. Therefore, other examples of the exemplary embodiments can have different values. It should be noted that similar reference numbers and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0075] As Figure 1 shown, according to the specific embodiments of the present application, a multi-agent centralized strategy updating method based on game confrontation is provided, which comprises:

[0076] S1, a communication undirected graph of our nodes is established, and each node in the communication undirected graph is assigned with an ID and a corresponding communication port, the ID is a serial number, and the nodes of our side include search nodes, tracking nodes and fire attack nodes;

[0077] S2, each node exchanges the ID with all adjacent nodes and determines the direction of the edge between each node and each adjacent node according to the size relationship of the ID;

[0078] S3, according to the direction of the edge, divide the two nodes connected by each edge into incoming adjacent nodes and outgoing adjacent nodes, and divide all nodes into source nodes, receiving nodes and internal nodes according to the direction of the edge between each node and all adjacent nodes;

[0079] S4, each source node transmits its ID to all adjacent nodes, each internal node waits to receive the ID from all incoming adjacent nodes, calculates the minimum value of the IDs and sends it to all outgoing adjacent nodes, each receiving node receives the ID of all adjacent nodes and calculates the minimum value of the IDs, the receiving node sends YES to the adjacent node that sends the minimum value to it and sends NO to the remaining adjacent nodes, if an internal node receives all YES messages, the internal node sends YES to all adjacent nodes that have sent messages to it and sends NO to all adjacent nodes that have not sent messages to it, if an internal node receives a NO message, the internal node sends NO to all adjacent nodes ， The source node waits until all information is received, if a source node receives all YES messages, the source node remains a candidate, otherwise the source node is no longer a candidate;

[0080] S5, when a node sends NO to its adjacent node, the direction of the corresponding edge is reversed;

[0081] S6, repeat S3 to S5 until only one node remains as a candidate, take the node with the candidate identity as the center node and the remaining nodes as non-center nodes;

[0082] S7, each non-center node determines its forward node according to the center node, and the center node obtains the path of each non-center node sending messages to the center node based on the forward node;

[0083] S8, generate an empty graph based on the friendly nodes and enemy nodes, connect the edges on the empty graph according to the information of the enemy nodes, and generate a tracking attack graph combining the communication undirected graph of the friendly nodes;

[0084] S9, the center node generates a strike chain for the enemy nodes that can be covered by the search node based on the path of each non-center node sending messages to it, the tracking attack graph, the tracking node and the fire attack node, and generates a final topology based on the generated strike chain.

[0085] With the configuration, a multi-agent centralized strategy updating method based on game confrontation is provided, which allocates ID and communication port for each node in the communication undirected graph by establishing the communication undirected graph of the nodes, selects a central node based on the node ID, and determines the path for sending messages between the non-central node and the central node, so that the central node can be used to collect local information, and a more reasonable strategy can be made in a short time, and the decision-making intelligent level of the multi-agent system can be improved to a certain extent. Compared with the prior art, the technical scheme of the present application can solve the technical problems of long strategy updating time and poor updating effect of the existing model in the prior art.

[0086] Further, in the embodiment of the present application, the direction of the edge between each node and each adjacent node is determined according to the size relationship of the ID, which includes: comparing the ID of the current node and its adjacent node, if the ID of the current node is smaller than the ID of the adjacent node, the direction of the corresponding edge is that the current node faces the adjacent node, and if the ID of the current node is greater than the ID of the adjacent node, the direction of the corresponding edge is that the adjacent node faces the current node. The two nodes connected by each edge are divided into incoming adjacent nodes and outgoing adjacent nodes according to the direction of the edge, which includes: if the direction of the edge is that the current node faces the adjacent node, the adjacent node is the outgoing node of the current node, and the current node is the incoming node of the adjacent node, and if the direction of the edge is that the adjacent node faces the current node, the current node is the outgoing node of the adjacent node, and the adjacent node is the incoming node of the current node.

[0087] Based on the above embodiment, in the embodiment of the present application, all nodes are divided into source nodes, receiving nodes and internal nodes according to the direction of the edge between each node and all adjacent nodes, which includes: if the direction of the edge between a node and all its adjacent nodes is that the node faces the corresponding adjacent node, the node is a source node; if the direction of the edge between a node and all its adjacent nodes is that the corresponding adjacent node faces the node, the node is a receiving node; if a node is neither a source node nor a receiving node, the node is an internal node.

[0088] That is, in the centralized updating strategy proposed in the present application, a key problem is to select a connected component center for each connected component in the topology formed by the multi-agent system. As a specific embodiment of the present application, the present application adopts the classical Y o -y oLeader election algorithm to determine the center node. This algorithm has leading performance in both data transmission volume and time consumption. In distributed computing, leader election is the process of designating a process as an organizer of certain tasks distributed among multiple computers (nodes). Before the task begins, all nodes either do not know which node will act as the "leader" of the task or cannot communicate with the current leader. However, after applying the leader election algorithm, every node in the entire network will identify a specific, unique node as the task leader.

[0089] In the practical application scenario of the present application, the main idea is to construct a spanning tree on a connected graph and select its root as the leader, as shown in Figure 5 Specifically, it includes two stages: a preprocessing stage and a series of iterations. Assuming that the identification (ID) of node N n in our node set N = {N1, N2,..., N i} is i, in the preprocessing stage, each node N i exchanges its ID with all its adjacent nodes N c = {N i1 , N i2 ,..., N im} and determines the direction of its associated edge according to the ID. For example, if the ID of node N i is less than that of node N j , then node N i is directed towards node N j , at which point node N j is called the outgoing adjacent node of node N i , and node N i is called the incoming adjacent node of node N j . If the ID of a node N i is less than that of all its adjacent nodes, then it is called a source node. Conversely, if the ID of a node N i is greater than that of all its adjacent nodes, then it is called a sink node. All nodes that are neither source nodes nor sink nodes are called internal nodes. When all edges are oriented, the iteration of the second part begins. Each iteration is an election, in which some candidates will be deleted. Each iteration is divided into two stages: a forward stage and a backward stage, the process being as follows:

[0090] Forward stage: 1. First, each source node transmits its ID to all adjacent nodes; 2. internal nodes wait to receive IDs from all incoming adjacent nodes, which calculate the minimum value of these values and send it to all outgoing adjacent nodes; 3. sink nodes receive all IDs and calculate their minimum value.

[0091] Backward phase: 1. The receiving node sends YES to the neighbor node sending the minimum value and NO to other nodes; 2. When an internal node receives all YES messages, it sends YES to all the neighbor nodes from which it has received messages and NO to other neighbor nodes, if an internal node receives a NO, it will send NO to all neighbor nodes; 3. The source node will wait until it receives all messages, if it receives all YES messages, it will remain a candidate, otherwise it will no longer be a candidate; 4. When a node N i sends NO to a neighbor node N j , the direction of the edge is reversed.

[0092] The iteration process continues until only one candidate node is left, the iteration ends, and the candidate node becomes the center node. After the center node is elected, each non-center node needs to determine its forward node, which helps the non-center node to transmit messages to the center node, i.e. each node needs to send all its information to the forward node, and then continuously transmit, until the information is transmitted to the center node. In this process, the center node first broadcasts a message to all neighbor nodes, then the node receiving the message broadcasts the message to all its neighbor nodes again, each node only broadcasts the message once after receiving the message, and does not propagate the message again after receiving it again. The first node that receives a message from each node is the forward node of the node.

[0093] After determining the forward node of each node, the path for each node to send messages to the center node is determined. However, the center node also needs to know the path to send messages to each node. In this process, each node sends its node name to the forward node, the forward node sends the received node name plus its own node name to its forward node, and so on until it is sent to the center node. The information received by the center node is the path to each node, and thus the election of the center node is completed.

[0094] However, when building a complete multi-agent system topology, the topology may not always form a connected graph, in which case there will be multiple connected branches in the topology. At this time, a connected branch center is selected for each connected branch, and the enemy unit information monitored by each node in the connected branch is transmitted to the connected branch center. The connected branch center aggregates all information and formulates a strategy for each enemy target, and transmits the formulated strategy to each node in the connected branch. Each node in the connected branch then acts according to the instructions issued by the connected branch center. This strategy evolution method is called centralized strategy evolution method.

[0095] At the beginning of the task, any node N iThey all know the location of the central node and will mess their own messages i , such as the location of the monitored enemy node and other information is sent to the central node. At this time, the central node will have global information and start to formulate strategies. The process of strategy formulation is as follows: for each enemy target, the central node calculates a shortest attack link from the current topology map to attack the enemy target, and then the central node sends a connection command to each of our nodes on this link. After receiving the connection command, our node connects until all enemy targets are traversed. The strategy formulation is completed, and the central node then passes the specified strategy to each node.

[0096] In the existing three policy update methods of comparison update, Fermi update and imitation update, the strategy of each node is determined by itself, that is, which nodes to connect to is only determined by the node itself through the corresponding policy update method. At this time, the node can only obtain all its own information, so it has limitations when updating node strategies. In the centralized strategy implementation method proposed in the present invention, the connected branch center obtains the information of all nodes in the connected branch and uniformly allocates strategies. Instead of each node making decisions on its own, the connected branch center can obtain local or even global information, so it can be more reasonable and effective when allocating strategies to each node.

[0097] Furthermore, after a central node is selected for each connected subgraph using the leader election algorithm, if there is a connected subgraph with a particularly large number of nodes, the time complexity of the subsequent central node calculation of the connection strategy will be very high. Therefore, in order to enable more efficient parallel computing, in an embodiment of the present invention, before the leader election algorithm is used to select the central node, a graph partitioning algorithm is used to partition the communication graph into many small subgraphs. In other words, a communication undirected graph of our node is established, and an ID and a corresponding communication port are assigned to each node in the communication undirected graph, including: partitioning the communication undirected graph into multiple sub-communication undirected graphs, and assigning an ID and a corresponding communication port to each node in each sub-communication undirected graph. As a specific embodiment of the present invention, the number of nodes in each subgraph does not exceed 150, and after partitioning, the central node election algorithm is run on each subgraph. The key part of the graph partitioning algorithm calls the part_graph function in the pymetis package in Python, which can partition a given connected graph into the required specific number of subgraphs. After performing the graph partitioning algorithm, assuming that we have obtained n small connected subgraphs, we perform the leader election algorithm on these n small connected subgraphs to select the central node of each small connected subgraph. After that, we can calculate the connection strategy of each small connected subgraph in parallel on these n small connected subgraphs, which can greatly optimize the time complexity of formulating the connection strategy.

[0098] Based on the above embodiment, in the embodiment of the application, the center node generates a strike chain for an enemy node that can be covered by the search node based on the path of each non-center node sending a message to it, the full fight graph, the track node and the firing node, which includes: for the current enemy node that can be covered by the search node, performing full permutation traversal on the track node and the firing node, traversing each pair of track->firing chains that can cover the current enemy target, then searching for the shortest path of track->firing in the full fight graph, and judging whether the strike chain is feasible, if feasible, connecting the nodes in the empty graph, and continuing to traverse the next enemy node, if not feasible, continuing to traverse the next pair of track->firing strike chains for the current enemy node, until all enemy targets are traversed, and the decision algorithm of the centralized update strategy proposed in the application is completed. The full fight graph is used to search for the shortest path, and all edges that can be connected on the graph are connected, without considering the restrictions of slot and limited.

[0099] Further, in the above-mentioned centralized update strategy algorithm, only when all information of all enemy targets (enemy nodes) is known can the damage chain be established, which is named as a traditional centralized update strategy algorithm, however, in a real scene, we may not be able to obtain all information of the enemy targets in advance, therefore, the application further proposes a centralized update strategy based on a probability graph distribution for generating a strike chain based on the imaginary enemy targets sampled based on a given probability distribution, that is, the method includes: when the information of the real enemy nodes cannot be obtained, the information of the imaginary enemy nodes is obtained by sampling the given enemy target probability distribution multiple times, S8 includes: connecting the edges on the empty graph according to the information of the imaginary enemy nodes obtained by each sampling, and generating a corresponding track fight graph in combination with the communication undirected graph of the our nodes.

[0100] The application considers two different centralized update strategy algorithms based on a probability graph distribution. The first one is a centralized update strategy algorithm based on a probability graph distribution inspired by the voting theory, and the second one is a centralized update strategy algorithm based on a probability graph distribution inspired by the Shapley value theory. We will introduce the implementation details of the two algorithms below.

[0101] The decision algorithm process of the centralization update strategy based on the probability graph distribution inspired by the voting theory is as follows. First, sample_number enemy target samples are generated. After using the decision algorithm of the aforementioned centralization update strategy for each sampled enemy sample, one topology structure of our side can be obtained. Therefore, a total of sample_number topology structures of our side can be generated. The number of times of all connected edges in the sample_number topology structures of our side is counted, and all connected edges are sorted in descending order according to the number of times of connection. Then, the edges are connected in our side topology structure in turn until all edges are connected or no more edges can be connected. Finally, the final topology structure of our side is obtained. After obtaining the final topology structure of our side, all track->firing chains in the architecture need to be found out and stored in a file as potential possible strike chains for use in the subsequent kill chain generation module and elastic anti-kill module.

[0102] That is, the central node generates a strike chain for the enemy node covered by the search node based on the path of each non-central node sending a message to it, the tracking strike graph, the tracking node, and the fire strike node, including: for the information of the imaginary enemy node obtained by each sampling, the central node generates a strike chain for the enemy node covered by the search node based on the path of each non-central node sending a message to it, the tracking strike graph corresponding to the information of the imaginary enemy node obtained by single sampling, the tracking node, and the fire strike node. Generating a final topology structure based on the generated strike chain includes: generating a sub-topology structure corresponding to the strike chain generated based on the information of the imaginary enemy node obtained by each sampling; counting and sorting the number of connections of nodes in all sub-topology structures, connecting the edges in the empty graph in descending order, and obtaining a final topology structure.

[0103] The decision algorithm process of the Shapley value theory inspired probability graph distribution based centralization update strategy is generally the same as that of the voting theory inspired probability graph distribution based centralization update strategy. The only difference is that in the Shapley value theory inspired probability graph distribution based centralization update strategy, for each of the friendly topology structures generated by the enemy samples, the number of attack chains in which each edge in the structure participates is counted as the marginal benefit contributed by the edge. Then the number of attack chains in which all the connected edges in the sample_number friendly topology structures participate is counted, and all the connected edges are sorted in descending order according to the number of times of participating in attack chains. Then these edges are connected in the friendly topology structure one by one until all the edges are connected or no more edges can be connected. Finally, the final topology structure of the friendly side is obtained. The generation method of the attack graph, the generation method of the attack chain, and the generation method of the topology structure in the probability graph distribution based centralization update strategy algorithm can refer to the related exemplary description in the foregoing traditional centralization update strategy algorithm, and will not be described here.

[0104] According to another aspect of the present application, an evaluation system for evaluating the centralization strategy update method proposed in the foregoing is provided, which comprises a graph connectivity evaluation module, a task matching degree evaluation module, a hit rate evaluation module, a task coverage evaluation module, and a flexible anti-attack evaluation module. The graph connectivity evaluation module is used to evaluate the connectivity of the multi-agent topology structure updated by the centralization strategy update method. The task matching degree evaluation module is used to evaluate the potential ability of the multi-agent topology structure updated by the centralization strategy update method to hit enemy targets. The hit rate evaluation module is used to evaluate the ability of the multi-agent topology structure updated by the centralization strategy update method to execute a hit task. The task coverage evaluation module is used to evaluate the rationality of the multi-agent topology structure updated by the centralization strategy update method. The flexible anti-attack evaluation module is used to evaluate the anti-attack ability of the multi-agent topology structure updated by the centralization strategy update method. The specific description is as follows:

[0105] 1) Connectivity

[0106] Connectivity is determined by the number of connected components and the average out-degree of nodes, and its value is the weighted average of the number of connected components and the average out-degree of nodes, that is, the graph connectivity evaluation module is used to evaluate the connectivity of the multi-agent topology structure updated by the centralization strategy update method according to the following formula:

[0107] C = w1C1 + w2C2,

[0108] In the above formula, C represents connectivity, C1 represents the number of connected branches in the multi-agent topology structure, C2 represents the average in-out degree of nodes in the multi-agent topology structure, w1<0 represents the weight coefficient of the number of connected branches, and w2>0 represents the weight coefficient of the average in-out degree of nodes.

[0109] The greater the connectivity index C is, the better the connectivity performance of the topology structure formed by the multi-agent system is. In the evaluation experiment, the connectivity is calculated by the connectivity function in utils.analysers.dynamic_analyser.py, where w1=-0.1 and w2=0.5.

[0110] 2) Task matching degree

[0111] The task matching degree is determined by the average number of kill chains and the average length of kill chains, and is used to measure the potential ability of the multi-agent system to destroy enemy targets. The task matching degree is the weighted average of the average number of kill chains and the average length of kill chains of enemy targets. That is, the task matching degree evaluation module is used to evaluate the potential ability of the multi-agent topology structure updated by the centralized strategy updating method to destroy enemy targets according to the following formula:

[0112] K=a1K1+a2K2,

[0113]

[0114] In the above formula, K represents the task matching degree, the greater the value is, the stronger the potential ability of the multi-agent system to destroy enemy targets is, K1 represents the average number of kill chains of enemy targets, K2 represents the average length of kill chains, a1>0 represents the weight coefficient of the average number of kill chains of enemy targets, a2<0 represents the weight coefficient of the average length of kill chains, N c represents the total number of kill chains of enemy targets, and M represents the number of enemy targets.

[0115] 3) Destruction rate

[0116] The destruction rate represents the ratio of enemy targets that can be destroyed by the multi-agent system after the task is completed, and can effectively measure the ability of the multi-agent system to perform the destruction task. In the embodiment of the present application, the destruction rate evaluation module is used to evaluate the ability of the multi-agent topology structure updated by the centralized strategy updating method to perform the destruction task according to the following formula:

[0117]

[0118] In the above formula, D represents the destruction rate, and m represents the number of enemy targets that have been destroyed by the multi-agent topology structure after the task is completed.

[0119] 4) Task coverage degree

[0120] Task coverage is: at the current time, the number of enemy targets that can be destroyed by the multi-agent system and the number of enemy targets that can be monitored, which is used to measure the rationality of the topology architecture formed by the multi-agent system.

[0121]

[0122] In the above formula, T represents the task coverage, represents the number of enemy targets that can be destroyed by the multi-agent topology structure, represents the number of enemy targets that can be monitored. The task coverage T is in the range of [0, 1], and the greater the task coverage, the more reasonable the topology architecture formed by the multi-agent system.

[0123] 5) Elasticity of anti-destroying

[0124] Elasticity of anti-destroying represents the anti-attack ability of the multi-agent system, that is, the ability that the multi-agent system can maintain after suffering a certain attack, that is, at the current time, the maintenance ratio of the four indicators after a certain proportion of node damage (random damage or damage to the node with the maximum degree) is calculated. That is, the elasticity of anti-destroying evaluation module is used to evaluate the anti-attack ability of the multi-agent topology structure updated by the centralized strategy updating method according to the following formula:

[0125]

[0126] In the above formula, R represents the elasticity of anti-destroying, and the greater the value, the better, C, K, D and T represent the connectivity, task matching degree, destruction rate and task coverage of the multi-agent topology structure before a certain proportion of node damage, and c', K', D' and T' represent the connectivity, task matching degree, destruction rate and task coverage of the multi-agent topology structure after a certain proportion of node damage.

[0127] Further, in the embodiment of the present application, for the special scene system architecture generation problem, a software system capable of independently evaluating the decision-making intelligent capability of the system game is developed based on the above evaluation system. The software system supported by the system node number should be not less than 10000 (including at least 50 node numbers not less than 100), can load the scene assumption through the pre-defined data interface, access the game strategy model meeting the interface requirements, for the game strategy combination under the scene, can efficiently calculate the decision-making intelligent capability evaluation index proposed by us, and simply display the system architecture generation process and result and the game confrontation process in a visual way. The multi-agent system decision-making intelligent capability level evaluation index system can reasonably and effectively evaluate the multi-agent system decision-making intelligent capability level.

[0128] Based on the above evaluation system, the application scenario of the multi-agent centralized strategy update method based on game confrontation proposed in this invention can be modeled as follows:

[0129] Identify participants: In this system's game-playing scenario, the participants are various friendly nodes and enemy aircraft nodes. These include search nodes, tracking nodes, command and control nodes, and firepower strike nodes.

[0130] Determine the strategy: In game theory, strategy refers to the way players choose actions. Let A i ={a1, a2, ..., a n} is player (node) N i The set of actions that can be selected, let s i (a i ) is player N i Select action a i The probability of s i (a i )satisfy:

[0131]

[0132] If there is an action a i , so that s i (a i )=1, then s i is a pure strategy, otherwise it is called s i For all kinds of our nodes, since our goal is to build a reasonable system to resist the attack of enemy nodes, it is very important to explore how to construct the connection communication between nodes. Assume that our node N i The nodes that can be connected to are {N i1 , N i2 ,...,N im}, define node N i The set of selectable actions is A i ={a i1 , a i2 ,...,a im}, where a ij Represents node N i Connect to node N ij , let s i (a ij ) represents our node N i Select action a ij The probability of s i (a ij )satisfy:

[0133]

[0134] Determine the strategy of our node N i i Therefore, for each node of our side, the strategy refers to how to determine the connection between nodes, such as for a given node, it can choose whether to connect to other nodes within its range that can be connected, and how to reconnect between nodes after the relevant node is destroyed, etc.

[0135] For enemy nodes, since the enemy's goal is to destroy our nodes, it is assumed that the enemy node E i n ) can attack, and the set of actions that the enemy node E i i ) can choose is defined as B n = {b1, b2,..., b i}, where b i represents the enemy node E i selecting to attack our node N i , and s i (b i ) represents the probability of the enemy node E i selecting action b i , which satisfies:

[0136]

[0137] Therefore, the strategy of each enemy node refers to the way to determine the attacked nodes of our side, i.e., the probability of each enemy node attacking each node of our side.

[0138] Determine the utility: for our nodes, the utility is calculated using the established evaluation index system, such as connectivity, task matching degree, task coverage, destruction rate, and elastic anti-destroying, etc. The larger the above indicators, the higher the utility obtained by our side, and the utility of our nodes is set as U. For enemy nodes, the number of destroyed nodes of our side is used to represent the utility of enemy nodes, i.e., the more the number of destroyed nodes of our side, the greater the utility of enemy nodes, and the utility of enemy nodes is set as V.

[0139] In this system game confrontation scenario, its essence is a multi-round game process, and in each round, the enemy node moves one unit to the destination, and then the enemy node and our node can choose strategies to play to obtain higher utility, so the game tree can be used to construct the detailed process of the game. At the beginning of the game, in the first round, our node determines the node connection strategy s ij (a n ), the way of connecting between nodes, and let N = {N1, N2,..., N ij ​​​|i=1,2,...,n;j=1,2,...,n} is the connection strategy, where e ij =1 means our node N i and our node N j Connected, e ij =0 means our node N i and our node N j Not connected. N and Edge constitute the topology of our system. The enemy node selects the attack strategy s i (b i ), that is, the way to attack our node, let E={E1,E2,...,E m} represents each enemy node, Attack={a ij |i=1,2,...,m;j=1,2,...,n} represents the attack target of each enemy node, where a ij =1 indicates the enemy node E i Will attack our node N j , a ij =0 means enemy node E i Will not attack our node N j In each subsequent round, our node can attack the enemy node through the established multi-agent system, and the enemy node can also attack our node within the attack range or move one unit towards the selected target node. At the same time, in each round, our node can calculate its own utility U according to the real-time situation and choose to update the connection strategy s i (a ij ) to establish a more rational and effective multi-agent system. As the adversarial game progresses, our nodes will continuously attack enemy nodes within their firepower range. Similarly, enemy nodes will continuously attack our target nodes within their firepower range until either one of our nodes is completely destroyed or all enemy nodes reach their destination, and the system game ends. Ultimately, the utility U gained by our nodes and the utility V gained by the enemy nodes can be calculated. The final results and data from the game process can be used to evaluate the intelligence level of our multi-agent system.

[0140] After building a complete multi-agent system intelligence capability assessment system, the present invention proposes to evaluate the multi-agent system intelligence capability under two different game paradigms: cooperative game paradigm and competitive game paradigm.

[0141] In the cooperative game paradigm, our nodes are in a cooperative relationship, and each node hopes to connect to more nodes with high degrees, so that it can connect with more nodes. Therefore, in the cooperative game paradigm, we can define node N iThe utility U of the node is the degree D of the node, that is, U(N i ) = D(N i ). At this time, each node can connect to a node with a higher degree according to the utility of the surrounding nodes.

[0142] In the competitive game paradigm, there is a competitive relationship between the nodes of our side in the competitive game paradigm scene, and each node hopes to be connected to more nodes, but at the same time, it does not want to become a key node to avoid being destroyed by the enemy. Therefore, in the adversarial game paradigm, when connecting nodes, it is first determined whether the degree has exceeded the maximum degree. For example, for node N i , assuming that the nodes it needs to connect are {N i1 , N i2 ,..., N im}, when judging whether it needs to connect node N ij , the following rules are followed:

[0143]

[0144] Further, the present application calculates the intelligent capability level of various game update strategy models such as comparison update, imitation update and Fermi update under two different game paradigms according to the existing evaluation index system. Specifically as follows:

[0145] As shown in Figure 2 , in the comparison update of the cooperative game paradigm, for node N i , assuming that the nodes that can be connected around it are {N1, N2,..., N m}, the probability of node N i connecting to node N j is where u i is the degree of node N i , u j is the degree of node N j , and the degree c of node N j is a constant, and c=0.5 in the experiment. After obtaining the nodes to be connected, the node currently performing the update needs to first disconnect the nodes that have been connected but are not in the nodes to be connected, and then connect the nodes that have not been connected but need to be connected. The probability of connecting each node in the comparison update of the competitive game paradigm is the same as that in the comparison update of the cooperative game paradigm, except that when connecting nodes, it is first determined whether the degree has exceeded the maximum degree among the surrounding nodes.

[0146] As shown in Figure 3 , in the Fermi update of the cooperative game paradigm, for node N i , assuming that the nodes that can be connected around it are {N1, N2,..., Nm}, then node N i Connect to node N j The probability of where u i For node N i The degree, c, and α are both constants. In this experiment, c = 1 and α = 1. After obtaining the nodes to be connected, the node currently updating must first disconnect any nodes that are already connected but not among the nodes to be connected, and then connect any unconnected nodes that need to be connected. The probability of connecting nodes in the Fermi update under the competitive game paradigm is the same as the comparative update under the cooperative game paradigm. The difference is that when connecting nodes, the node first determines whether its degree exceeds the maximum degree of the surrounding nodes.

[0147] like Figure 4 As shown, in the imitation update under the cooperative game paradigm, for node N i , assuming that the nodes that can be connected to it are {N1, N2, ..., N m}, and the utility of these nodes decreases in turn, that is, u1≥u2≥…≥u m , then node N i Prioritizes connecting to nodes with higher degrees and the nodes they are connected to. After determining the order of connecting nodes, the node currently being updated must first disconnect the already connected nodes and then sequentially connect to the nodes it needs to connect to until the slot is exhausted. The order of connecting nodes in the imitation update under the competitive game paradigm is the same as that under the cooperative game paradigm. The difference is that when connecting nodes, the node first determines whether its degree exceeds the highest degree among the surrounding nodes.

[0148] During the evaluation, the kill chain generation module inputs the current topological architecture of our node, the information of the enemy target, and the update strategy used, and outputs all possible kill chains and enemy targets that can be searched and tracked for subsequent indicator calculations.

[0149] If the three traditional update strategies, namely comparison update, Fermi update and imitation update, are used, all feasible kill chains and enemy targets that can be searched are directly calculated.

[0150] If the centralized update strategy proposed in this invention is used, directly searching for the shortest path to calculate all feasible kill chains for each enemy target in the topology generated by the centralized update strategy results in very high time complexity. Therefore, to reduce this time complexity, we have optimized the kill chain generation module. The specific optimization process is as follows.

[0151] First, we have divided the topology of our side into many small connected subgraphs by using the graph partition algorithm before the leader election algorithm. So in this kill chain generation module, we can calculate the kill chain that each subgraph can form in parallel on all small connected subgraphs, which greatly reduces the time complexity of the kill chain generation. Second, in the decision algorithm based on the centralized update strategy of the probability graph distribution, we have found out all the track—>firing chains generated in the topology architecture as potential possible attack chains and stored them in the file. Therefore, we do not need to search for the shortest path for each enemy target here, but directly traverse all track—>firing chains. For each track—>firing chain, detect whether it can attack the enemy target. If it can, save the kill chain and continue to traverse the next enemy target. If it cannot, continue to traverse the subsequent track—>firing chain until a feasible kill chain is found or all track—>firing chains are traversed. Since all track—>firing chains are saved in advance, we save the search process of the shortest path, which greatly reduces the time complexity of the kill chain generation module.

[0152] In addition, in the evaluation, the input of the elastic anti-destroy module is the current topology of our side, the enemy target information, the probability of our side nodes being destroyed, the update strategy taken, and the game paradigm. The output is the kill chain that can be generated after destroying part of our side nodes and the enemy targets that can be searched.

[0153] When the update strategy taken is the traditional three update strategies, namely, comparison update, Fermi update and imitation update, the actual destroyed nodes of our side are randomly generated according to the probability of our side nodes being destroyed. Then the destroyed nodes and the corresponding connected edges in the topology structure of our side are deleted. Finally, the new feasible kill chain and the enemy targets that can be searched are calculated by the kill chain generation module for the new topology architecture generated.

[0154] When the update strategy taken is the centralized update strategy, the kill chain that can be generated after destroying part of the nodes is calculated first. In this module, the time complexity of recalculating the kill chain for the topology architecture after destroying part of the nodes is very high. Therefore, in order to reduce the time complexity, we optimize the elastic anti-destroy module. The specific optimization process is as follows.

[0155] In the module, for each subgraph in the topology architecture of our side, the destroyed nodes in the subgraph are randomly generated first. Then the kill chain with the destroyed nodes is deleted in the previously saved potential kill chain. Next, for the enemy targets that can be attacked before but cannot be attacked now, new kill chains are re-detected, and if the kill chain can be detected, the kill chain is saved, and if the new kill chain cannot be detected, the attack on the enemy target is abandoned. After collecting the results of all subgraphs, the final feasible kill chain and the enemy target that can be tracked are obtained.

[0156] The heterogeneous multi-agent system is established, and the comparison update, the imitation update and the Fermi update can only be applied to the strategy update between homogeneous multi-agents, and cannot be applied to the strategy update between heterogeneous multi-agent systems.

[0157] The present application compares the effects of the centralized update strategy method and the traditional strategy update method in various evaluation indexes and the average running time. In the experiment, 1000 nodes of our side and 1000 enemy nodes are randomly generated for confrontation, and the results are as shown in the following table.

[0158] Destroy rate Task coverage Task matching degree Time Comparison update 0.756 0.551 0.756 2381 seconds Fermi update 0.778 0.661 0.778 2311 seconds Imitation update 0.512 -0.35 0.512 4085 seconds Centralized update 0.870 1.205 1.0 3.6 seconds

[0159] In the traditional strategy update method, each node only makes decisions according to its current local information, such as in the comparison update, each node makes connection decisions according to the degree information of the surrounding nodes, so the macroscopic evaluation index performance cannot be guaranteed. In the centralized strategy update method, the center node uniformly collects all the information detected by other nodes, including the enemy targets that can be detected and the enemy targets that can be attacked, and the center node uniformly establishes the feasible attack chain according to the collected information, so the attack chain can be directly established according to the macroscopic target, and the evaluation index can be more effectively evaluated.

[0160] In summary, the present application provides a multi-agent centralized strategy update method and evaluation system based on game confrontation, which establishes a communication undirected graph of our nodes, allocates an ID and a communication port to each node in the communication undirected graph, selects a center node based on the node ID, and determines the path for sending messages between the non-center node and the center node. The center node can be used to summarize local information, so that more reasonable strategies can be formulated in a short time, and the decision-making intelligence level of the multi-agent system can be improved to a certain extent. Compared with the prior art, the technical scheme of the present application can solve the technical problems of long strategy update time and poor update effect of the existing model in the prior art.

[0161] For purposes of the description hereinafter, the terms "upper", "lower", "right", "left", "rear", "front", "vertical", "horizontal", and derivatives thereof shall relate to the application as oriented in the drawing. The terms "forward" and "aft" refer to a direction toward the front of the device and a direction toward the rear of the device, respectively. However, it is to be understood that the application can assume various alternative orientations, except where expressly omitted.

[0162] In addition, it should be noted that the use of "first", "second", and the like herein does not indicate any order, quantity, or importance, but rather is used to distinguish one element from another, and the terms "first", "second", and the like are not otherwise limiting on the scope of the application.

[0163] The application has been described herein in relation to particular embodiments, which are in no way to be considered limiting of the application rather the opposite: the application is to be administered broadly to all embodiments falling within the scope of the appended claims. As such, individual features of the different embodiments can be combined together in other combinations than those explicitly stated in the above description. Also, the order of steps can be changed. It is therefore anticipated that any changes and modifications can be made by those skilled in the art to adapt the application without departing from the central idea of the application. Accordingly, the scope of the application should be determined only by the claims appended hereto.

Claims

1. A multi-agent centralized strategy updating method based on game confrontation, characterized in that: The method comprises: S1, establish a communication undirected graph of our nodes, and assign an ID and a corresponding communication port to each node in the communication undirected graph, where the ID is a sequence number. Our nodes include search nodes, tracking nodes, and firepower strike nodes; S2, each node exchanges IDs with all its neighboring nodes and determines the direction of the edge between each node and each neighboring node based on the size relationship of the IDs; S3, divide the two nodes connected by each edge into incoming neighbor nodes and outgoing neighbor nodes according to the direction of the edge, and divide all nodes into source nodes, receiving nodes and internal nodes according to the direction of the edges between each node and all neighbor nodes; S4, each source node transmits its ID to all neighboring nodes, each internal node waits to receive the IDs from all its incoming neighboring nodes, calculates the minimum value among these IDs and sends it to all outgoing neighboring nodes, each receiving node receives the IDs of all its neighboring nodes and calculates the minimum value among these IDs, the receiving node sends YES to the neighboring node that sent the minimum value to it, and sends NO to the remaining neighboring nodes, if an internal node receives all YES messages, then the internal node sends YES to all neighboring nodes that have sent messages to it, and sends NO to all neighboring nodes that have not sent messages to it, if an internal node receives a NO, then the internal node sends NO to all neighboring nodes ， The source node waits until all the information is received. If the information received by a source node is all YES, the source node remains a candidate, otherwise the source node is no longer a candidate; S5, when a node sends NO to its neighboring node, the direction of the corresponding edge is reversed; S6, repeat S3 to S5 until only one node is left as a candidate, and the node with the candidate identity is regarded as the central node, and the remaining nodes are regarded as non-central nodes; S7, each non-central node determines its own forward node according to the central node, and the central node obtains a path for each non-central node to send a message to the central node based on the forward node; S8, generating an empty graph based on the friendly node and the enemy node, connecting edges on the empty graph according to the information of the enemy node, and generating a tracking and attack graph in combination with the communication undirected graph of the friendly node; S9, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path through which each non-central node sends messages to it, the tracking strike map, the tracking node and the fire strike node, and generates a final topology structure based on the generated strike chain.

2. The method according to claim 1, characterized in that Determining the direction of the edge between each node and each neighboring node based on the size relationship of the ID includes: Compare the IDs of the current node and its neighboring nodes. If the ID of the current node is smaller than the ID of the neighboring node, the direction of the corresponding edge is from the current node to the neighboring node. If the ID of the current node is larger than the ID of the neighboring node, the direction of the corresponding edge is from the neighboring node to the current node.

3. The method according to claim 2, characterized in that The two nodes connected by each edge are divided into incoming neighbor nodes and outgoing neighbor nodes according to the direction of the edge, including: If the direction of the edge is from the current node to the neighboring node, the neighboring node is the outgoing node of the current node, and the current node is the incoming node of the neighboring node. If the direction of the edge is from the neighboring node to the current node, the current node is the outgoing node of the neighboring node, and the neighboring node is the incoming node of the current node.

4. The method according to claim 3, characterized in that According to the direction of the edges between each node and all neighboring nodes, all nodes are divided into source nodes, receiving nodes and internal nodes, including: If the direction of the edges between a node and all its neighboring nodes is from the node to the corresponding neighboring node, then the node is the source node; If the direction of the edges between a node and all its neighboring nodes is that the corresponding neighboring nodes are facing the node, then the node is a receiving node; If a node is neither a source node nor a sink node, it is an internal node.

5. The method according to claim 4, characterized in that Establishing a communication undirected graph of our node and assigning an ID and a corresponding communication port to each node in the communication undirected graph includes: dividing the communication undirected graph into multiple sub-communication undirected graphs, and assigning an ID and a corresponding communication port to each node in each sub-communication undirected graph.

6. The method according to claim 5, characterized in that The central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path of each non-central node sending a message to the central node, the tracking strike map, the tracking node, and the fire strike node, including: For the current enemy node that can be covered by the search node, the tracking node and the fire strike node are fully permuted and traversed, and each pair of track->firing chains that can cover the current enemy target is traversed. Then, the shortest path of track->firing is searched in the tracking and firing graph, and it is judged whether the firing chain is feasible. If feasible, the nodes are connected in the empty graph, and the traversal is continued for the next enemy node. If not feasible, the next pair of track->firing firing chains is continued for the current enemy node.

7. The method according to claim 5, characterized in that The method includes: when it is impossible to obtain information about a real enemy node, obtaining information about a hypothetical enemy node by sampling a given enemy target probability distribution multiple times, and S8 includes: According to the information of the imaginary enemy node obtained in each sampling, edges are connected on the empty graph, and the corresponding tracking and attack graph is generated in combination with the communication undirected graph of our node.

8. The method according to claim 7, characterized in that The central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path of each non-central node sending a message to the central node, the tracking strike map, the tracking node, and the fire strike node, including: For the information of the hypothetical enemy node obtained by each sampling, the central node generates a strike chain for the enemy nodes that can be covered by the search node based on the path through which each non-central node sends messages to it, the tracking and strike map corresponding to the information of the hypothetical enemy node obtained by a single sampling, and the tracking node and the fire strike node.

9. The method according to claim 8, characterized in that The final topology generated based on the generated strike chain includes: The strike chain generated based on the information of the hypothetical enemy node obtained by each sampling generates a corresponding sub-topology structure; The number of connections of the nodes in all sub-topological structures is statistically sorted, and the edges in the empty graph are connected in descending order to obtain the final topological structure.

10. An evaluation system for evaluating the centralized strategy update method according to any one of claims 1 to 9, characterized in that: The evaluation system includes a graph connectivity evaluation module, a task matching evaluation module, a kill rate evaluation module, a task coverage evaluation module and a resilience and survivability evaluation module; The graph connectivity evaluation module is used to evaluate the connectivity capability of the multi-agent topology structure updated by the centralized strategy update method; the task matching evaluation module is used to evaluate the potential ability of the multi-agent topology structure updated by the centralized strategy update method to destroy enemy targets; the destruction rate evaluation module is used to evaluate the ability of the multi-agent topology structure updated by the centralized strategy update method to perform destruction tasks; the task coverage evaluation module is used to evaluate the rationality of the multi-agent topology structure updated by the centralized strategy update method; and the elastic anti-destruction evaluation module is used to evaluate the anti-strike capability of the multi-agent topology structure updated by the centralized strategy update method.

11. The evaluation system according to claim 10, wherein: The graph connectivity evaluation module is used to evaluate the connectivity of the multi-agent topology structure updated by the centralized strategy updating method according to the following formula: C=w1C1+w2C2, In the above formula, C represents connectivity, C1 represents the number of connected branches in the multi-agent topology structure, C2 represents the average in-degree and out-degree of nodes in the multi-agent topology structure, w1<0 represents the weight coefficient of the number of connected branches, and w2>0 represents the weight coefficient of the average in-degree and out-degree of nodes.

12. The evaluation system according to claim 10, wherein: The task matching evaluation module is used to evaluate the potential ability of the multi-agent topology structure updated by the centralized strategy updating method to destroy the enemy target according to the following formula: K=a1K1+a2K2, In the above formula, K represents the task matching degree, K1 represents the average number of kill chains of enemy targets, K2 represents the average length of kill chains, a1>0 represents the weight coefficient of the average number of kill chains of enemy targets, a2<0 represents the weight coefficient of the average length of kill chains, N c Represents the total number of kill chains for enemy targets, and M represents the number of enemy targets.

13. The evaluation system according to claim 10, wherein: The destruction rate evaluation module is used to evaluate the ability of the multi-agent topology structure updated by the centralized strategy updating method to perform the destruction task according to the following formula: In the above formula, D represents the destruction rate, and m represents the number of enemy targets destroyed by the multi-agent topology after the mission is completed.

14. The evaluation system according to claim 10, wherein: The task coverage evaluation module is used to evaluate the rationality of the multi-agent topology structure updated by the centralized strategy updating method according to the following formula: In the above formula, T represents the task coverage, represents the number of enemy targets that the multi-agent topology can destroy, Indicates the number of enemy targets that can be monitored.

15. The evaluation system according to claim 10, wherein: The resilience and invulnerability evaluation module is used to evaluate the anti-attack capability of the multi-agent topology structure updated by the centralized strategy update method according to the following formula: In the above formula, R represents resilience and indestructibility, C, K, D, and T respectively represent the connectivity, task matching, destruction rate, and task coverage of the multi-agent topology structure before a certain proportion of nodes are damaged, and C', K', D', and T' respectively represent the connectivity, task matching, destruction rate, and task coverage of the multi-agent topology structure after a certain proportion of nodes are damaged.

Images