A method and system for multi-agent network evolutionary game analysis

Abstract

The application provides a method and system for multi-agent network evolution game analysis, comprising: determining the type of network topology structure formed by current multi-agent interaction; if it is a static network structure, initializing the strategy of the current agent and executing the strategy update timing rule judgment process; if it is a dynamic network structure, initializing the strategy of the current agent, executing the agent movement state update process first, and then executing the strategy update timing rule judgment process. The application can be applied to the cooperation behavior analysis of any static network structure and dynamically moving agent, and can provide a network structure evolution behavior analysis method from the perspective of static network structure and individual flight.

Description

Technical Field

[0001] This invention relates to the field of game theory technology for multi-agent unmanned systems, and specifically provides a method and system for multi-agent network evolution game analysis. Background Technology

[0002] With the development of artificial intelligence technology, various intelligent unmanned systems have emerged, forming various independent intelligent agents with decision-making capabilities. Coordination among multiple agents can effectively improve the overall performance of the system. Studying multi-agent cooperative behavior helps to understand the emergence of swarm intelligence and social collaboration mechanisms in nature, and also provides theoretical support for engineering applications. Evolutionary game theory provides a systematic analytical tool for studying the cooperative-competitive interaction among groups. With the introduction of zero-determinant strategies, evolutionary game theory has been enriched from the perspective of individual decision-making, providing richer evolutionary paths for the emergence of cooperative behavior in multi-agent network evolutionary games. Network evolutionary game theory, combining game theory and network science theory, provides an important theoretical framework for studying strategy games and the emergence of swarm cooperative behavior in multi-agent systems.

[0003] In complex network environments, the cooperative behavior of multi-agent systems is influenced by topology, interaction rules, and policy update mechanisms. Static networks, with their fixed topology, facilitate the analysis of the formation and evolution of cooperative clusters, and thus most studies are conducted in static networks. However, real-world agents are not static individuals; for example, drones, mobile robots, and vehicle-to-everything (V2X) nodes all possess mobility, and their interactions change dynamically over time. Studies based solely on static networks are insufficient to reveal the true evolutionary patterns of cooperative behavior in dynamic groups.

[0004] Therefore, it is necessary to design a multi-agent evolutionary game analysis system that can be applied to both static and dynamic networks. Summary of the Invention

[0005] To address the aforementioned problems, in a first aspect, the present invention provides a method for multi-agent network evolution game analysis, comprising: determining the type of network topology structure formed by the current interaction of multiple agents; if it is a static network structure, initializing the current agent's policy and executing a policy update timing rule determination process; if it is a dynamic network structure, initializing the current agent's policy and first executing an agent movement state update process, and then executing a policy update timing rule determination process.

[0006] In one technical solution of the above-mentioned method for multi-agent network evolutionary game analysis, the agent's movement state update process specifically includes: the current agent adjusts its flight angle according to preset flight rules, updates its spatial position by combining its own speed with the adjusted angle, and simultaneously calculates the neighbor interaction list within the perception radius.

[0007] In one technical solution of the above-mentioned method for multi-agent network evolution game analysis, the policy update timing rule determination process specifically includes: if it is a synchronous update rule, each agent in the agent cluster completes the game interaction with all its neighbor agents respectively, and calculates its own payoff in this round of game according to the game strategy adopted by itself and its neighbor agents and a preset payoff matrix.

[0008] In one technical solution of the above-mentioned method for multi-agent network evolutionary game analysis, after calculating the self-payment of the current round of the game according to the preset payoff matrix, it further includes: if the neighbor nodes of the current agent are not empty, then derive the policy imitation probability according to the preset learning rules to complete the individual policy update.

[0009] In one technical solution of the above-mentioned method for evolutionary game analysis of multi-agent networks, the process of deriving the policy imitation probability according to preset learning rules and completing the individual policy update specifically includes: calculating the current agent... Learn a randomly selected neighbor The probability of the strategy And update the individual strategy according to the formula: in, , Represents intelligent agents Adopt a strategy, Represents intelligent agents strategy, Indicates the current time step.

[0010] In one technical solution of the above-mentioned method for multi-agent network evolutionary game analysis, after completing the individual policy update, it also includes: recording the agent cluster policy at the current time step and returning to the termination condition judgment step.

[0011] In one technical solution of the above-mentioned method for multi-agent network evolution game analysis, the policy update timing rule determination process further includes: if it is an asynchronous update rule, each agent in the agent cluster randomly selects one of its neighboring agents to complete the game interaction, and calculates its own game gain with the neighboring agent in this round according to the game strategy of both parties.

[0012] In one technical solution of the above-mentioned method for multi-agent network evolution game analysis, after calculating the current game payoff between itself and the neighboring agent based on the game strategies of both parties, the method further includes: if the neighboring nodes of the current agent are not empty, then deriving the strategy imitation probability according to the preset learning rules and completing the individual strategy update.

[0013] In one technical solution of the above-mentioned method for evolutionary game analysis of multi-agent networks, the specific steps of deriving the policy imitation probability according to the preset learning rules and completing the individual policy update include:

[0014] The intelligent agent Calculate your payoff in a game with your neighbor. Randomly select neighbors Neighbors need to engage in a game with their neighbors to generate benefits. and according to the learning rules Calculate the current intelligent agent Learn from your neighbors The probability of the strategy And according to: Update individual strategies, among which If the neighboring nodes of the current agent are not empty, the process includes: deriving the strategy imitation probability according to the preset learning rules and completing the individual strategy update, as well as: saving the game parameters and the steady-state agent strategy.

[0015] Secondly, the present invention provides a system for multi-agent network evolution game analysis, comprising: a network topology type determination module, used to determine the network topology structure type formed by the current multi-agent interaction; a first execution module, used to initialize the current agent's policy and execute a policy update timing rule determination process if the network structure is static; and a second execution module, used to initialize the current agent's policy and first execute an agent movement state update process, and then execute a policy update timing rule determination process if the network structure is dynamic.

[0016] The beneficial effects of this invention are as follows: This invention is applicable to the analysis of cooperative behavior in any static network structure and dynamically moving agents, providing a method for analyzing network structure evolution behavior from the perspectives of static network structures and individual flight. In static networks, it allows for the study of the impact of different topologies on cooperative maintenance; in dynamic networks, it can characterize the mechanisms by which mobility and time-varying interactions affect cooperative behavior. Through a unified analytical framework, the cooperative evolution characteristics under different network environments can be systematically compared. Attached Figure Description

[0017] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:

[0018] Figure 1 This is a schematic flowchart of a method for evolutionary game analysis of multi-agent networks according to an embodiment of the present invention;

[0019] Figure 2 This is a static scale-free network policy evolution game diagram according to an embodiment of the present invention;

[0020] Figure 3 This is a dynamic random flight network strategy evolution game diagram according to an embodiment of the present invention;

[0021] Figure 4 This is a dynamic Vicsek model flight network strategy evolution game diagram according to an embodiment of the present invention. Detailed Implementation

[0022] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0023] Example 1

[0024] like Figure 1 As shown, the present invention provides a method for multi-agent network evolutionary game analysis. In the multi-agent network evolutionary game analysis system, the game parameters are input to generate agents, and the learning rules are input into the evolutionary game. During the evolutionary game process, after the individuals calculate the game payoff, the agent node policies are updated synchronously according to the learning rules, and finally the game results are output for strategy analysis.

[0025] Initialize node size , Maximum number of games Learning rules for And the game parameters are calculated. The payoff matrix is ​​as follows:

[0026]

[0027] intelligent agent Adopting intelligent agents In a game, the payoff can be represented as a Markov process:

[0028]

[0029] strategy set ,in , Exploitative factors , If is a regulating factor that guarantees the probability falls within the range of 0 to 1, then the policy satisfy:

[0030]

[0031] matrix There exists a steady-state vector v that satisfies So, intelligent agents Long-term benefits .

[0032] Step S1: If the current agent has a static network structure, input the agent node and its neighbor information into the system and initialize the agent nodes. strategy ;

[0033] If the current agent is in a dynamic network structure, then set the flight speed of the individual agents in the agent cluster. ,density Perception radius Flight rules Flight space boundary and initialize the agent node. strategy Position in space Current direction and angle Proceed to step S2.

[0034] Step S2: If the maximum number of iterations has been reached, proceed to step S9; otherwise, proceed to step S3.

[0035] Step S3: If the current agent evolution game is a static network structure, proceed to step S5; otherwise, proceed to step S4.

[0036] Step S4: The intelligent agent, according to its own flight rules renew:

[0037]

[0038] And according to speed ,angle Update its own position Calculate the current sensing radius Find the neighbor list within the range and proceed to step S5.

[0039] Step S5: If the current rule is synchronous update, each individual in the agent cluster plays a game with all its current neighbors and calculates the payoff according to the strategy. Proceed to step S6;

[0040] If the current agent is updating rules asynchronously, then proceed to step S7.

[0041] Step S6: If the agent's neighbor nodes are not empty, the agent cluster follows the specified learning rules. Calculate the current intelligent agent Learn a randomly selected neighbor The probability of the strategy And according to the formula:

[0042]

[0043] Update individual strategies, among which Proceed to step S8.

[0044] Step S7: Each individual agent in the agent cluster executes: If the agent's neighbor nodes are not empty, the agent... Calculate your payoff in a game with your neighbor. Randomly select neighbors Neighbors need to engage in a game with their neighbors to generate benefits. and according to the learning rules Calculate the current intelligent agent Learn from your neighbors The probability of the strategy And according to the formula:

[0045]

[0046] Update individual strategies, among which Proceed to step S8.

[0047] Step S8: Record the current time step The agent cluster strategy is to repeat step S2.

[0048] Step S9: Save the game parameters and the steady-state agent strategy.

[0049] like Figure 2-4 As shown, the multi-agent network evolution game analysis system and method proposed in this invention can be used for evolution game analysis of static networks and mobile flight networks. Figure 2 The graph shows the evolution of strategies in a static scale-free network. Blue represents strategy C, red represents strategy D, and green represents strategy E. The three strategies are initially randomly distributed in the scale-free network and evolve over 10,000 time steps. Figure 2 (a), (b) and Figure 2 (c), (d), and larger b inhibit cooperative behavior in the agent cluster.

[0050] Figure 3 This is a game graph illustrating the evolution of strategies in a dynamic random flight network. Blue represents strategy C, red represents strategy D, and green represents strategy E. These three strategies are initially randomly distributed in space, and the agent's initial direction is randomly pointed in any direction within space. Similarly, as... Figure 3 For (a)-(d), at a velocity v=0.001, a larger b inhibits cooperative behavior in the agent cluster; while at the same b, Figure 3(a), (b) and Figure 3 (e) and (f) show that greater speed is detrimental to cooperative behavior among agents.

[0051] Figure 4 This is a game diagram illustrating the evolution of flight network strategies in a dynamic Vicsek model. Blue represents strategy C, red represents strategy D, and green represents strategy E. These three strategies are initially randomly distributed in space, and the agents initially point randomly in any direction within the space. In the Vicsek model, the agents' flight directions synchronize, and higher speeds also inhibit cooperative behavior. Figure 4 (d) and Figure 3 As shown in (f), the Vicsek model has a higher velocity than random flight, and cooperative clusters still exist at time step 10000.

[0052] Example 2

[0053] This invention discloses a system for multi-agent network evolution game analysis, comprising: a network topology type determination module for determining the network topology structure type formed by the current multi-agent interaction; a first execution module for executing a policy update timing rule determination process if the network structure is static; and a second execution module for executing an agent movement state update process first, and then executing a policy update timing rule determination process if the network structure is dynamic.

[0054] The technical solution of the present invention has been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles and objectives of the present invention, those skilled in the art can make equivalent changes or substitutions to the original technical features, specific structures, materials, connection methods, arrangement methods, etc., and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for evolutionary game analysis in multi-agent networks, characterized in that, include: Determine the type of network topology formed by the current multi-agent interactions; If it is a static network structure, the policy of the current agent is initialized and the policy update timing rule determination process is executed; If it is a dynamic network structure, the current agent's policy is initialized and the agent's movement state update process is executed first, followed by the policy update timing rule determination process. The intelligent agent's movement state update process specifically includes: the current intelligent agent adjusts its flight angle according to preset flight rules, updates its spatial position by combining its own speed with the adjusted angle, and simultaneously calculates the neighbor interaction list within the perception radius.

2. The method according to claim 1, characterized in that, The policy update timing rule determination process specifically includes: If the rules are updated synchronously, each agent in the agent cluster will engage in a game with all its neighboring agents. Based on the game strategies adopted by itself and its neighboring agents, it will calculate its own payoff for this round of the game according to the preset payoff matrix.

3. The method according to claim 2, characterized in that, After calculating the payoff for this round of the game according to the preset payoff matrix, the following is also included: If the neighbor nodes of the current agent are not empty, the policy imitation probability is derived according to the preset learning rules to complete the individual policy update.

4. The method according to claim 3, characterized in that, The process of deriving the policy imitation probability according to preset learning rules and completing individual policy updates specifically includes: Calculate the current agent Learn a randomly selected neighbor The probability of the strategy And update the individual strategy according to the formula: in, , Represents intelligent agents Adopt a strategy, Represents intelligent agents strategy, Indicates the current time step.

5. The method according to claim 3, characterized in that, After completing the individual strategy update, it also includes: recording the current time step. The intelligent agent cluster strategy returns to the termination condition judgment step.

6. The method according to claim 2, characterized in that, The policy update timing rule determination process also includes: If the update rule is asynchronous, each agent in the agent cluster randomly selects one of its neighboring agents to complete the game interaction, and calculates its own game gain with that neighboring agent in this round according to the game strategy of both parties.

7. The method according to claim 6, characterized in that, After calculating the payoff for this round of the game between itself and its neighboring agent based on the game strategies of both parties, the following is also included: If the neighboring nodes of the current agent are not empty, the policy imitation probability is derived according to the preset learning rules and the individual policy is updated.

8. The method according to claim 7, characterized in that, The process of deriving the strategy imitation probability according to preset learning rules and completing individual strategy updates specifically includes: The intelligent agent Calculate your payoff in a game with your neighbor. Randomly select neighbors Neighbors need to engage in a game with their neighbors to generate benefits. and according to the learning rules Calculate the current intelligent agent Learn from your neighbors The probability of the strategy And according to: Update individual strategies, among which ; If the neighboring nodes of the current agent are not empty, the process includes: deriving the policy imitation probability according to the preset learning rules and completing the individual policy update, as well as: saving the game parameters and the steady-state agent policy.

9. A system for evolutionary game analysis in multi-agent networks, characterized in that, include: The network topology type determination module is used to determine the type of network topology structure formed by the current multi-agent interaction; The first execution module is used to initialize the current agent's policy and execute the policy update timing rule determination process if the network structure is static. The second execution module is used to initialize the current agent's policy and execute the agent's movement state update process first, and then execute the policy update timing rule determination process, if the network structure is dynamic. The intelligent agent's movement state update process specifically includes: the current intelligent agent adjusts its flight angle according to preset flight rules, updates its spatial position by combining its own speed with the adjusted angle, and simultaneously calculates the neighbor interaction list within the perception radius.

Images