Method for establishing complex continuous action iterative dilemma model with incremental dynamics

Abstract

The application discloses a method for establishing a complex continuous action iterative dilemma model with incremental dynamics, and belongs to the technical field of artificial intelligence, and comprises the following steps: step 1, establishing an original intelligent agent communication topology; step 2, continuously updating the incremental action iterative dilemma; step 3, all drones containing the incremental intelligent agent after updating continue to carry out the game; and step 4, analyzing the convergence of the value-increment evolution dynamics. Through the above method, the complex continuous action iterative dilemma model with incremental dynamics is used to describe the dynamic dimension change of the drone system, so that the ability of the intelligent agent to face the complex and changeable environment in the drone group combat is improved. Secondly, the incremental updating method is used to avoid invalid refreshing of the original drone group, the relationship between the drones is represented by a weighted adjacency matrix, and the incremental updating method is used to update the state matrix and the adjacency matrix, so that the dynamic of the intelligent agent when the number of the intelligent agent changes can be better analyzed.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of artificial intelligence, and particularly relates to a method for establishing a complex continuous action iterative dilemma model with incremental dynamics. BACKGROUND

[0002] In recent years, the evolution of network agent social cooperation has attracted widespread attention, and the modeling analysis and stability application research of evolutionary game in time change have developed rapidly. Evolutionary game regards the adjustment process of group behavior as a dynamic system, and prisoner's dilemma and snowball game are regarded as classic models and are deeply studied. In order to be more in line with the actual situation, people have been paying attention to the evolutionary game in continuous strategy space. However, these models are limited by their binary nature, and the dimension change of intelligent agents and some simulation defects have not been paid attention to.

[0003] The intelligent agents in CAID are dynamic, and new intelligent agents can join at any time of the game. In the dynamic updating process of the traditional method, repeated refreshing will waste computing resources.

[0004] So far, many works have studied the convergence proof of evolutionary game. The large amount of calculation brings great difficulty to the analysis of the convergence of evolutionary game. Liapunov function can be used to analyze linear invariant system, and distinguish the stability of nonlinear system and time-invariant system.

[0005] In summary, in view of the dimension change of intelligent agents in game and simulation defects, and the waste of computing resources caused by repeated refreshing and existing calculation methods in the dynamic updating process of traditional method.

[0006] Based on this, the application designs a method for establishing a complex continuous action iterative dilemma model with incremental dynamics to solve the above problems. SUMMARY

[0007] In view of the above shortcomings of the prior art, the application provides a method for establishing a complex continuous action iterative dilemma model with incremental dynamics.

[0008] To achieve the above purpose, the application is implemented by the following technical solutions:

[0009] The method for establishing a complex continuous action iterative dilemma model with incremental dynamics comprises the following steps:

[0010] Step 1, obtaining initial unmanned aerial vehicle system state information, establishing an original intelligent agent communication topology;

[0011] Step 2, continuously updating the incremental action iterative dilemma; the system requests an incremental intelligent agent, and updates the communication topology and the state of the unmanned aerial vehicle by using an incremental method;

[0012] Step 3, the whole UAVs containing incremental agents after updating continue to game;

[0013] Step 4, the convergence analysis of the incremental evolution dynamics, the convergence of the evolution dynamics model of Lyapunov function is used.

[0014] Further, the specific process of step 1 comprises:

[0015] The communication topology of the agent is established, and the initial position information, environmental information and task target position information of the original UAV group are obtained, which are output as image coordinate information, and the number of original agents is set to N;

[0016] The individual and the relationship are represented by a graph , wherein ; is an adjacency matrix, representing the connection relationship in the network; , respectively represent the connection strength of agents i and j, and an undirected graph is used to represent the relationship between UAVs, that is ;

[0017] The strategy evolution rule is introduced: (1);

[0018] wherein, is the energy of UAV i in the complex network, is the strategy adaptation probability of the player, is the difference between the fitness, is the connection relationship, , represent the strategy evolution rule of agents i and j, and let , k=t, the dynamic equation of UAV i is calculated : (2);

[0019] Further, the specific process of step 2 comprises:

[0020] Step 2.1: when the UAV is increased, the incremental agent strategy is obtained by using the incremental updating method, and the evolution dynamics of the incremental strategy is updated;

[0021] Step 2.2: the next strategy is obtained by the incremental strategy acquisition method from the updated evolution dynamics model.

[0022] Further, step 2.1 specifically comprises the following steps:

[0023] Let M be the number of nodes in the incremental system, N be the number of nodes in the original system, be the initial strategy, For incremental strategy, R is a linear space, and the incremental updating method is used to obtain the strategy: (3);

[0024] The rth round of incremental unmanned aerial vehicle is added: (4);

[0025] Wherein is the strategy of the r incremental agents, r M+N is the total number of unmanned aerial vehicles in the updating system;

[0026] The network dynamics is obtained from formula (2): (5);

[0027] ;

[0028] Then, the evolution dynamics containing the incremental strategy is: (6);

[0029] Wherein is the strategy of the incremental unmanned aerial vehicle, and is the connection relationship of the original and incremental unmanned aerial vehicles;

[0030] ;

[0031] The updating dynamics formula (6) is: (7);

[0032] Further, step 2.2 specifically comprises the following steps:

[0033] According to formula (2), there is: (8);

[0034] For the incremental unmanned aerial vehicle, formula (8) is written as: (9);

[0035] Wherein, is the identity vector, denotes the Kronecker product, denotes the Hadamard product;

[0036] When the incremental unmanned aerial vehicle is added, the strategy of the participant is updated; the current strategy of the participant is obtained by using the incremental updating method formula (4), and the next strategy is obtained by using the incremental strategy acquisition method formula (9) through the updated evolution dynamics model formula (7).

[0037] Further, the specific process of the step 3 comprises:

[0038] Step 3.1: The fitness of the agent in the complex network is extended by using the hawk-dove game model, and the evolutionary dynamics model of the hawk-dove game model (CAIHD) is obtained;

[0039] Step 3.2: The fitness of the agent in the complex network is extended by using the chicken game model, and the evolutionary dynamics model of the chicken game model (CAICD) is obtained.

[0040] Further, step 3.1 specifically includes the following steps:

[0041] Assume that the total resource is b, and each hawk will bear a cost of after fierce competition; the payoff matrix of the hawk-dove game is: , wherein b>c (10);

[0042] The fitness of the agent i is calculated as: (11);

[0043] The difference in fitness is calculated as: (12);

[0044] Wherein is the cooperation level of the agent k, , are the connection relationships between the agents;

[0045] Substitute formula (12) into formula (2) to obtain the evolutionary dynamics model of the hawk-dove game model (CAIHD), wherein .

[0046] Further, step 3.2 specifically includes the following steps:

[0047] The loser who gives in first in the game gets a return of 0, and the brave one gets a benefit a, and both giving in has a benefit , and the payoff matrix is: , a>d (13);

[0048] The difference in fitness is calculated as: (14);

[0049] Substitute formula (14) into formula (2) to obtain the evolutionary dynamics model of the chicken game model (CAICD), wherein .

[0050] Advantages

[0051] The application adopts a complex continuous action iterative dilemma model with incremental dynamics to describe dynamic dimension changes of a UAV system, and improves the ability of intelligent agents to face complex and changeable environments in UAV group combat; secondly, the incremental updating method is used to avoid invalid refreshing of the original UAV group, a weighted adjacent matrix is used to represent the relationship between UAVs, and the incremental updating method is used to update the state matrix and the adjacent matrix, so that the dynamic of the intelligent agent when the number of intelligent agents changes can be better analyzed.

[0052] In addition, the application is analyzed based on Lyapunov function, the convergence of CAID under incremental dynamics is proved, and the effectiveness of the method is verified through simulation, and the calculation cost is effectively reduced.

[0053] The application is specifically applied to UAV group combat, and can improve the response ability of intelligent agents to complex and changeable conditions, promote machine self-learning and cooperative evolution, speed up the updating of game strategies, and reduce the waste of computing resources. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0055] Figure 1 The flow chart of the method for establishing the complex continuous action iterative dilemma model with incremental dynamics of the application;

[0056] Figure 2 The schematic diagram of the method for establishing the complex continuous action iterative dilemma model with incremental dynamics of the application;

[0057] Figure 3 The network instance diagram of the method for establishing the complex continuous action iterative dilemma model with incremental dynamics of the application;

[0058] Figure 4 The incremental method updating flow chart of the method for establishing the complex continuous action iterative dilemma model with incremental dynamics of the application;

[0059] Figure 5 The coward game model schematic diagram of the method for establishing the complex continuous action iterative dilemma model with incremental dynamics of the application;

[0060] Figure 6 The dynamic behavior evolution diagram of CAIHD in different networks;

[0061] Figure 7Dynamic behavior evolution for CAIHD and CAICD Figure 1 ;

[0062] Figure 8 Dynamic behavior evolution for CAIHD and CAICD Figure 2 . DETAILED DESCRIPTION

[0063] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0064] The present application is further described below with reference to the embodiments.

[0065] Embodiment 1

[0066] Please refer to the drawings in the description Figures 1-8 , the method for establishing a complex continuous action iterative dilemma model with incremental dynamics includes the following steps:

[0067] Step 1, obtaining initial unmanned aerial vehicle system state information, establishing an original intelligent agent communication topology;

[0068] The specific process of step 1 includes:

[0069] The communication topology of the intelligent agent is established, and the initial position information, environmental information and task target position information of the original unmanned aerial vehicle group are obtained, which are output as image coordinate information, and the number of original intelligent agents is set to N;

[0070] Use a graph to represent individuals and their relationships, where ; is an adjacency matrix representing the connection relationship in the network; , respectively represent the connection strength of intelligent agents i and j, and an undirected graph is used to represent the relationship between unmanned aerial vehicles, that is ;

[0071] is the fitness of intelligent agent i;

[0072] ;

[0073] wherein, is a benefit matrix, , , , represents the payoff of each player; is the cooperation level of agent i;

[0074] The strategy evolution rule is introduced: (1);

[0075] wherein, is the energy of UAV i in the complex network, is the player strategy adaptation probability, β is a parameter that plays a scaling role, and in the final simulation process of this paper, β is taken as 1, is the difference in fitness, is the connection relationship, , represents the strategy evolution rule of agents i and j; let , ij A is a calculation replacement, has no specific meaning, k = t, and the dynamics equation of UAV i is calculated :

[0076] Step 2, continuous action iterative dilemma incremental update; as shown in the accompanying Figure 4 , when the environmental conditions change, the system requests incremental agents, and the communication topology and the state of the UAV are updated using the incremental method;

[0077] The specific process of the step 2 comprises:

[0078] Step 2.1: When the UAV is increased, the incremental agent strategy is obtained by using the incremental update method, and the evolution dynamics of the incremental strategy is updated;

[0079] Specifically, the following steps are included:

[0080] Let M nodes in the incremental system and N nodes in the original system, is the initial strategy, is the incremental strategy, R is a linear space, and the strategy is obtained by using the incremental update method: (3);

[0081] The rth round of incremental UAV is added: (4);

[0082] wherein is the strategy with r incremental agents, r M+N is the total number of UAVs in the updated system;

[0083] The network dynamics is obtained from formula (2): , (5);

[0084] wherein, ;

[0085] ;

[0086] Then, the evolutionary dynamics of the incremental strategy is: (6);

[0087] wherein is the strategy of the incremental UAV, x is the initial strategy, is the network dynamics model, and is the connection relationship of the original and incremental UAVs; A is the adjacency matrix corresponding to the strategy , of the incremental UAV, for simplicity, it is defined as: ;

[0088] The updating dynamics formula (6) is: (7);

[0089] Step 2.2: Obtain the next strategy by the incremental strategy acquisition method from the updated evolutionary dynamics model.

[0090] Comprising the following steps:

[0091] According to formula (2), we have: (8);

[0092] For the incremental UAV, formula (8) is written as: (9);

[0093] wherein is the identity vector, denotes the Kronecker product, denotes the Hadamard product;

[0094] When the incremental UAV is added, the current strategy of the participant is obtained by using the incremental updating method formula (4), and the next strategy is obtained by the incremental strategy acquisition method formula (9) from the updated evolutionary dynamics model formula (7).

[0095] Step 3, the whole UAV containing the incremental agent after updating continues to play the game, as shown in the accompanying drawings. Figure 6

[0096] The specific process of the step 3 comprises:

[0097] Step 3.1: The fitness of the agent in the complex network is obtained by using the hawk-dove game model, and the evolutionary dynamics model of the hawk-dove game model (CAIHD) is obtained.

[0098] Comprising the following steps:

[0099] ​​Assume that the total resource is b, and each hawk will bear the cost after fierce competition The payoff matrix of hawk-dove game is: Where b>c (10);

[0100] The fitness of agent i is calculated as: (11);

[0101] Where, , is the cooperation level;

[0102] The difference in fitness is calculated as: (12);

[0103] Where is the cooperation level of agent k, , are the connection relationships between agents;

[0104] Substitute formula (12) into formula (2) to obtain the evolutionary dynamics model of the hawk-dove game model (CAIHD), where ;

[0105] Step 3.2: Use the chicken game model to extend the fitness of agents in complex networks to obtain the evolutionary dynamics model of the chicken game model (CAICD), as shown in the attached Figure 5 ;

[0106] Including the following steps:

[0107] The loser who gives in first in the game gets a return of 0, and the brave one gets a benefit a, while both giving in has a benefit Both of them insist on driving will bear the cost d, then the payoff matrix is: a>d (13);

[0108] The difference in fitness is calculated as: (14);

[0109] Substitute formula (14) into formula (2) to obtain the evolutionary dynamics model of the chicken game model (CAICD), where ; represents the connection strength of agent i, is the cooperation level of agent i;

[0110] Step 4, convergence analysis of value-added incremental evolutionary dynamics, use Lyapunov function evolutionary dynamics model of convergence;

[0111] To prove its effectiveness in different networks, experiments are conducted in regular networks and scale-free networks, as shown in the attachedFigures 7-8 As shown, the results prove its convergence in these cases;

[0112] In order to achieve the optimal game result of the UAV group combat, the existing evolutionary game model is broken through the limitation of binary property, the evolution of the dimension of the intelligent agent can be better simulated, the complex continuous action iterative dilemma model with incremental dynamics is used to describe the dynamic dimension change of the UAV system, the ability of the intelligent agent to face the complex and changeable environment in the UAV group combat is improved, secondly, the original UAV group is avoided by the incremental updating method Invalid refresh, the relationship between the UAVs is represented by a weighted adjacency matrix, the state matrix and the adjacency matrix are updated by the incremental updating method, the dynamic of the intelligent agent when the number of intelligent agents changes can be better analyzed, furthermore, the convergence of CAID under the incremental dynamics is proved based on Lyapunov function, and the effectiveness of the method is verified through simulation, and the calculation cost is effectively reduced;

[0113] The application is also suitable for cooperation, competition and artificial intelligence field in the development of society and technology, and the construction of man-machine playing simulation neural network, and focuses on providing an effective framework in group intelligence and multi-machine playing, such as robot football battle and multi-robot path planning.

[0114] The application is specifically applied to the UAV group combat, when the external environment changes or enemy intelligent agent information is detected, the incremental intelligent agent is added and the state of the UAV group is updated, and the combat strategy is updated, and the specific implementation manner is as follows: an original system communication topology is established in the original UAV group interaction network, and initial state information of a cluster system model is acquired; the UAV carries exploration information to join the original cluster, the system requests an incremental intelligent agent, and the communication topology between the UAV groups is updated by using incremental information; the state of the UAV group is updated according to the dynamics model of the cluster system and the updated communication topology; the whole UAV group containing the incremental intelligent agent continues to play, and the next combat strategy is obtained; in the UAV group combat, the implementation can improve the response ability of each intelligent agent to complex and changeable conditions, promote machine self-learning and cooperative evolution, speed up the updating of the game strategy, and reduce the waste of computing resources.

[0115] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements will not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for establishing a complex iterative dilemma model of continuous actions with incremental dynamics, characterized in that, Includes the following steps: Step 1: Obtain initial UAV system status information and establish the initial intelligent agent communication topology; Step 2, Incremental Update in the Continuous Action Iteration Dilemma: The system requests an incremental agent to update the communication topology and the state of the UAV using an incremental method. Step 2.1: When an additional drone is added, the incremental agent policy is obtained using the incremental update method, and the evolution dynamics of the incremental policy are updated. Suppose the incremental system has M nodes and the original system has N nodes. As the initial strategy, For an incremental strategy, where R is a linear space, the strategy is obtained using an incremental update method: (3); The rth round of incremental drone additions: (4); in It is a strategy with r incremental agents. This updates the total number of drones in the system; The network dynamics are obtained from formula (2): (5); ； in, , It is the energy of drones in complex networks. It is the probability of player strategy adaptation. It represents a connection relationship; The evolutionary dynamics including the incremental strategy are as follows: (6); in It is an incremental drone strategy. and For the connection relationship between the original and incremental drones, For corresponding strategies The adjacency matrix; ； Update the dynamic formula (6): (7); Step 2.2: Obtain the next policy from the updated evolutionary dynamics model using the incremental policy acquisition method; the formula (9) for the incremental policy acquisition method is: (9); in, It is an identity vector. Indicates the Kronecker product. Indicates Adama's product; Step 3: After the update, all drones including the incremental agents continue the game. Step 4: Convergence analysis of incremental evolution dynamics.

2. The method for establishing a complex continuous action iterative dilemma model with incremental dynamics according to claim 1, characterized in that, The specific process of step 1 includes: Establish the communication topology of the intelligent agents, and obtain the initial position information, environmental information, and mission target position information of the original UAV swarm. The output is image coordinate information, and the number of original intelligent agents is set to N. Using diagrams Indicates individuals and their relationships, where ; This is an adjacency matrix, representing the connection relationships in the network; , Let i and j represent the connection strengths of agents i and j, respectively. An undirected graph is used to represent the relationships between drones, i.e. ; Introducing the laws of strategy evolution: (1); in, It is the energy of drones in complex networks. It represents the player's strategy adaptation probability, where β is the parameter that plays a scaling role. Due to poor adaptability, For connection relationship, , To represent the policy evolution law of agents i and j, let k=t, and calculate the dynamic equation of drone i. : (2).

3. The method for establishing a complex continuous action iterative dilemma model with incremental dynamics according to claim 2, characterized in that, Step 2.2 specifically includes the following steps: According to formula (2), we have: (8); When adding drones Update; use the incremental update method formula (4) to obtain the current strategy of the participants, and then use the updated evolutionary dynamics model formula (7) to obtain the next strategy through the incremental strategy acquisition method formula (9).

4. The method for establishing a complex continuous action iterative dilemma model with incremental dynamics according to claim 3, characterized in that, The specific process of step 3 includes: Step 3.1: Extend the fitness of agents in complex networks using the Eagle-Dove game model to obtain the evolutionary dynamics model of the Eagle-Dove game model; Step 3.2: Extend the fitness of agents in complex networks using the coward game model to obtain the evolutionary dynamics model of the coward game model.

5. The method for establishing a complex continuous action iterative dilemma model with incremental dynamics according to claim 4, characterized in that, Step 3.1 specifically includes the following steps: Assuming the total resources are b, after fierce competition, each eagle will bear the burden. The cost; the payoff matrix of the hawk-dove game is: , where b>c (10) Calculate the fitness of agent i: (11); Calculate the difference in fitness: (12); in For the cooperation level of agent k, , All of these are connection relationships between intelligent agents; Substituting formula (12) into formula (2), we obtain the evolutionary dynamics model of the eagle-dove game model.

6. The method for establishing a complex continuous action iterative dilemma model with incremental dynamics according to claim 4, characterized in that, Step 3.2 specifically includes the following steps: In a game theory game, the loser who concedes first receives a reward of 0, while the braver player gains benefit 'a'. Concessions at all levels yield a benefit. If all insist that driving will incur a cost d, then the payoff matrix is: , a>d (13; Calculate the difference in fitness: (14); Substituting formula (14) into formula (2), we obtain the evolutionary dynamics model of the coward game model.

Images