Model-free optimal grouping cooperative control method for unmanned system swarm under event triggering

Abstract

The present invention claims a model-free optimal grouping collaborative control method for unmanned system clusters under event triggering. Unmanned system cluster control is a typical application of multi-agent system consistency collaborative control. This patent content uses a multi-agent system for specific elaboration. It includes: using matrix theory to transform the dynamic model of the second-order agent to obtain the dynamic equation containing position and velocity information; designing a local neighborhood tracking error protocol based on cooperative competitive interaction to obtain the local neighborhood tracking error of the agent; using the neighborhood tracking error and Bellman optimal principle to establish the performance index function of the agent, and calculate the optimal control strategy of the agent; using a neural network-based adaptive dynamic programming method to fit the optimal control; setting an event trigger threshold for each agent, and when the local neighborhood tracking error of the agent exceeds the threshold, updating its control strategy, and finally achieving the optimal grouping consistency of the multi-agent system.

Description

Technical Field

[0001] The present invention relates to the field of multi-agent system control. Unmanned system cluster control is a typical application of multi-agent system consistent collaborative control. The multi-agent system is used for specific description in this patent content. Background Art

[0002] In recent decades, the consensus control of multi-agent systems has been greatly developed and widely used, such as autonomous mobile robot formation control, swarm, multi-UAV formation control, satellite formation flight, sensor network target tracking, etc. It should be noted that the related work on the consensus control of multi-agent systems mainly focuses on the design of system models and the acquisition of stability conditions, but rarely considers the optimal performance. Therefore, the optimal coordination control problem has become a hot and important research topic because it can greatly reduce the control cost and has important practical application value. Traditional optimal control relies on the analysis of the coupled Hamilton-Jacobi-Bellman equation. However, solving this equation is a very difficult task. Adaptive dynamic programming method is considered to be an effective and potential method to solve this problem, which combines the advantages of optimal control and reinforcement learning. However, most of the existing consensus research works using adaptive dynamic programming methods must rely on accurate system models. But in practical applications, due to the complex structure, it is difficult to obtain accurate system models for most systems, which limits their scope of application. In addition, these works usually use periodic sampling control between agents, which consumes a lot of system resources.

[0003] At present, most of the work on optimal control of multi-agent systems is based on first-order agents, but in fact, most mechanical systems are second-order dynamic systems, such as swarms of vehicles, robots, drones, etc. In addition to position information, we also need to know their velocity information to accurately describe their dynamic characteristics. In addition, existing work rarely involves competitive interactions between agents, but in fact, agents need to compete for limited system resources and cooperate with each other to complete consistency tasks. Therefore, it is necessary to study the cooperative and competitive interactions between agents. Finally, as the scale and complexity of multi-agent systems increase, they may need to be divided into different subnets, and the consensus values ​​of agents may vary with changes in the environment or tasks.

[0004] After searching, the application publication number CN108897229A is a leader-follower ratio consistency control method for a second-order multi-agent system. The method determines the follower agents and the leader according to the topological structure of the network, and determines a directed spanning tree structure with the leader as the root node; secondly, according to the expected proportional relationship of the multi-agent states, the proportional parameters are selected for the follower agents so that the second-order multi-agent states reach the expected specified proportional relationship; finally, when one or more follower agents can obtain the leader's information, a multi-agent leader-follower ratio consistency protocol is designed to achieve that the states of the follower agents strictly follow the states of the leader in a certain proportion, and the states of all follower agents achieve proportional consistency.

[0005] This patent is similar to the above patents only in the leader-follower model. The innovation of this patent focuses on cooperative competition network, multi-agent optimal control, and event triggering. The model of the system is just a material that anyone can use, such as writing words on paper. The focus is on what is made of this material. Summary of the invention

[0006] The present invention aims to solve the above problems of the prior art. A model-free optimal grouping collaborative control method for unmanned system clusters under event triggering is proposed. Unmanned system cluster control is a typical application of multi-agent system consistency collaborative control. The multi-agent system is used in this patent content for specific explanation. The technical solution of the present invention is as follows:

[0007] A model-free optimal grouping collaborative control method for an unmanned system cluster under event triggering comprises the following steps:

[0008] S1. First, obtain the initial position and velocity information of the agent, and use matrix theory to transform the dynamic model of the second-order agent to obtain the dynamic equation that contains both the position and velocity information of the agent;

[0009] S2, processing the state information of the agents respectively according to the local neighborhood tracking error protocol including cooperative competitive interaction of the agents, and obtaining the local neighborhood tracking error of the agents;

[0010] S3, using the neighborhood tracking error and Bellman optimal principle to establish the performance index function of the intelligent agent, and calculate the optimal control strategy of the intelligent agent;

[0011] S4, fitting the optimal control strategy of the agent using an adaptive dynamic programming method based on an actor-critic neural network;

[0012] S5, setting an event trigger threshold for each agent, and updating its control strategy when the local neighborhood tracking error of the agent exceeds the threshold;

[0013] S6. The agent continuously updates its state information according to the control strategy, and finally achieves the optimal grouping consensus of the multi-agent system.

[0014] Furthermore, S1 uses matrix theory to transform the dynamics model of the second-order agent to obtain a dynamics equation that contains both the position information and the speed information of the agent, specifically including:

[0015] The model of the second-order agent is a leader-follower model, where the dynamic equation of the follower agent is:

[0016]

[0017] Among them, x i (k),u i (k), v i (k) represents the position information, control input and speed information of the agent respectively; matrices A, B, C and T i is the system parameter matrix and is completely unknown; k is the time step of the agent iteration;

[0018] The dynamic equation of the leader agent is:

[0019]

[0020] Where x0(k)∈R n ,v0(k)∈R m They are the position and velocity information of the leader agent respectively;

[0021] The converted agent dynamics model is:

[0022]

[0023] The state of the follower agent is The state of the leader agent is Matrix P, S i is the new system parameter matrix after conversion, matrix A, B, C and T i is the original system parameter matrix and is completely unknown; k is the time step of the agent iteration;

[0024] Furthermore, the S2 processes the agent state information according to the local neighborhood tracking error protocol including cooperative competitive interaction of agents to obtain the local neighborhood tracking error of the agent, specifically including:

[0025] The local neighborhood tracking error protocol involving cooperative competitive interaction of intelligent agents includes:

[0026]

[0027] Among them, Γ ij is the competition coefficient between agents, Γ ij <0 means that there is a competitive interaction between agent i and agent j. On the contrary, Γ ij > 0 represents cooperative interaction between agents; if agent i can receive communication information from agent j, then a ij > 0, otherwise, a ij = 0; if and only if follower agent i has an in-degree edge from the leader agent, b i > 0, otherwise, b i =0. j (k) is the state information of agent j after conversion, N i represents the neighboring agents of agent i, δ i (k) is the local neighborhood tracking error protocol of agent i.

[0028] Furthermore, S3 uses the neighborhood tracking error and Bellman optimal principle to establish the performance index function of the intelligent agent and calculate the optimal control strategy of the intelligent agent, which specifically includes:

[0029] The performance index function of the agent is:

[0030] J i (δ i (k),u i (k),u j (k))=U i (δ i (k),u i (k),u j (k))+αJ i (δ i (k+1),u i (k+1),u j (k+1))

[0031] in is the reward function of agent i, α∈(0,1] is the discount factor in the field of reinforcement learning; Q ii ≥0, R ii ≥0, R ij ≥0 is a positive symmetric weighting matrix, usually set to the unit matrix; for the convenience of research, the simplified formula J i (δ i (k),u i (k),u j (k)) = J i (δ i (k)) and U i (δ i (k),ui (k),u j (k))=U i (δ i (k),u i (k));

[0032] The optimal control strategy is:

[0033]

[0034] Among them, b i is the relationship parameter between agent i and the leader agent, if and only if follower agent i has an in-degree edge from the leader agent, b i > 0, otherwise, b i =0.d i is the in-degree of agent i, is the optimal control strategy of agent i, is the matrix R ii The inverse matrix of .

[0035] Furthermore, the S4 uses an adaptive dynamic programming method based on an actor-critic neural network to fit the optimal control strategy of the intelligent agent, specifically including:

[0036] The Actor neural network is:

[0037]

[0038] Among them, k si is the event triggering time of agent i, k si+1 is the next event triggering time of agent i; is the actor neural network weight of agent i, z ci () is the tracking error δ j The input vector of information, ψ ci (·) is the activation function; is the approximate performance indicator function.

[0039] The Critic neural network is:

[0040]

[0041] in, is the approximate optimal control strategy of agent i, ψ ai (·) is the activation function, z ai (·) is the tracking error δ i The input vector of information, The actor neural network weights for agent i.

[0042] In the update rule of the actor-critic neural network weights, only the past and current data of the system are used, and the accurate system model of the multi-agent system is not involved, even if the system matrices P and S i is unknown and will not affect the update of the neural network weights.

[0043] Furthermore, S5 sets an event triggering threshold for each agent, and updates its control strategy when the local neighborhood tracking error of the agent exceeds the threshold, specifically including:

[0044] The event triggering threshold of the agent is:

[0045]

[0046] Where Z = η -σk is a filter function, and the event trigger parameters F, η, σ are positive numbers and satisfy F 2 -2σlnη<0, is the actor neural network weight of agent j, z aj (k sj +1) contains the tracking error δ j (k sj ) information input vector, e i (k) is the event triggering error of the agent; k1 is the first event triggering time of agent i, k si+1 is the next event triggering time of agent i.

[0047] Furthermore, the S6 agent continuously updates its state information according to the control strategy, and finally achieves the optimal grouping consistency of the multi-agent system, which specifically includes:

[0048] When a heterogeneous multi-agent system meets the following conditions, all agents can reach grouping consensus; the conditions met are:

[0049]

[0050] If the i-th agent and the leader agent are in the same subnet, then c i =1, otherwise c i = -1; x0(k) and v0(k) represent the position and speed information of the leader agent at time k, respectively. i (k), v i (k) represents the position and velocity information of agent i at time k respectively.

[0051] Consider a discrete second-order multi-agent system consisting of a leader agent and N follower agents, whose communication topology can be abstracted as a graph in is the set of vertices of the graph, each vertex represents an agent in the multi-agent system; is the set of all the edges of the graph; in a directed graph, a directed path (v i ,v j ) refers to the i To the vertex v j A series of continuous edges {(v i ,v n ),(v n ,v m )...,(v m ,v j )},matrix is the weighted adjacency matrix of the multi-agent system. If agent i can receive the communication information of agent j, then a ij > 0, otherwise, a ij =0; define vertex v i The adjacent vertex set of Then the vertex v i The in-degree is D = diag{d1,d2,...,d N},in The advantages and beneficial effects of the present invention are as follows:

[0052] 1. The multi-agent system of the present invention includes multi-agents in a cooperative-competitive relationship. The following local neighborhood tracking error protocol is designed based on the cooperative-competitive relationship between the agents in the multi-agent system:

[0053]

[0054] Among them, Γ ij is the competition coefficient between agents, Γ ij <0 means that there is a competitive interaction between agent i and agent j. On the contrary, Γ ij >0 represents cooperative interaction between agents; a ij is the adjacency relationship between agent i and agent j. If agent i can receive the communication information of agent j, then a ij > 0, otherwise, a ij = 0; if and only if follower agent i has an in-degree edge from the leader agent, b i > 0, otherwise, b i =0. j (k) is the state information of agent j after conversion, N i represents the neighboring agents of agent i, δ i(k) is the local neighborhood tracking error protocol of agent i. This protocol divides the agents into two groups. Compared with a single cooperative relationship or a competitive relationship, this internal relationship of competition and cooperation is more in line with the internal relationship of a real complex system, which is conducive to the agent system to perform parallel distributed tasks, and ultimately makes the agents in the same group converge to the same value, while the agents in different groups converge to opposite values.

[0055] 2. The multi-agent system of the present invention is a second-order multi-agent system with position information and speed information. Compared with the first-order multi-agent system, the second-order multi-agent system can more accurately describe the actual engineering project system.

[0056] 3. The present invention designs a novel event triggering mechanism including a filtering function for each agent:

[0057]

[0058] Where Z = η -σk is a filter function, and the event trigger parameters F, η, σ are positive numbers and satisfy F 2 -2σlnη<0, is the actor neural network weight of agent j, z aj (k sj +1) contains the tracking error δ j (k sj ) information input vector, e i (k) is the event triggering error of the agent; k1 is the first event triggering time of agent i, k si+1 is the next event triggering time of agent i. The filtering function in this event triggering mechanism can effectively reduce the over-triggering problem caused by the ordinary event triggering mechanism, reduce the control cost, and greatly reduce the consumption of system resources.

[0059] 4. The present invention uses a model-free adaptive dynamic programming algorithm based on a neural network to perform optimal control fitting, so that the algorithm does not need to consider the precise model of the real system, and its application is more extensive.

[0060] The actor neural network is:

[0061]

[0062] in, is the approximate optimal control strategy of agent i, ψ ai (·) is the activation function, z ai (·) is the tracking error δ i The input vector of information, The actor neural network weights for agent i.

[0063] The output error function is defined as:

[0064]

[0065] in, is the critic neural network approximate optimal performance indicator function, parameter Typically set to 0.

[0066] The weight update formula for the actor neural network is designed as:

[0067]

[0068] Among them, the parameters parameter κ ai is the learning rate of the actor neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is a system control flow chart of a preferred embodiment of the present invention;

[0070] Figure 2 is a system topology diagram of an embodiment of the present invention;

[0071] Figure 3 It is a graph of the evolution of the position of an intelligent agent according to an embodiment of the present invention;

[0072] Figure 4 Graph showing the evolution of agent speed according to an embodiment of the present invention.

[0073] Figure 5 This is a diagram of the agent event triggering time of an embodiment of the present invention. DETAILED DESCRIPTION

[0074] Unmanned system cluster control is a typical application of multi-agent system consensus collaborative control, and this patent content uses a multi-agent system for specific explanation. The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and in detail describe the technical solutions in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.

[0075] The technical solution of the present invention to solve the above technical problems is:

[0076] like Figure 1 As shown, a model-free optimal grouping cooperative control method for an unmanned system cluster under event triggering in a cooperative competition network includes but is not limited to the following steps:

[0077] S1. Use matrix theory to transform the dynamic model of the second-order intelligent agent and obtain the dynamic equation that contains both the position information and velocity information of the intelligent agent.

[0078] The second-order agent model is a leader-follower model, where the dynamic equation of the follower agent is:

[0079]

[0080] Among them, x i (k)∈R n , v i (k)∈R m Respectively represent the position information, control input and speed information of the intelligent agent; the system matrix A∈R n×n , B∈R n×m , C∈R m×m as well as They are all completely unknown matrices.

[0081] The dynamic equation of the leader agent is:

[0082]

[0083] Where x0(k)∈R n ,v0(k)∈R m They are the position and velocity information of the leader agent respectively.

[0084] The converted agent dynamics model is:

[0085]

[0086] The state of the follower agent is The state of the leader agent is

[0087] S2. According to the local neighborhood tracking error protocol including cooperative competitive interaction of the agents, the agent state information is processed respectively to obtain the local neighborhood tracking error of the agent.

[0088] The local neighborhood tracking error protocol involving cooperative competitive interaction of intelligent agents includes:

[0089]

[0090] Among them, Γ ij is the competition coefficient between agents. ij <0 means that there is a competitive interaction between agent i and agent j. On the contrary, Γ ij > 0 represents cooperative interaction between agents. If agent i can receive communication information from agent j, then a ij > 0, otherwise, a ij= 0; if and only if follower agent i has an in-degree edge from the leader agent, b i > 0, otherwise, b i =0.

[0091] S3, using the neighborhood tracking error and Bellman optimal principle to establish the performance index function of the intelligent agent, and calculate the optimal control strategy of the intelligent agent;

[0092] The performance index function of the agent is:

[0093] J i (δ i (k),u i (k),u j (k))=U i (δ i (k),u i (k),u j (k))+αJ i (δ i (k+1),u i (k+1),u j (k+1))

[0094] in is the reward function of agent i, α∈(0,1] is the discount factor in the field of reinforcement learning; Q ii ≥0, R ii ≥0, R ij ≥0 is a positive symmetric weighting matrix, usually set to the unit matrix. For the convenience of research, the simplified formula J i (δ i (k),u i (k),u j (k)) = J i (δ i (k)) and U i (δ i (k),u i (k),u j (k))=U i (δ i (k),u i (k)).

[0095] The optimal control strategy is:

[0096]

[0097] S4. Since the Hamilton-Jacobi-Bellman equations for the optimal control strategy of the intelligent agent are very difficult to analyze, an adaptive dynamic programming algorithm based on the Actor-Critic neural network is used.

[0098] Fitting an optimal control policy for the agent.

[0099] The Actor neural network is:

[0100]

[0101] Among them, k si is the event triggering time of agent i, k si+1 is the next event triggering time of agent i. is the actor neural network weight of agent i, z ci () is the tracking error δ j The input vector of information. ci (·) is the activation function.

[0102] The Critic neural network is:

[0103]

[0104] In the update rule of the actor-critic neural network weights, only the past and current data of the system are used, and the accurate system model of the multi-agent system is not involved, even if the system matrices P and S i It is unknown and will not affect the update of the neural network weights. It is more applicable to situations where it is impossible to obtain an accurate system model in actual engineering.

[0105] The adaptive dynamic programming algorithm based on the Actor-Critic neural network includes:

[0106]

[0107]

[0108] S5. Set an event trigger threshold for each agent, and update its control strategy when the local neighborhood tracking error of the agent exceeds the threshold.

[0109] The event triggering threshold of the agent is:

[0110]

[0111] Where Z = η -σk is a filter function, F, η, σ are positive and satisfy F 2 -2σlnη<0, is the actor neural network weight of agent j, z aj (k sj +1) contains the tracking error δ j (k sj ) information input vector, e i(k) is the event triggering error of the agent.

[0112] S6, the agent continuously updates its state information according to the control strategy, and finally achieves the optimal grouping consensus of the multi-agent system;

[0113] In the present invention, when the heterogeneous multi-agent system meets the following conditions, all agents can achieve grouping consistency; the conditions met are:

[0114]

[0115] If the i-th agent and the leader agent are in the same subnet, then c i =1, otherwise c i =-1.

[0116] Consider a discrete second-order multi-agent system consisting of a leader agent and N follower agents, whose communication topology can be abstracted as a graph in is the set of vertices of the graph, each vertex represents an agent in the multi-agent system; is the set of all the edges of the graph; in a directed graph, a directed path (v i ,v j ) refers to the vertex v i To the vertex v j A series of continuous edges {(v i ,v n ),(v n ,v m )...,(v m ,v j )}.matrix is the weighted adjacency matrix of the multi-agent system. If agent i can receive the communication information of agent j, then a ij > 0, otherwise, a ij = 0. Define vertex v i The adjacent vertex set of Then the vertex v i The in-degree is D = diag{d1,d2,...,d N},in i=1,2,...,N.

[0117] In order to verify the effect of the proposed cooperation-competition containment binary grouping consensus algorithm, Matlab is used for simulation verification. In the description of this specification, a node represents an intelligent agent.

[0118] Consider having Figure 2The multi-agent system with the communication topology shown in the figure, where node 0 represents the leader agent and the others are follower agents. The multi-agent system shown in the figure is divided into two subnets, with agents 1, 3, and 6 forming subnet one and agents 2, 4, 5, and 7 forming subnet two. For the multi-agent system, set the system and network parameters as follows:

[0119] System Matrix: S1=[0.019 0.38] T , S2=[0.026 0.43] T , S3 = [0.0410.52] T , S4 = [0.0250.37] T , S5 = [0.0330.40] T , S6 = [0.0210.30] T , S7 = [0.0420.43] T . Pinning gain parameters: b1 = 1, b2 = b3 = b4 = b5 = b6 = b7 = 0. Cooperative competition coefficient: Γ 21 =-0.1,Γ 25 =2.1,Γ 31 =Γ 42 =Γ 54 =Γ 75 =1,Γ 63 =3,Γ 67 =-1, other coefficients are 0; discount factor: α = 0.95; learning rate: κ ci =κ ai =0.03; event trigger parameters: F=0.47, σ=0.8, η=1.23.

[0120] From the simulation results, it can be concluded that Figure 3 , Figure 4 The figure shows the evolution of the position state and speed state of all agents. The agent nodes 2, 4, 5, and 7 form subnet 2, which is controlled by the leader agent node 0. Therefore, the agents in subnet 2 eventually reach the same state (position state and speed state) as the leader agent. The agent nodes 1, 3, and 6 form subnet 1, which reaches the opposite state (position state and speed state) of the leader. The convergence value that the agents in the same group tend to be consistent and the agents in different groups tend to be opposite is achieved, which is in line with the internal relationship of cooperation-competition and reaches the state of group consistency. Figure 5 The triggering moment of the intelligent agent under the event triggering mechanism designed by the present invention is demonstrated. It can be clearly seen that compared with the time triggering mode in which the triggering is performed at every moment, under the event triggering mechanism, the triggering moment of the intelligent agent is discontinuous and greatly reduced.

[0121] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0122] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A model-free optimal grouping collaborative control method for unmanned system clusters under event triggering, characterized in that: The following steps are involved: S1. First, obtain the initial position and velocity information of the agent, and use matrix theory to transform the dynamic model of the second-order agent to obtain the dynamic equation that contains both the position and velocity information of the agent; S2, processing the state information of the agents respectively according to the local neighborhood tracking error protocol including cooperative competitive interaction of the agents, and obtaining the local neighborhood tracking error of the agents; S3, using the neighborhood tracking error and Bellman optimal principle to establish the performance index function of the intelligent agent, and calculate the optimal control strategy of the intelligent agent; S4, fitting the optimal control strategy of the agent using an adaptive dynamic programming method based on an actor-critic neural network; S5, setting an event trigger threshold for each agent, and updating its control strategy when the local neighborhood tracking error of the agent exceeds the threshold; S6, the agent continuously updates its state information according to the control strategy, and finally achieves the optimal grouping consensus of the multi-agent system; The S1 uses matrix theory to transform the dynamics model of the second-order intelligent agent to obtain a dynamics equation that contains both the position information and the speed information of the intelligent agent, specifically including: The model of the second-order agent is a leader-follower model, where the dynamic equation of the follower agent is: Among them, x i (k),u i (k), v i (k) represents the position information, control input and speed information of the agent respectively; matrices A, B, C and T i is the system parameter matrix and is completely unknown; k is the time step of the agent iteration; The dynamic equation of the leader agent is: Among them, x0(k) and v0(k) are the position and velocity information of the leader agent respectively; The converted agent dynamics model is: The state of the follower agent is The state of the leader agent is Matrix P, S i is the new system parameter matrix after conversion, matrix A, B, C and T i is the original system parameter matrix and is completely unknown; k is the time step of the agent iteration; The S2 processes the state information of the agent according to the local neighborhood tracking error protocol including cooperative competitive interaction of the agents to obtain the local neighborhood tracking error of the agent, specifically including: The local neighborhood tracking error protocol involving cooperative competitive interaction of intelligent agents includes: Among them, Γ ij is the competition coefficient between agents, Γ ij <0 means that there is a competitive interaction between agent i and agent j. On the contrary, Γ ij >0 represents cooperative interaction between agents; a ij is the adjacency relationship between agent i and agent j. If agent i can receive the communication information of agent j, then a ij > 0, otherwise, a ij = 0; if and only if follower agent i has an in-degree edge from the leader agent, b i > 0, otherwise, b i =0;y j (k) is the state information of agent j after conversion, N i represents the neighboring agents of agent i, δ i (k) is the local neighborhood tracking error protocol of agent i; The S3 uses the neighborhood tracking error and Bellman optimal principle to establish the performance index function of the intelligent agent and calculate the optimal control strategy of the intelligent agent, which specifically includes: The performance index function of the agent is: J i (δ i (to i (to j (k))=U i (δ i (to i (to j (k))+αJ i (δ i (k+1),u i (k+1),u j (k+1)) in is the reward function of agent i, α∈(0,1] is the discount factor in the field of reinforcement learning; Q ii ≥0, R ii ≥0, R ij ≥0 is a positive symmetric weighting matrix, usually set to the unit matrix; for the convenience of research, the simplified formula J i (δ i (k),u i (k),u j (k)) = J i (δ i (k)) and U i (δ i (k),u i (k),u j (k))=U i (δ i (k),u i (k)); The optimal control strategy is: Among them, b i is the relationship parameter between agent i and the leader agent, if and only if follower agent i has an in-degree edge from the leader agent, b i > 0, otherwise, b i =0,d i is the in-degree of agent i, is the optimal control strategy of agent i, is the matrix R ii The inverse matrix of The S4 uses an adaptive dynamic programming method based on an actor-critic neural network to fit the optimal control strategy of the intelligent agent, specifically including: The critic neural network is: Among them, k si is the event triggering time of agent i, k si+1 is the next event triggering time of agent i; is the critic neural network weight of agent i, z ci (·) is the tracking error δ i The input vector of information, ψ ci (·) is the activation function; is the approximate performance index function; The critic neural network is: in, is the approximate optimal control strategy of agent i, ψ ai (·) is the activation function, z ai (·) is the tracking error δ i The input vector of information, The actor neural network weights of agent i; In the update rule of the actor-critic neural network weights, only the past and current data of the system are used, and the accurate system model of the multi-agent system is not involved, even if the system parameter matrices P and S i It is unknown and will not affect the update of the neural network weights; S5 sets an event trigger threshold for each agent, and updates its control strategy when the local neighborhood tracking error of the agent exceeds the threshold, specifically including: The event triggering threshold of the agent is: Where Z = η -σk is a filter function, and the event trigger parameters F, η, σ are positive numbers and satisfy F 2 -2σlnη<0, is the actor neural network weight of agent j, z aj (k sj +1) contains the tracking error δ j (k sj ) information input vector, e i (k) is the event triggering error of the agent; k1 is the first event triggering time of agent i, k si+1 is the next event triggering time of agent i; The S6 agent continuously updates its state information according to the control strategy, and finally achieves the optimal grouping consistency of the multi-agent system, specifically including: When a heterogeneous multi-agent system meets the following conditions, all agents can reach grouping consensus; the conditions met are: If the i-th agent and the leader agent are in the same subnet, then the containment parameter c i =1, otherwise c i = -1; x0(k) and v0(k) represent the position and speed information of the leader agent at time k, respectively. i (k), v i (k) represents the position and velocity information of agent i at time k respectively; Consider a discrete second-order multi-agent system consisting of a leader agent and N follower agents, whose communication topology can be abstracted as a graph in is the set of vertices of the graph, each vertex represents an agent in the multi-agent system; is the set of all the edges of the graph; in a directed graph, a directed path (v i ,v j ) refers to the i To the vertex v j A series of continuous edges {(v i ,v n ),(v n ,v m )...,(v m ,v j )},matrix is the weighted adjacency matrix of the multi-agent system. If agent i can receive the communication information of agent j, then a ij > 0, otherwise, a ij =0; Define vertex v i The adjacent vertex set of Then the vertex v i The in-degree is D = diag{d1,d2,...,d N },in

Images