Input-limited optimal formation tracking control method based on adaptive dynamic programming

Abstract

The invention relates to an input-limited optimal formation tracking control method based on adaptive dynamic programming. The method comprises the following steps: firstly, establishing a formation tracking consistency error model and a cluster communication model of each agent, and designing a feedforward control rate; defining a performance index function of each agent; secondly, deriving a Hamiltonian-Jacobian-Bellman equation corresponding to the defined performance index function according to a Bellman optimality principle, and solving an optimal feedback control rate theoretical formula; and finally, establishing a single evaluation network structure by using an adaptive dynamic programming method, performing approximate solution on the optimal feedback control rate, and designing a network weight update rate. The invention establishes an input-limited optimal formation tracking control method based on adaptive dynamic programming, and the method can be used for unmanned aerial vehicle cluster formation tracking control.

Description

Technical Field

[0001] The present invention belongs to the field of cluster collaborative control, and specifically relates to an input-constrained optimal formation tracking control method based on adaptive dynamic programming, which can complete the tracking of a specified trajectory by a cluster according to a specific formation configuration under input-constrained conditions and obtain optimal control performance. Background Art

[0002] With the rapid development of UAV technology, the application demand of swarm formation is increasing day by day. In emergency search and rescue, formation performance and environmental monitoring, multi-agent system is required to complete related tasks through formation. However, swarm formation tracking control requires that the agents meet the formation geometric configuration constraints while tracking the predetermined trajectory, which makes the formation tracking control task more demanding and difficult. Studying the control algorithm of swarm formation tracking is the key to ensure that the swarm can give full play to the advantages of multi-agent intelligence.

[0003] At present, the commonly used formation control methods include behavioral method, leader-follower method, virtual structure method and artificial potential field method. The leader-follower method and artificial potential field method are simple to calculate and easy to implement; the virtual structure method can avoid the interference problem of the actual leader individual in the leader-follower method. Compared with the behavioral method, the artificial potential field method has better real-time performance and is suitable for systems with high latency requirements such as quadcopters, but it is difficult to solve the problem of non-holonomic kinematic constraints. In terms of formation transformation, the artificial potential field method is more flexible than the pilot method and can meet the functional requirements of coordinated flight of systems such as quadcopters. However, the above control methods are difficult to guarantee the control performance of the cluster, such as the energy consumption of cluster control and the performance of the transition process. In addition, the control amount that can be provided in the actual application scenario is bounded, and this constraint should be fully considered when designing the control algorithm. Therefore, studying the optimal formation tracking control algorithm for clusters under input constraints is of great significance to improving the working efficiency of clusters and giving full play to the working ability of clusters.

[0004] The adaptive dynamic programming method provides a solution to the optimal control problem. At present, the control algorithm based on the adaptive dynamic programming method has achieved many research results in the field of single trajectory tracking control, cluster consistency control and inclusion control, but the optimal control algorithm in the field of formation tracking control under the condition of cluster input constraints still needs further research. Summary of the invention

[0005] In order to complete the cluster formation tracking control task under input constraints, the present invention provides an optimal formation tracking control algorithm based on an adaptive dynamic programming method. The optimal controller is designed according to the Bellman optimality principle, and the optimal controller is solved by the adaptive dynamic programming method. The cluster control performance can be guaranteed while completing the formation tracking task.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] An input-constrained optimal formation tracking control method based on adaptive dynamic programming comprises the following steps:

[0008] Step (1): Establish the tracking consistency error system of each intelligent agent formation and the cluster communication model, and design the feedforward control rate ;

[0009] Step (2), define a performance indicator function of the input restricted type for each agent;

[0010] Step (3), combined with the performance index function of each agent in step (2) , derive the corresponding Hamilton-Jacobi-Bellman equation and solve the optimal control rate Theoretical formula;

[0011] Step (4), using the adaptive dynamic programming method to establish a single evaluation network structure, the optimal control rate Perform approximate solution and design the update rate of evaluation network weights;

[0012] Furthermore, the step (1) comprises:

[0013] The cluster system consists of The same intelligent agents are composed, and the leader-follower formation control method is adopted. The dynamic equation of each intelligent agent is:

[0014] ;

[0015] in, , The requirement is that it is Lipschitz continuous and contains the origin. , represents the feedforward control quantity, Represents the feedback control amount.

[0016] The leader dynamics model is expressed as:

[0017] ;

[0018] in, Indicates the virtual leader status.

[0019] Agent The formation information dynamics model is:

[0020] ;

[0021] Definition The error of an agent is:

[0022] ;

[0023] Taking the derivative of the above error, we get:

[0024] ;

[0025] In order to save communication resources and reduce computing pressure, some followers cannot communicate directly with the leader and can only communicate with neighboring followers. In order to ensure that the cluster as a whole can synchronize with the leader and form a designated formation, it is necessary to ensure that the leader can communicate with some followers. The communication model of the cluster adopts a directed graph. describe, represents a non-empty point set, Represents a set of edges between ordered sets of points. Respectively represent An intelligent agent, ; Indicated by arrive If , indicating the The agent can receive The adjacency matrix corresponding to this graph is Indicates that if ,but ,on the contrary The degree matrix corresponding to this graph is ,in . Define the leader adjacency matrix , Representing an Agent It can receive the leader's information, otherwise it cannot be received. The following consistency error is defined for each follower:

[0026] ;

[0027] in, Representing an Agent A collection of individuals that are capable of communicating.

[0028] The above consistency error is derived to obtain the following consistency error dynamics model:

[0029] ;

[0030] in, express With Agent Composed of a collection.

[0031] The feedforward control solution formula is:

[0032] ;

[0033] assumed exists, then the agent The feedforward control can be obtained as:

[0034] ;

[0035] Furthermore, the performance indicator function designed in step (2) is:

[0036] ;

[0037] in, is a positive definite symmetric matrix, is a positive definite integral function, defined as:

[0038] ;

[0039] in, , , , is the maximum control input;

[0040] Furthermore, the step (3) includes:

[0041] The corresponding HJB equation obtained by deriving the performance index function of each agent is:

[0042] ;

[0043] The optimal feedback control under input constraints can be solved as:

[0044] ;

[0045] Furthermore, the step (4) comprises:

[0046] For intelligent agents , its value function and its derivative can be approximated as:

[0047] ;

[0048] in, Representing an Agent The ideal weight parameter of the evaluation network is Representing an Agent The activation function, represents the partial derivative of the activation function with respect to the consistency error, represents the approximation error and is bounded, represents the partial derivative of the approximation error with respect to the consistency error.

[0049] The following standard assumptions are made for the approximation formula:

[0050] (1) Approximation error of each agent’s value function and its partial derivative with respect to the consistency error is bounded, satisfying the condition , , is a constant greater than 0

[0051] (2) Activation function of each agent and its partial derivative with respect to the consistency error is bounded, satisfying the condition , , is a constant greater than 0

[0052] (3) Ideal weight of each agent’s evaluation network is bounded, satisfying the condition , is a constant greater than 0

[0053] Define the actual weight of the evaluation network as , the optimal feedback control rate can be approximated as:

[0054] ;

[0055] Substituting into the HJB equation, the approximate error is:

[0056] ;

[0057] in,

[0058] The optimization objective function is defined as:

[0059] ;

[0060] The weight update rate of the evaluation network designed using the gradient descent method is:

[0061] ;

[0062] in,

[0063] The present invention can control the cluster to track a specified trajectory and maintain a specified geometric formation configuration.

[0064] Compared with the existing technology, the present invention has the following beneficial effects:

[0065] (1) Taking into account the constraints of limited input of the intelligent agent, the present invention designs an input-constrained optimal formation tracking control algorithm based on adaptive dynamic programming. Compared with the traditional formation tracking control method, the control algorithm of the present invention has controllable control rate amplitude, better control effect, and lower energy consumption.

[0066] (2) The control algorithm designed by the present invention only uses the information of the intelligent agent and its neighboring intelligent agents, and is a distributed controller. Compared with a centralized controller, the control algorithm of the present invention has lower requirements for cluster communication and a smaller computational burden.

[0067] (3) The present invention utilizes a neural network to construct a single evaluation network structure for online approximate solution of the optimal controller. Compared with the offline iterative solution of the optimal control method, the present invention has real-time and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A flowchart of an input-constrained optimal formation tracking control method based on adaptive dynamic programming of the present invention;

[0069] Figure 2 This is a schematic diagram of formation tracking control in leader-follower mode;

[0070] Figure 3 Schematic diagram of the neural network structure of a single evaluation network. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0072] According to one embodiment of the present invention, Figure 1 As shown, the specific implementation steps of the input-constrained optimal formation tracking control method based on adaptive dynamic programming of the present invention are:

[0073] Step (1), establish the tracking consistency error system of each intelligent agent formation and the cluster communication model, the cluster object is composed of The same intelligent agents are composed, and the leader-follower formation control method is adopted. The dynamic equation of each intelligent agent is:

[0074] ;

[0075] in, represents the agent number, Representing an Agent The state quantity, express dimensional real number space, Representing an Agent The state derivative of represents the nonlinear term of the state quantity in the dynamic equation, represents the control quantity coefficient term in the dynamic equation, Representing an Agent The total control volume, express dimensional real number space, The requirement is that it is Lipschitz continuous and contains the origin. , represents the feedforward control quantity, Represents the feedback control amount.

[0076] The leader dynamics model is expressed as:

[0077] ;

[0078] in, Indicates the virtual leader status, represents the virtual leader state derivative, Represents the nonlinear term of the state quantity in the virtual leader dynamics model.

[0079] Agent The formation information dynamics model is:

[0080] ;

[0081] in, Represents the state of the formation, represents the derivative of the formation state quantity, Represents the nonlinear term of formation state in the formation information dynamics model.

[0082] Definition The error of the agent for:

[0083] ;

[0084] Taking the derivative of the above error, we get:

[0085] ;

[0086] For clusters, use Figure 2In the leader-follower formation tracking control mode shown in the figure, in order to save communication resources and reduce computing pressure, some followers cannot communicate directly with the leader and can only communicate with neighboring followers. In order to ensure that the cluster as a whole can synchronize with the leader and form a designated formation, it is necessary to ensure that the leader can communicate with some followers. The communication model of the cluster adopts a directed graph describe, represents a non-empty point set, Represents a set of edges between ordered sets of points. Respectively represent An intelligent agent, ; Indicated by arrive If , indicating the The agent can receive Information about an agent. Directed graph The corresponding adjacency relationship is represented by the adjacency matrix It means that the adjacency matrix No. Line The column elements are ,like ,but ,on the contrary . Directed Graph The corresponding degree matrix ,in , represents a diagonal matrix, Represents the number of agents in the cluster. Define the leader adjacency matrix , Representing an Agent Can receive leader information, otherwise cannot receive it.

[0087] For each follower, the following consistency error is defined: :

[0088] ;

[0089] in, Representing an Agent A collection of individuals that are capable of communicating.

[0090] The above consistency error is derived to obtain the following consistency error dynamics model:

[0091]

[0092] in, express With Agent The collection composed of represents the derivative of the consistency error, Representing an Agent The state quantity, Representing an Agent The total control volume, Represents the leader adjacency matrix No. Line Column elements, Represents a directed graph The Laplace matrix of No. Line Column elements, directed graph The Laplace matrix of Defined as .

[0093] The feedforward control solution formula is:

[0094] ;

[0095] assumed exists, then the agent Feedforward control It can be obtained as:

[0096] ;

[0097] Step (2), define a performance indicator function of input restricted type for each agent; for the agent Designed performance indicator function for:

[0098] ;

[0099] in, Indicates the starting time, Representing an Agent The feedback control quantity, represents the integral variable, is a positive definite symmetric matrix, with the superscript represents the transpose of a matrix, is a positive definite integral function, defined as:

[0100] ;

[0101] in, represents the integral variable, represents the weight matrix of feedback control quantity, Indicates dimensional feedback control weight, represents the weight vector of feedback control quantity, All 1 dimensional column vector, is the maximum control input;

[0102] Step (3), combined with the performance index function of each agent in step (2) , derive the corresponding Hamilton-Jacobi-Bellman equation and solve the optimal control rate Theoretical formula of

[0103] The corresponding Hamilton-Jacobi-Bellman equation is obtained by deriving the performance index function of each agent:

[0104] ;

[0105] in, represents the Hamilton–Jacobi–Bellman equation, Representing an Agent The optimal feedback control quantity is represents the integral variable, Representing an Agent The optimal performance indicator function The derivative of .

[0106] The optimal feedback control under input constraints can be solved as:

[0107] ;

[0108] in, Representing an Agent The set of allowable control quantities, represents the inverse matrix of the feedback control weight matrix, Represents a directed graph The Laplace matrix of No. Line Column elements, Represents the leader adjacency matrix No. Line Column elements, is the maximum control input, Representing an Agent The optimal performance indicator function The derivative of .

[0109] Step (4), using the adaptive dynamic programming method to establish a single evaluation network structure, the optimal control rate Perform an approximate solution and design the weight update rate of the evaluation network.

[0110] The designed single evaluation network structure of each intelligent agent is as follows Figure 3As shown, Represents the agent consistency error No. Item element. For Agent , its value function and its derivatives It can be approximated as:

[0111] ;

[0112] in, Representing an Agent The ideal weight parameter of the evaluation network is Representing an Agent The activation function, represents the partial derivative of the activation function with respect to the consistency error, represents the approximation error and is bounded, represents the partial derivative of the approximation error with respect to the consistency error.

[0113] The following standard assumptions are made for the approximation formula:

[0114] (1) Approximation error of each agent’s value function and its partial derivative with respect to the consistency error is bounded, satisfying the condition , , represents the modulus of the orientation quantity, is a constant greater than 0;

[0115] (2) Activation function of each agent and its partial derivative with respect to the consistency error is bounded, satisfying the condition , , is a constant greater than 0;

[0116] (3) Ideal weight of each agent’s evaluation network is bounded, satisfying the condition , is a constant greater than 0;

[0117] Define the actual weight of the evaluation network as , the approximate value of the optimal feedback control rate for:

[0118] ;

[0119] in, represents the optimal feedback control rate approximation, represents the inverse matrix of the feedback control weight matrix, represents the partial derivative of the activation function with respect to the consistency error, Representing an Agent The ideal weight parameters of the evaluation network.

[0120] Substituting into the Hamilton-Jacobi-Bellman equation, the approximate error is:

[0121] ;

[0122] in, Representing an Agent The Hamilton-Jacobi-Bellman equation approximation error is .

[0123] Define the optimization objective function for:

[0124] ;

[0125] The weight update rate of the evaluation network designed using the gradient descent method is:

[0126] ;

[0127] in, Representing an Agent Evaluation network weight learning rate,

[0128] in, Representing an Agent The approximate value of the optimal feedback control quantity is: Representing an Agent The formation status information.

[0129] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, and it should be clear that the present invention is not limited to the scope of the specific embodiments, for those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. An input-constrained optimal formation tracking control method based on adaptive dynamic programming, characterized in that: The steps include: Step (1), establish the tracking consistency error system and cluster communication model of each intelligent agent formation, and design the feedforward control rate; Step (2), define a performance indicator function of the input restricted type for each agent; Step (3), combined with the performance index function of each agent in step (2) , derive the corresponding Hamilton-Jacobi-Bellman equation and solve the optimal control rate Theoretical formula; Step (4), using the adaptive dynamic programming method to establish a single evaluation network structure, the optimal control rate Perform an approximate solution and design the weight update rate of the evaluation network.

2. The input-constrained optimal formation tracking control method based on adaptive dynamic programming according to claim 1, characterized in that: The step (1) comprises: The cluster system consists of The same intelligent agents are composed, and the leader-follower formation control method is adopted. The dynamic equation of each intelligent agent is: ， in, , The requirement is that it is Lipschitz continuous and contains the origin. , represents the feedforward control quantity, represents the feedback control quantity, The leader dynamics model is expressed as: ， in, Indicates the leader status, Agent The formation information dynamics model is: ， Definition The error of an agent is: ， Taking the derivative of the above error, we get: ， In order to save communication resources and reduce computing pressure, some followers cannot communicate directly with the leader and can only communicate with neighboring followers. In order to ensure that the cluster as a whole can synchronize with the leader and form a designated formation, it is necessary to ensure that the leader can communicate with some followers. The communication model of the cluster adopts a directed graph. describe, represents a non-empty point set, represents the set of edges between ordered sets of points, Respectively represent An intelligent agent, ; Indicated by arrive If , indicating the The agent can receive The information of each agent, the adjacency relationship corresponding to this graph is represented by the adjacency matrix Indicates that if ,but ,on the contrary , the degree matrix corresponding to this graph is ,in , define the leader adjacency matrix , Representing an Agent It can receive the leader's information, otherwise it cannot be received. The following consistency error is defined for each follower: ， in, Representing an Agent A collection of individuals who can communicate. The above consistency error is derived to obtain the following consistency error dynamics model: ， in, express With Agent The collection composed of represents the Laplacian matrix diagonal elements, the directed graph Laplacian matrix is ​​defined as , The feedforward control solution formula is: ， assumed exists, then the agent The feedforward control can be obtained as: 。 3. The input-constrained optimal formation tracking control method based on adaptive dynamic programming according to claim 1, characterized in that: The performance indicator function of step (2) is: ， in, is a positive definite symmetric matrix, is a positive definite integral function, defined as: ， in, , , , is the maximum control input.

4. The input-constrained optimal formation tracking control method based on adaptive dynamic programming according to claim 1, characterized in that: The step (3) comprises: The corresponding HJB equation obtained by derivation of each agent's performance index function is: ， The optimal feedback control under input constraints can be solved as: 。 5. The input-constrained optimal formation tracking control method based on adaptive dynamic programming according to claim 1, characterized in that: The step (4) comprises: For intelligent agents , its value function and its derivative can be approximated as: ， in, Representing an Agent The ideal weight parameter of the evaluation network is Representing an Agent The activation function, represents the partial derivative of the activation function with respect to the consistency error, represents the approximation error and is bounded, represents the partial derivative of the approximation error with respect to the consistency error, The following standard assumptions are made for the approximation formula: (1) Approximation error of each agent’s value function and its partial derivative with respect to the consistency error is bounded, satisfying the condition , , is a constant greater than 0, (2) Activation function of each agent and its partial derivative with respect to the consistency error is bounded, satisfying the condition , , is a constant greater than 0, (3) Ideal weight of each agent’s evaluation network is bounded, satisfying the condition , is a constant greater than 0, Define the actual weight of the evaluation network as , the optimal feedback control rate can be approximated as: ， Substituting into the HJB equation, the approximate error is: ， in, , The optimization objective function is defined as: ， The weight update rate of the evaluation network designed using the gradient descent method is: ， in, .

Images