Approximate robust Nash equilibrium method for distributed formation control

Abstract

The invention relates to an approximate robust Nash equilibrium method for distributed formation control, and the method comprises the following specific steps: firstly, building a communication topological structure between agents through a graph theory, and constructing an agent model constrained by external interference; thirdly, converting a distributed formation problem into a distributed robust differential game, and constructing a rolling zero-sum game model based on a contraction time domain theory; secondly, analyzing the existence of a local Nash equilibrium solution by using a Pontryagin minimum principle, and giving an expression form of an optimal strategy; furthermore, terminal state observation is designed to solve an approximate optimal strategy so as to ensure convergence of approximate Nash equilibrium. Finally, it is demonstrated that the difference between the approximate Nash equilibrium and the real Nash equilibrium is related to the initial state of the system from the mathematical perspective, and meanwhile the stability of the system is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-agent coordinated control, and in particular to an approximate robust Nash equilibrium method for distributed formation control. Background Art

[0002] Multi-agent coordinated control is a striking topic in the study of multi-agent systems, which requires each agent to complete the overall task through collaboration. Formation control is particularly important in this process. However, in practical applications, multi-agent systems often face limitations in communication capabilities, which means that changes in the formation of agents may affect the communication relationship with neighbors. Specifically, changes in tasks may cause an agent to lose a neighbor or gain a new neighbor. Therefore, it is particularly necessary to study distributed formation control under communication constraints.

[0003] In formation control, since there may be conflicts between global and local goals, the benefit function of each agent is also different. In order to give full play to the initiative of the agent, it is appropriate to use game theory to solve the formation problem. Game theory is an effective tool for dealing with conflicts between individuals, and differential games have been widely used in individual coordination control.

[0004] The following are studies on differential games in solving formation problems: The literature (Mylvaganam T, Astolfi A. Towards a systematic solution for differential games with limited communication [C] / / 2016 American Control Conference (ACC). IEEE, 2016: 3814-3819.) combines distributed control and optimal control to solve the formation control problem under limited communication constraints, and ultimately ensures that the control strategy can converge to the Nash equilibrium solution. The literature (Fu Y, Chai T. Online solution of two-player zero-sum games for continuous-time nonlinear systems with completely unknown dynamics [J]. IEEE transactions on neural networks and learning systems, 2015, 27 (12): 2577-2587.) studies the formation problem with external interference and transforms the problem into a robust differential game, ensuring the robustness of the control strategy. However, in previous work, there are mainly two problems: first, in formation control, the communication topology is assumed to be fixed, which is unreasonable in practice; second, the external interference problem under distributed formation is not considered, which reduces the adaptability of the control strategy.

[0005] In response to the above two problems, the literature (Lin W, Li C, Qu Z, et al. Distributed formation control with open-loop Nash strategy [J]. Automatica, 2019, 106: 266-273.) combines contraction time domain control with differential game to establish a differential game model based on contraction time domain to solve the formation problem in a dynamic environment. However, this method assumes that the communication connection is fixed and does not consider external interference, which makes the desired Nash equilibrium solution not robust and adaptable.

[0006] Therefore, for the distributed formation problem, we need to consider how to ensure the robustness and adaptability of the formation strategy under the constraints of communication capability and external interference. To this end, an approximate robust Nash equilibrium method for distributed formation control is proposed. This method transforms the distributed formation problem into a robust differential game model by considering the interaction between multiple agents and external interference. In this model, the agent relies on the information of its neighbors to calculate its own behavior. Compared with the existing differential game that is only applicable to static environments, the shrinking time domain control is introduced to improve the transient performance. In order to solve the interference problem, the optimal strategy under the worst interference conditions is sought, and it is proved that the strategy can converge to an approximate Nash equilibrium solution. In addition, a terminal state estimator is designed for the limited communication capability between agents, and a distributed Nash strategy that does not rely on global state information is constructed. In theory, this method ensures the stability of the system and makes the formation strategy robust. Summary of the invention

[0007] In view of this, an object of the present invention is to provide an approximate robust Nash equilibrium method for distributed formation control, aiming to solve the above problems.

[0008] In order to solve the above technical problems, the present invention provides a technical solution: an approximate robust Nash equilibrium method for distributed formation control, comprising the following steps:

[0009] Step S1: construct a linear model of intelligent agents subject to external interference constraints, design the information transmission relationship of intelligent agents through a network topology diagram, and adopt a leader-follower strategy;

[0010] Step S2: In view of the formation adjustment caused by the task change, the formation problem is transformed into a distributed robust differential game, and a rolling zero-sum game model is constructed based on the shrinking time domain theory;

[0011] Step S3: Based on the Pontryagin minimum principle, an optimal control strategy expression is provided that depends on the terminal position information of itself and neighboring agents;

[0012] Step S4: construct a state observer for the participants, estimate the terminal states of the game participants, transmit and receive state estimates through the communication topology graph, solve the optimal strategy based on the above state estimates, and achieve convergence of the approximate Nash equilibrium solution;

[0013] Step S5: Analyze the relationship between the approximate Nash equilibrium solution and the true Nash equilibrium solution to ensure system stability.

[0014] Preferably, in step S1, the leader grasps the target position, and the follower maintains a specific distance from the neighboring intelligent agent, and further includes:

[0015] Step S11: Establishing a system model:

[0016] The dynamic equation of each agent is:

[0017]

[0018] Where: t is the time scale; is the rate of change of the position information of the ith agent at time t; x i (t) is the position information of the i-th agent at time t; u i (t) and ω(t) are the control strategy and external disturbance of the ith agent at time t, respectively; B ii and E ii are the weight matrices corresponding to the control strategy of the ith agent and the external interference;

[0019] The dynamic equation of the multi-agent system is:

[0020]

[0021] In the formula, is the rate of change of the state information of the multi-agent system at time t; B i and E i are the i-th control strategy weight matrix and external interference weight matrix in the multi-agent system respectively.

[0022] Step S12: Distributed architecture establishment:

[0023] There are N agents and N-1 followers, and a communication topology graph G(v,ε) is constructed;

[0024] Where: v = {1,...,N} represents the set of agents, N represents the total number of agents, A set of edges representing connections between agents;

[0025] The neighbor information matrix of agent i is expressed as:

[0026]

[0027] In the formula, is the neighbor matrix of agent i at time t; is an N×1 vector with the jth row being 1 and the rest of the rows being 0; for The transpose of ; I3 is the identity matrix;

[0028] When the formation task changes, the neighbor information matrix of agent i is time-varying, and the neighbor set of agent i is also time-varying. If the formation is at t s The following function is introduced to avoid disconnection of the formation:

[0029]

[0030] In the formula, represents the sigmoid function of agent j at time t, e represents the exponential function, α represents a constant, and each of the agents is constrained by dynamic equations, network topology and external interference.

[0031] Preferably, the step S2 comprises:

[0032] The intelligent agents participating in the formation task are defined as game participants. Based on the idea of ​​shrinking time domain control, the game is replayed in each given rolling window, environmental information is fed back in real time, and a benefit function corresponding to the game participants is constructed.

[0033] Participant i in the time domain window t∈[t k ,t f The benefit function of ] is:

[0034]

[0035] Where, t k is the initial time corresponding to the sampling instant k; u ik (t) and u -ik (t) are the control strategies of the participant at time t corresponding to the sampling instant k and the set of control strategies of the neighboring participants except participant i; ω ik (t) and ω -ik (t) are the interference of the participant at time t corresponding to the sampling instant k and the interference set of the neighboring participants except participant i; x(t k ) is the initial time t corresponding to the sampling instant k k The initial position information of the multi-agent system at time x(t f ) is the terminal time t f The corresponding terminal position information of the multi-agent system; is the terminal time t corresponding to the sampling instant k f The adjustable weight matrix of the ith agent; is the neighbor information matrix of the i-th agent at time t corresponding to the sampling instant k; for The transposed matrix of and are the positive definite weight matrices of the i-th agent corresponding to the sampling instant k;

[0036] Participant i and his neighbor participants converge to a local robust Nash equilibrium, that is, the strategy set at time t corresponding to sampling instant k satisfy:

[0037]

[0038]

[0039] In the formula, and are the optimal strategy of participant i at time t corresponding to sampling instant k and the optimal strategy set of neighbor participants; and are the interference of participant i and the interference set of neighbor participants at time t corresponding to sampling instant k; is the optimal cost of participant i relative to his neighbor participants at time t corresponding to sampling instant k; is the suboptimal cost of participant i at time t corresponding to sampling instant k relative to the neighboring participants when the interference strategy is the largest; is the suboptimal cost of participant i relative to his neighbor participants when the control strategy of participant i at time t corresponding to sampling instant k is optimal.

[0040] Preferably, step S3 comprises:

[0041] The open-loop local robust Nash equilibrium solution of each participant at time t corresponding to sampling instant k satisfies:

[0042] The optimal control strategy form:

[0043]

[0044] Worst forms of interference:

[0045]

[0046] In the formula, I 3N is the 3N-dimensional identity matrix;

[0047] Transformed into the following form:

[0048]

[0049]

[0050] Preferably, step S4 comprises:

[0051] Design the terminal state observer:

[0052]

[0053] In the formula, is the estimated change rate of the terminal time position of participant i at time t corresponding to sampling instant k; is the estimated position information of participant i at time t corresponding to sampling instant k; s i is the gain coefficient of participant i;

[0054] Regarding the final convergence of the estimated terminal state to the true value:

[0055] That is:

[0056]

[0057] In the formula, is the position estimate of the multi-agent system at time t corresponding to sampling instant k; x f (t k ) is the position estimate of the multi-agent system at t k corresponding to sampling instant k; x k (t f ) is the position of the multi-agent system at the end time corresponding to sampling instant k;

[0058] Among them, the estimated optimal control strategy and the worst disturbance respectively satisfy the following forms:

[0059]

[0060]

[0061] And the differences between the above formulas and the true optimal strategy and the worst disturbance strategy are exponentially convergent, that is:

[0062]

[0063]

[0064] In the formula, is the difference between the estimated optimal strategy and the actual optimal strategy of participant i at time t corresponding to sampling instant k; is the difference between the estimated worst disturbance and the actual worst disturbance of participant i at time t corresponding to sampling instant k; Δx f (t k ) is the deviation between the estimated initial position and the true initial position of the multi-agent system at t k corresponding to sampling instant k; γ = min m Re[λ m (SΛ k )] is the minimum value of the real part among the eigenvalues of the product of matrix S and matrix Λ k ; The matrix denotes the diagonal matrix composed of the gain coefficients of N participants; s1 is the gain coefficient of participant 1, s NThe gain coefficient of participant N completes the convergence of the approximate Nash equilibrium solution.

[0065] Preferably, step S5 comprises:

[0066] The control strategy set forms a local robust open-loop approximate Nash equilibrium solution:

[0067]

[0068]

[0069] Where ζ is a function related to the initial state of the system;

[0070] The estimated optimal strategy of participant i and the estimated worst interference form a local robust open-loop approximate Nash equilibrium solution with the neighbors. The difference between the approximate Nash equilibrium solution and the true Nash equilibrium solution at sampling instant k is:

[0071]

[0072] In the formula, ζ ik is the difference between the approximate Nash equilibrium solution of agent i and the true Nash equilibrium solution corresponding to sampling instant k; N represents agent N; where ζ ik Satisfy the following formula:

[0073]

[0074] In the above formula Ψ k , as well as These are the constructed correlation coefficients:

[0075]

[0076]

[0077]

[0078]

[0079]

[0080]

[0081]

[0082]

[0083]

[0084] In a multi-agent system with N agents, agent i updates its terminal estimated state according to the terminal state observer. The agent adopts the estimated optimal strategy and the estimated worst interference to achieve distributed formation.

[0085] The advantages of the present invention over the prior art are as follows: the present invention designs a distributed approximate Nash equilibrium method for the distributed formation control problem by combining shrinking time domain control and differential game theory. Compared with the existing differential game theory which is only applicable to static environments, shrinking time domain control is introduced to improve the transient performance of the system;

[0086] In order to eliminate interference, the optimal strategy under the worst interference is solved, and it is proved that the strategy can converge to an approximate Nash equilibrium solution; in addition, in order to solve the limited communication capability between intelligent agents, a terminal state estimator is designed, and a distributed Nash strategy that does not depend on global state information is constructed, which theoretically guarantees the stability of the system; this method can make the formation strategy robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 is a method principle block diagram of an embodiment of the present invention;

[0088] Figure 2 It is a multi-agent communication topology diagram adopted in the simulation example implemented by the present invention;

[0089] Figure 3 It is a display diagram of the distance deviation between each intelligent agent and the obstacle in the comparison method in the simulation example implemented by the present invention;

[0090] Figure 4 is a diagram showing the distance deviation between each intelligent agent and an obstacle in the method proposed in the simulation example implemented by the present invention;

[0091] Figure 5 It is a diagram showing the distance deviation between each neighboring intelligent agent of the two methods in the simulation example implemented by the present invention;

[0092] Figure 6 It is a diagram showing the distance deviation between each non-neighboring intelligent agent of the two methods in the simulation example implemented by the present invention;

[0093] Figure 7 It is a display diagram of the state deviation of a multi-agent system of the method proposed in the simulation example implemented in the present invention. DETAILED DESCRIPTION

[0094] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0095] like Figure 1 As shown, this embodiment provides an approximate robust Nash equilibrium method for distributed formation control, including the following steps:

[0096] A linear model of the agent is established, and the constraints of external interference are considered. A distributed architecture is used to solve the problem of limited communication between agents. At the same time, the information acquisition relationship between agents is designed through the network topology diagram, and the leader-follower formation strategy is used for formation control, in which the leader knows the location of the target point and the follower maintains a specific formation distance with its neighbors.

[0097] Aiming at the formation adjustment problem caused by task changes, the distributed formation problem is transformed into a distributed robust differential game, and a rolling zero-sum game model is constructed based on the shrinking time domain theory. In the multi-agent system, the controlled objects participating in the formation are regarded as game participants, and their goals are the benefit functions of the participants, which include the operation benefits, control benefits and interference benefits of the participants.

[0098] Using the Pontryagin minimum principle, the existence of local robust Nash equilibrium solutions is explored from a mathematical perspective, and an optimal control strategy expression that only depends on the terminal position information of itself and its neighboring agents is constructed.

[0099] The state observer of the participants is designed to estimate the terminal state of the game participants. According to the communication topology, the estimated state is transmitted to the neighboring participants, and the state estimation from the neighbors is received at the same time. Then, the optimal strategy is solved based on the obtained terminal state, and finally converges to the approximate Nash equilibrium solution.

[0100] The obtained approximate Nash equilibrium solution is theoretically analyzed, which shows that the difference between this solution and the true Nash equilibrium solution is related to the initial state of the system and guarantees the stability of the system.

[0101] In this embodiment, the task of the agent is to enable the multi-agent system subject to communication constraints to achieve the desired formation in an environment with external interference, thereby enhancing the robustness of the system.

[0102] In this embodiment, the dynamic equation of each agent is modeled as:

[0103]

[0104] In the formula, t is the time scale; is the rate of change of the position information of the ith agent at time t; x i (t) is the position information of the i-th agent at time t; u i (t) and ω(t) are the control strategy and external disturbance of the ith agent at time t, respectively; B ii and E ii are the weight matrices corresponding to the control strategy of the ith agent and the external interference respectively.

[0105] Assumption 1: The external disturbance ω(t) is square integrable, that is, for any terminal time t f , there is a constant So that the following equation holds,

[0106]

[0107] Let x(t) [x1(t),...,x i (t),...,x N (t)] is the state of the multi-agent system at time t, where x1(t) and x N (t) are the states of agent 1 and agent N at time t respectively;

[0108] The dynamic equation of the multi-agent system is:

[0109]

[0110] In the formula, is the rate of change of the state information of the multi-agent system at time t; B i and E i are the i-th control strategy weight matrix and external interference weight matrix in the multi-agent system respectively.

[0111] Distributed architecture establishment:

[0112] The communication between the agents is limited, and a distributed architecture is used to calculate the control strategy of each agent. In this architecture, each agent only needs information about itself and its neighbors. A communication topology graph G(v,ε) between agents is established, where v = {1,...,N} represents the set of agents, and N represents the total number of agents; represents the set of connecting edges between agents; the communication relationship between agent i and agent j is represented by e ij Indicates; e ij ∈ε means that agent i can obtain the location and strategy information of agent j, and agent j is called the neighbor of agent i; the neighbor set of agent i is defined as Let the neighbor information matrix of agent i be:

[0113]

[0114] In the formula, is the neighbor matrix of agent i at time t; is an N×1 vector with the jth row being 1 and the rest of the rows being 0; for is the transpose of ; I3 is the identity matrix.

[0115] Note that due to the change of the formation task, the neighbor information matrix of agent i is time-varying, and the neighbor set of agent i is also time-varying, then assuming that the formation is s The following function is introduced to avoid disconnection of the formation.

[0116]

[0117] In the formula, represents the sigmoid function of agent j at time t, e represents the exponential function, and α represents a constant.

[0118] Assumption 2: The communication topology graph G(v,ε) between agents is directed and connected.

[0119] In the distributed formation control problem, there are N agents, among which there is a leader who knows the final target position and N-1 followers. The followers maintain a formation distance with their neighbors. The position error of agent i at time t is defined as:

[0120]

[0121] In the formula, is the position error information of the i-th agent at time t; is the expected position information of the multi-agent system; α i and β i Define the i-th agent as the leader (α i =1, βi=0) or a follower (α i =0,β i =1); x j (t) is the location information of the neighboring agent j of the ith agent at time t; x d is the desired target location; is the expected formation distance between the ith agent and its neighbor agent j;

[0122] Given the following assumptions:

[0123] Assumption 3: The initial positions of each agent do not overlap, that is:

[0124] x i (0)≠x j (0) (7)

[0125] In the formula, x i (0) and x j (0) are the positions of the i-th agent and the j-th agent at the initial moment;

[0126] Assumption 4: The desired formation distance satisfies the following conditions:

[0127]

[0128] In the formula, A i and a i are the sensing radius and safety radius of the i-th agent respectively;

[0129] In order to achieve the goal of enabling the leader to reach the desired target point in a multi-agent system while maintaining the desired distance between followers, it is necessary to determine the optimal control strategy for each agent, where each agent is constrained by the dynamic equation (1), the network topology G(v,ε) and the external disturbance ω(t).

[0130] In order to achieve the control goal of the distributed formation problem of the multi-agent system, the distributed formation problem is modeled as a distributed robust differential game problem. In addition, the agents participating in the formation task are defined as game participants.

[0131] Each game participant has a corresponding benefit function. Note that the participant needs to feedback the control strategy to reflect the actual environment in real time. However, the analytical solution of the differential game with state feedback is difficult to obtain, and the existence of the solution cannot be guaranteed in theory. Therefore, based on the idea of ​​shrinking time domain control, the game is replayed in each given rolling window to feedback the environmental information in real time.

[0132] Participant i in the time domain window t∈[t k ,t f The benefit function of ] is as follows:

[0133]

[0134] Where, t k is the initial time corresponding to the sampling instant k; u ik (t) and u -ik (t) are the control strategies of the participant at time t corresponding to the sampling instant k and the set of control strategies of the neighboring participants except participant i; ω ik (t) and ω -ik (t) are the interference of the participant at time t corresponding to the sampling instant k and the interference set of the neighboring participants except participant i; x(t k ) is the initial time t corresponding to the sampling instant k k The initial position information of the multi-agent system at time x(t f ) is the terminal time t f The corresponding terminal position information of the multi-agent system; is the terminal time t corresponding to the sampling instant k f The adjustable weight matrix of the ith agent; is the neighbor information matrix of the i-th agent at time t corresponding to the sampling instant k; for The transposed matrix of and are the positive definite weight matrices of the i-th agent corresponding to the sampling instant k;

[0135] The control goal of the distributed formation control problem of the multi-agent system is to design the optimal coordination strategy for each game participant constrained by the dynamic equation (1), the network topology G(v,ε) and the external interference ω(t), and to ensure that the leader reaches the desired target point safely while the followers maintain their relative formation positions. In addition, the participant i and its neighbor participants can converge to a local robust Nash equilibrium, that is, the strategy set at time t corresponding to the sampling instant k satisfy:

[0136]

[0137] In the formula, and are the optimal strategy of participant i at time t corresponding to sampling instant k and the optimal strategy set of neighbor participants; and are the interference of participant i and the interference set of neighbor participants at time t corresponding to sampling instant k; is the optimal cost of participant i relative to his neighbor participants at time t corresponding to sampling instant k; is the suboptimal cost of participant i at time t corresponding to sampling instant k relative to the neighboring participants when the interference strategy is the largest; is the suboptimal cost of participant i relative to his neighbor participants when the control strategy of participant i at time t corresponding to sampling instant k is optimal.

[0138] The theorem of the existence of local robust Nash equilibrium solutions is given: for a team consisting of N participants, where the dynamic equation of the participants is (1) and the corresponding benefit function is (9), the open-loop local robust Nash equilibrium solution of each participant at time t corresponding to the sampling instant k satisfies the following form:

[0139]

[0140] In the formula, I 3N is the 3N-dimensional identity matrix.

[0141] To observe the optimal control strategy form (11a) and the worst interference form (11b), each agent needs to know the initial state information of all agents. However, due to the limitations of communication between agents, the forms (11a) and (11b) cannot be realized.

[0142] Formulas (11a) and (11b) can be transformed into the following forms:

[0143]

[0144] According to formulas (12a) and (12b), each agent obtains the optimal control strategy And the worst interference It only needs to know the terminal location information of itself and its neighbors.

[0145] From this point of view, obtaining the local robust open-loop Nash equilibrium solution is transformed into estimating the terminal position information itself.

[0146] Design the terminal state observer as follows:

[0147]

[0148] In the formula, is the estimated change rate of the terminal time position of participant i at time t corresponding to sampling instant k; is the estimated terminal location information of participant i at time t corresponding to sampling instant k; s i is the gain coefficient of participant i;

[0149] The following theorem is given about how the terminal state estimate eventually converges to the true value:

[0150] Consider a differential game involving N players, each player is subject to the dynamic equation (1) and the benefit function (9). If 1) the matrix Λ k All eigenvalues ​​of contain positive real parts. 2) The participants update their terminal states according to the observer (9), then the estimated terminal states of the participants can converge to the true value, that is,

[0151]

[0152] In the formula, is the terminal time position estimate of the multi-agent system at time t corresponding to the sampling instant k; f (t k ) is the t corresponding to the sampling instant k k The multi-agent system estimates the terminal time position at the moment x k (t f) is the position of the multi-agent system at the end time corresponding to the sampling instant k. Among them, the estimated optimal control strategy and the worst interference respectively satisfy the following forms:

[0153]

[0154] And the differences between the above equations (13a) and (13b) and the true optimal strategy and the worst interference strategy respectively are exponentially convergent, that is:

[0155]

[0156] In the formula, is the difference between the estimated optimal strategy and the actual optimal strategy of player i at time t corresponding to the sampling instant k; is the difference between the estimated worst interference and the actual worst interference of player i at time t corresponding to the sampling instant k; Δx f (t k ) is the deviation between the estimated initial position and the true initial position of the multi-agent system at time t corresponding to the sampling instant k; γ = min k Re[λ m (SΛ m )] is the minimum value of the real part among the eigenvalues of the product of matrix S and matrix Λ k ; the matrix k denotes the diagonal matrix composed of the gain coefficients of N players; s1 is the gain coefficient of player 1, s is the gain coefficient of player N; N Based on the above theorem, the players and their neighbors finally converge to an approximate Nash equilibrium solution.

[0157] First, the definition of the approximate Nash equilibrium is given as follows:

[0158] Considering player (1) and the corresponding benefit function (9), the control strategy set

[0159] If the following inequality group is satisfied, then this control strategy set forms a local robust open-loop approximate Nash equilibrium solution.

[0160]

[0161] In the formula, ζ is a certain function related to the initial state of the system.

[0162] Furthermore, it is theoretically shown that the difference between this approximate Nash equilibrium solution and the true Nash equilibrium solution is related to the initial state of the system, and the following related theorem is given.

[0163] ​Consider a differential game (9) with N participants. The dynamic equation of each participant is (1). The estimated optimal strategy (12a) and estimated worst interference (12b) of participant i form a local robust open-loop approximate Nash equilibrium solution with its neighbors, and at the sampling instant k, the difference between the approximate Nash equilibrium solution and the true Nash equilibrium solution satisfies the following equation.

[0164]

[0165] In the formula, ζ ik is the difference between the approximate Nash equilibrium solution and the true Nash equilibrium solution of agent i corresponding to sampling instant k; N represents agent N; Formula (18) indicates that the agent with the largest difference is the difference between the approximate Nash equilibrium solution and the true Nash equilibrium solution at sampling instant k; where, ζ ik Satisfy the following formula.

[0166]

[0167] In the above formula (19) Ψ k , as well as are the constructed correlation coefficients, defined as follows:

[0168]

[0169] According to equations (19)-(20), it can be concluded that it is related to the initial state of the system.

[0170] Consider a multi-agent system with N agents. The dynamic equation of each agent is (1), the benefit function is (9), and agent i updates its terminal estimated state according to the observer (13). Furthermore, the agent adopts the estimated optimal strategy (12a) and the estimated worst interference (12b) to achieve distributed formation if and only if the matrix Λ k All eigenvalues ​​of contain positive real parts, which ultimately ensures the stability of the system.

[0171] In this embodiment, a specific example is given to demonstrate the effectiveness and superiority of the proposed formation strategy method of approximate robust Nash equilibrium solution in solving the multi-agent formation problem with obstacle constraints.

[0172] In order to prove that the obtained approximate robust Nash equilibrium solution can handle unknown obstacles in the environment, this example uses the proposed method to compare with the existing static differential game method (comparison method) and conducts simulation experiments. Taking six agents as an example, the specific model expression of agent i is given as follows:

[0173]

[0174] The communication topology of the four agents is shown in the figure Figure 2 As shown in the figure, agent 1 is defined as the leader and the other three agents are defined as followers. The initial position and the expected target point position of agent 1 are: x1(t0) = [-30, -30, -30] T , x1(t f ) = [300,300,100] T , the initial positions of followers 2 to 4 are: x2(t0)=[-50,-20,-30] T , x3(t0)=[-40,-70,-30] T , x4(t0)=[-20,-50,-30] T ,,,The expected formation distance between follower 2 and leader 1 is: d 12 (t0) = [100, 0, 0] T , the expected formation distance between follower 3 and follower 2 is: d 23 (t0) = [-100, -100, 0] T , the expected formation distance between follower 4 and follower 3 is: d 34 (t0) = [100, -100, 0] T , the sensing radius is A i =25.

[0175] Figure 3 The distance deviation diagram between the agent and the obstacles in the environment of the static robust differential game method (comparison method) is given in Figure 4 The distance deviation diagram between the agent and the obstacles in the environment of the approximate robust Nash equilibrium method (the proposed method) is given in Figure 3 and Figure 4 It can be seen that agent 1 cannot avoid unknown obstacles (obstacle 2) in the environment using the contrast method. However, the proposed method enables all agents to avoid known and unknown obstacles in the environment. Therefore, the proposed strategy of approximate robust Nash equilibrium solution is effective.

[0176] Figure 5 The distance display diagrams between neighboring intelligent agents of the two methods are given, from which it can be seen that both methods can prevent collisions between neighboring intelligent agents. Figure 6 The distance between non-neighboring agents of the two methods is shown in Fig. 2. However, since the distance between agent 1 and agent 3 is less than the safe distance, the comparison method does not consider the collision avoidance problem between non-neighboring agents. The proposed method can avoid collisions between not only neighbors but also non-neighboring agents. Finally, according to Figure 7It can be seen that the proposed method can make the state error of the multi-agent system converge to 0, that is, the proposed method can make each agent reach its desired formation position without collision.

[0177] It should be noted that the present invention is not limited to the contents shown in the above exemplary examples, and the present invention can be implemented in other forms without violating the basic features of the present invention. Therefore, the example should be regarded as an exemplary example rather than a restrictive one. The scope of the present invention is determined by the appended claims rather than the above description, and it is intended to include all changes within the meaning and scope of the equivalent elements of the claims. Without violating the principles of the present invention, several modifications and improvements made to the present invention should be regarded as within the scope of protection of the present invention.

Claims

1. An approximate robust Nash equilibrium method for distributed formation control, characterized in that: The following steps are involved: Step S1: construct a linear model of intelligent agents subject to external interference constraints, design the information transmission relationship of intelligent agents through a network topology diagram, and adopt a leader-follower strategy; Step S2: In view of the formation adjustment caused by the task change, the formation problem is transformed into a distributed robust differential game, and a rolling zero-sum game model is constructed based on the shrinking time domain theory; Step S3: Based on the Pontryagin minimum principle, an optimal control strategy expression is provided that depends on the terminal position information of itself and neighboring agents; Step S4: construct a state observer for the participants, estimate the terminal states of the game participants, transmit and receive state estimates through the communication topology graph, solve the optimal strategy based on the above state estimates, and achieve convergence of the approximate Nash equilibrium solution; Step S5: Analyze the relationship between the approximate Nash equilibrium solution and the true Nash equilibrium solution.

2. The approximate robust Nash equilibrium method for distributed formation control according to claim 1, characterized in that: In step S1, the leader grasps the target position, and the follower maintains a specific distance from the neighboring intelligent agent, and further includes: Step S11: Establishing a system model: The dynamic equation for each agent is: Where: t is the time scale; is the rate of change of the position information of the ith agent at time t; x i (t) is the position information of the i-th agent at time t; u i (t) and ω(t) are the control strategy and external disturbance of the ith agent at time t, respectively; B ii and E ii are the weight matrices corresponding to the control strategy of the ith agent and the external interference; The dynamic equation of the multi-agent system is: In the formula, is the rate of change of the state information of the multi-agent system at time t; B i and E i are the i-th control strategy weight matrix and external interference weight matrix in the multi-agent system respectively. Step S12: Distributed architecture establishment: There are N agents and N-1 followers, and a communication topology graph G(v,ε) is constructed; Where: v = {1,...,N} represents the set of agents, N represents the total number of agents, A set of edges representing connections between agents; The neighbor information matrix of agent i is expressed as: In the formula, is the neighbor matrix of agent i at time t; is an N×1 vector with the jth row being 1 and the rest of the rows being 0; for The transpose of ; I3 is the identity matrix; When the formation task changes, the neighbor information matrix of agent i is time-varying, and the neighbor set of agent i is also time-varying. If the formation is at t s The following function is introduced to avoid disconnection of the formation: In the formula, represents the sigmoid function of agent j at time t, e represents the exponential function, α represents a constant, and each of the agents is constrained by dynamic equations, network topology and external interference.

3. The approximate robust Nash equilibrium method for distributed formation control according to claim 2, characterized in that: The step S2 comprises: The intelligent agents participating in the formation task are defined as game participants. Based on the idea of ​​shrinking time domain control, the game is replayed in each given rolling window, environmental information is fed back in real time, and a benefit function corresponding to the game participants is constructed. Participant i in the time domain window t∈[t k ,t f The benefit function of ] is: Where, t k is the initial time corresponding to the sampling instant k; u ik (t) and u -ik (t) are the control strategies of the participant at time t corresponding to the sampling instant k and the set of control strategies of the neighboring participants except participant i; ω ik (t) and ω -ik (t) are the interference of the participant at time t corresponding to the sampling instant k and the interference set of the neighboring participants except participant i; x(t k ) is the initial time t corresponding to the sampling instant k k The initial position information of the multi-agent system at time x(t f ) is the terminal time t f The corresponding terminal position information of the multi-agent system; is the terminal time t corresponding to the sampling instant k f The adjustable weight matrix of the ith agent; is the neighbor information matrix of the i-th agent at time t corresponding to the sampling instant k; for The transposed matrix of and are the positive definite weight matrices of the i-th agent corresponding to the sampling instant k; Participant i and his neighbor participants converge to a local robust Nash equilibrium, that is, the strategy set at time t corresponding to sampling instant k satisfy: In the formula, and are the optimal strategy of participant i at time t corresponding to sampling instant k and the optimal strategy set of neighbor participants; and are the interference of participant i and the interference set of neighbor participants at time t corresponding to sampling instant k; is the optimal cost of participant i relative to his neighbor participants at time t corresponding to sampling instant k; is the suboptimal cost of participant i at time t corresponding to sampling instant k relative to the neighboring participants when the interference strategy is the largest; is the suboptimal cost of participant i relative to his neighbor participants when the control strategy of participant i at time t corresponding to sampling instant k is optimal.

4. The approximate robust Nash equilibrium method for distributed formation control according to claim 3, characterized in that: The step S3 comprises: The open-loop local robust Nash equilibrium solution of each participant at time t corresponding to sampling instant k satisfies: The optimal control strategy form: Worst forms of interference: In the formula, I 3N is the 3N-dimensional identity matrix; Transformed into the following form: 。 5. The approximate robust Nash equilibrium method for distributed formation control according to claim 4, characterized in that: The step S4 comprises: Design the terminal state observer: In the formula, is the estimated change rate of the terminal time position of participant i at time t corresponding to sampling instant k; is the estimated terminal location information of participant i at time t corresponding to sampling instant k; s i is the gain coefficient of participant i; The estimated value of the terminal state eventually converges to the true value: Right now: In the formula, is the terminal time position estimate of the multi-agent system at time t corresponding to the sampling instant k; f (t k ) is the t corresponding to the sampling instant k k The multi-agent system estimates the terminal time position at the moment x k (t f ) is the position of the multi-agent system at the terminal moment corresponding to the sampling instant k; Among them, the estimated optimal control strategy and the worst interference satisfy the following forms: And the difference between the above formula and the real optimal strategy and the worst interference strategy is exponentially convergent, that is: wherein, is the difference between the estimated optimal strategy and the actual optimal strategy of participant i at time t corresponding to sampling instant k; is the difference between the estimated worst interference and the actual worst interference of participant i at time t corresponding to sampling instant k; Δx f (t k ) is the deviation between the estimated initial position and the true initial position of the multi-agent system at time t corresponding to sampling instant k; γ = min k Re[λ m (SΛ m )] is the minimum value of the real part among the eigenvalues of the product of matrix S and matrix Λ k ; matrix k denotes the diagonal matrix composed of the gain coefficients of N participants; s1 is the gain coefficient of participant 1, s is the gain coefficient of participant N, and the convergence of the approximate Nash equilibrium solution is completed.​​ 6. The approximate robust Nash equilibrium method for distributed formation control according to claim 1, characterized in that: The step S5 comprises: The control strategy set forms a local robust open-loop approximate Nash equilibrium solution: Where ζ is a function related to the initial state of the system; The estimated optimal strategy of participant i and the estimated worst interference form a local robust open-loop approximate Nash equilibrium solution with the neighbors. The difference between the approximate Nash equilibrium solution and the true Nash equilibrium solution at sampling instant k is: In the formula, ζ ik is the difference between the approximate Nash equilibrium solution of agent i and the true Nash equilibrium solution corresponding to sampling instant k; N represents agent N; where ζ ik Satisfy the following formula: In the above formula as well as These are the constructed correlation coefficients: In a multi-agent system with N agents, agent i updates its terminal estimated state according to the terminal state observer. The agent adopts the estimated optimal strategy and the estimated worst interference to achieve distributed formation.

Images