Accelerated distributed Nash equilibrium search method based on event trigger control

Abstract

The invention discloses an event trigger control-based accelerated distributed Nash equilibrium search method, which comprises the following steps of: S1, adopting an undirected graph to construct communication among nodes in a multi-agent system to realize information intercommunication; s2, constructing a gradient descent model; s3, determining whether a triggering condition of communication is met or not through a condition detector, and optimizing a communication network in the gradient descent model; s4, meeting a convergence requirement through a Lyapunov consensus, and calculating an upper bound of fixed time convergence; and S5, iteratively updating the equation through a distributed neurodynamics method to realize fixed-time Nash equilibrium for controlling the distributed multi-agent system. According to the method, the unnecessary communication times can be effectively reduced, so that the network resource use is optimized, and a strategy of participant control input is formulated for a specific engineering application scene so as to cope with factors which are not considered by the model.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed control and event-triggered control, and in particular to an accelerated distributed Nash equilibrium search method based on event-triggered control. Background Art

[0002] Nash equilibrium represents the optimal decision-making behavior of participants in the framework of game theory. When Nash equilibrium is reached, each participant has made the best response based on the other party's expected actions, and cannot further optimize its objective function or reduce the cost function by adjusting its own behavior strategy alone. Nash equilibrium not only reflects a dynamic equilibrium state in the game, but also a combination of strategies jointly reached by all players in pursuit of maximizing their own interests. Distributed algorithms do not rely on centralized information processing, and can converge to Nash equilibrium points only through mutual communication between nodes, which significantly improves the security and communication efficiency of the system, reduces energy loss, and effectively protects the information security of each node and the privacy of the objective function.

[0003] In recent years, as the complexity of engineering projects has gradually increased, a single intelligent system has begun to be unable to meet the needs. Inspired by the cluster phenomenon in nature, combined with complex network analysis methods and distributed thinking, scholars have designed a large number of distributed algorithms to solve the Nash equilibrium problem of multi-agent games. For example, zero-sum game algorithms under undirected and weighted equilibrium graphs, zero-sum game Nash equilibrium search algorithms under non-smooth cost functions, and Nash equilibrium search algorithms considering communication delays and zero-sum game Nash equilibrium search algorithms with a common constraint set under two time-varying multi-agent subnetworks. In addition, generalized Nash equilibrium algorithms have also been widely studied to deal with constraints in practical applications. Distributed Nash equilibrium search strategies have been widely studied and applied to game analysis in environments with limited decision information, including a variety of discrete time and continuous time strategies, due to their high communication efficiency, flexibility and reliability.

[0004] Among many distributed computing technologies, consensus algorithms are particularly important. This type of method ensures that each agent with execution capabilities can reach a common state. Especially in optimization problems, there is a close connection between consensus algorithms and optimization problems. In the paper Distributed control of networked uncertain Euler-Lagrange systems in the presence of stochastic disturbances: a prescribed performance approach published in Nonlinear Dynamics, a distributed nonlinear non-affine consensus algorithm is proposed for multi-agent systems. In the paper Finite-time consensus problems for networks of dynamic agents published in IEEE Transactions on Automatic Control, an innovative cooperative adaptive neural network control strategy is developed based on the minimum learning parameter technique and established performance criteria. This control strategy ensures that all closed-loop network signals are locally consistent in the final bounded state, and at any time, the tracking error is limited to the preset limit. In order to further improve the convergence speed and overcome the problem of inaccurate estimation of the upper bound of the convergence time by the initial state in the finite-time control method, a fixed-time control method is proposed. This type of method can ensure that the system can reach a consistent state within a predefined fixed time regardless of its initial state.

[0005] Finite-timeconsensusandfinite-timeH∞ on IEEETransactionsonNetworkScienceandEngineering

[0006] The consensus of multi-agent systems under directed topology was first proposed in the study of the stability of homogeneous vector fields. Under certain conditions, such as the presence of directed communication topology and input restrictions, the research has been further extended to the finite-time consensus problem and the finite-time consensus of dual-integrator multi-agent systems under input saturation.

[0007] Although there are a lot of relevant research results, there are still limited studies on cluster game problems involving nonlinear systems or hybrid heterogeneous systems consisting of linear and nonlinear systems, as well as Nash equilibrium search algorithms considering changes in communication topology and communication loss. At the same time, such strategies need to rely on centralized control gains, which in turn rely on the knowledge of the game scale, the monotonicity of the participants' gradients, and the Lipschitz condition. However, obtaining this global information in a distributed environment is often costly or even impossible, making it difficult to implement strategies that rely on centralized gains. In some cases, traditional algorithms have been unable to meet the needs of modern applications due to problems such as insufficient robustness, low efficiency, and limited practicality. Summary of the invention

[0008] The object of the present invention is to provide an accelerated distributed Nash equilibrium search method based on event-triggered control, which can ensure that the player's estimate converges to a small neighborhood around the actual action within a fixed time, and this time can be predefined only by design parameters.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] An accelerated distributed Nash equilibrium search method based on event-triggered control comprises the following steps:

[0011] S1: Use undirected graphs to build communication between nodes in a multi-agent system to achieve information exchange;

[0012] S2: Use the gradient descent method to adjust the step size that approaches the optimal solution and build a gradient descent model;

[0013] S3: Determine whether the gradient descent model meets the triggering condition for communication through a condition detector, and optimize the communication network in the gradient descent model;

[0014] S4: Satisfy the convergence requirements through Lyapunov consensus and calculate the upper bound of fixed-time convergence;

[0015] S5: Iteratively update equations through distributed neural dynamics methods to achieve fixed-time Nash equilibrium, which is used to control distributed multi-agent systems, reduce unnecessary communications and optimize network resources.

[0016] Preferably, in S1, the communication method between nodes is constructed by using an undirected graph: since adjacent intelligent agents can communicate information through an undirected connected graph, if there is at least one path between any two nodes, the undirected graph is connected, and if there is a connected edge between the nodes, intelligent agents i and j can communicate with each other, and intelligent agents can obtain strategy information of adjacent intelligent agents through the undirected graph;

[0017] Preferably, in S2, the gradient descent model is:

[0018]

[0019]

[0020] The cost function of each agent is defined as:

[0021]

[0022] Cost function f i The gradient of (x) satisfies and This results in the following defined parameters:

[0023]

[0024] Where, the agent set N = {1, 2, ..., N}; x is the action set of all agents; is the action strategy of the ith agent, Used to estimate the entire action set x, y ij is the approximate estimate of agent i’s action on agent j, is an N-dimensional real Euclidean space; H i The transpose of γ=[γ 11 ,γ 12 ,…,γ 1N ,γ 21 ,…,γ N1 ,…,γ NN ] T and

[0025]

[0026] B is a diagonal matrix with the initial connected graph as the diagonal. is the Laplace matrix, 1 N is an N-order unit matrix; is the time when the triggering event occurs; i is the set parameter, η>>1, s>1 is the ratio of two non-negative odd numbers. By designing the parameters β, a1, a2, a3, the step size of the gradient descent can be adjusted to achieve the best optimization effect.

[0027] Preferably, in S3, the calculation expression of the trigger condition is:

[0028]

[0029] When the above trigger conditions are met, the event is triggered. The function is:

[0030] The restrictions are ω>0 and κ∈(0,1);

[0031] in,

[0032]

[0033] In the formula, χ ij (t) is an internal dynamic variable, and its derivative satisfies:

[0034]

[0035] a1, a2, a3, a4, a5, a6 are constants, a1, a2, a3, a4, a5, a6>0; δ∈(0,1), ω, κ are design parameters; E ij (t) is the measurement error.

[0036] Preferably, in S4, for all agents, if the following conditions are met:

[0037] and a2(1-κ)>β,

[0038] In a fixed time, the estimates of all agents can converge to a neighborhood centered on the true action;

[0039] The fixed time convergence upper bound is:

[0040]

[0041] in,

[0042]

[0043] ρ∈(0,1), And ζ∈(0,1);λ min ,λ max is the minimum and maximum value of the variable λ with parameters defined,

[0044] Preferably, in S5, the distributed neural dynamics method comprises the following steps:

[0045] Input and initialize parameters: including communication topology G, cost function f i (x), initial value x i , initial prediction value And parameter a i ,ω,β,η,s,δ;

[0046] The initial time Assign a value of 0 and set x i , the initial value of the participant, y i , the participant’s initial estimate, assigned to ψ ij

[0047] when hour,

[0048] calculate

[0049] The cost function f i (x) Calculate H(x) and calculate the following expression;

[0050]

[0051] Return x k , and finally output x * .

[0052] A computer storage medium is used to store program data. When the program data is executed by a processor, it is used to implement the accelerated distributed Nash equilibrium search method based on event-triggered control.

[0053] The present application also provides a computer program product, wherein the computer program product includes a computer program, and the computer program can be operated to enable a computer to execute the accelerated distributed Nash equilibrium search method based on event-triggered control. The computer program product can be a software installation package.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention can ensure that the player's estimate converges to a small neighborhood around the actual action within a fixed time, and this time can be predefined only by design parameters. The present invention uses an additional internal dynamic variable to further reduce energy consumption, and is activated only when there is a need to update the estimated dynamics. By introducing dynamic variables, the number of triggers is reduced, and the number of unnecessary communications is effectively reduced, thereby optimizing the use of network resources, and for specific engineering application scenarios, a strategy for participant control input is formulated to deal with factors not considered by the model. Under the same conditions, the present invention can save the size of the memory for data storage, while reducing the number of communications, and can reduce the heating problem of the aircraft's energy consumption, busy calculations and other problems. In terms of energy allocation issues, dynamic allocation of resources can be quickly achieved, effectively solving high-precision control problems in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of the process of the present invention;

[0057] Figure 2 A schematic diagram of a communication network topology in an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of the evolution of player actions according to an embodiment of the present invention;

[0059] Figure 4 A schematic diagram of the error between the player's behavior and the Nash equilibrium in an embodiment of the present invention;

[0060] Figure 5 A schematic diagram of a player's estimation of other participants according to an embodiment of the present invention;

[0061] Figure 6 A schematic diagram of triggering communication timing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0063] Embodiment 1:

[0064] See also Figure 1-Figure 6 , an accelerated distributed Nash equilibrium search method based on event-triggered control, comprising the following steps:

[0065] Step 1: Use undirected graphs to build communication between nodes in a multi-agent system to achieve information exchange.

[0066] In a non-cooperative game involving N players, the set of participants is recorded as N = {1, 2, ..., N}. Each participant i corresponds to a cost function The set x consisting of an ordered array of all N real numbers is defined as the set of action configurations. is an N-dimensional Euclidean space, N is a positive integer, and the set x=[x1,x2,…,x N ] T , represents the action strategy of the i-th participant. is an estimate of the entire action set x, where y ijis the approximate estimate of participant i on the action of participant j. Adjacent agents can communicate information through an undirected connected graph. If there is at least one path between any two nodes, the undirected graph is connected. If there is an edge between the nodes, agent i and agent j can communicate with each other. Agents can obtain the strategy information of adjacent agents through the undirected graph.

[0067] Step 2: Use the gradient descent method to adjust the appropriate step size and establish a gradient descent model.

[0068] Create the following dynamic system:

[0069]

[0070] in, is an N-dimensional Euclidean space; g(x,t) is a nonlinear function that changes with time t and satisfies g(0 N ,t)=0; t is time.

[0071] The momentum term is introduced into the iterative formula by adding the previous information as the momentum coefficient.

[0072] Consider the following strategy for finding a Nash equilibrium, whereby participants update their strategies:

[0073]

[0074] Among them, the agent set N = {1, 2, ..., N}; x is the action set of all agents; is the action strategy of the ith agent, To estimate the entire action set x, where y ij is the approximate estimate of agent i’s action for agent j, is an N-dimensional real Euclidean space.

[0075] The cost function of each agent is defined as:

[0076]

[0077] Define the cost function f i The gradient of (x) satisfies:

[0078] and

[0079] This gives us the defined parameters:

[0080]

[0081] Hi The transpose of

[0082] γ=[γ 11 ,γ 12 ,…,γ 1N ,γ 21 ,…,γ N1 ,…,γ NN ] T and

[0083]

[0084] for B is a diagonal matrix with the initial connected graph as the diagonal. is the Laplace matrix, 1 N is an N-order unit matrix; is the time when the triggering event occurs; i is a set parameter, η>>1, s>1 is the ratio of two non-negative odd numbers. By designing parameters β, a1, a2, a3, the step size of gradient descent can be adjusted to achieve the best optimization effect.

[0085] Step 3: Introduce a condition detector to determine whether the triggering conditions for communication are met and optimize the communication network in the model.

[0086] The calculation expression of the trigger condition is:

[0087]

[0088] Among them, the event triggering function that meets the above conditions is:

[0089] The restrictions are ω>0 and κ∈(0,1);

[0090] Measurement error E ij (t) is defined as:

[0091]

[0092] In the formula, χ ij (t) is an internal dynamic variable, and its derivative satisfies

[0093] are constants, a1, a2, a3, a4, a5, a6>0; δ∈(0,1), ω, κ are design parameters.

[0094] This embodiment adopts a consensus-based estimation method to correct the strategies of each participant. In formula (4), It accelerates the process of getting the estimated value close to the true value. It plays an accelerating role when there is a significant difference between the estimated value and the true value. It plays an accelerating role in achieving exponentially fast convergence.

[0095] Step 4: Satisfy the convergence requirements through Lyapunov consensus and calculate the upper bound of fixed-time convergence.

[0096] Construct the Lyapunov function V1(t). For the convenience of expression, the function is divided into two parts Y1(t) and Y2(t) for proof. V1(t) = Y1(t) + Y2(t), and the definitions of Y1(t) and Y2(t) are as follows;

[0097] The function is defined as:

[0098]

[0099] λ min is the minimum value of a variable λ with parameters defined such that:

[0100]

[0101] Based on -gamma ij E ij (t)≤|γ ij ||E ij | and for any number Then there is an inequality holds, where β>>1 and

[0102] Depending on the identity, for any number F1 and F2, And for any real number sequence γ1,γ2,…,γ N ,inequality Established, of which

[0103] Similarly, for Also true.

[0104] Derivative of V1(t) with respect to time t:

[0105]

[0106] in,

[0107]

[0108] The upper bound of the convergence time can be obtained by calculation satisfy:

[0109]

[0110] Where ρ∈(0,1), there exists

[0111]

[0112] Under the above conditions, the ratio of the left limit of the measurement error corresponding to the next trigger to the upper limit of its rate of change is less than the interval between the execution times, that is:

[0113]

[0114] The numerator is strictly non-negative and the denominator is upper bounded, and Zeno's phenomenon does not exist.

[0115] set up The time derivative of V2 is obtained by calculation:

[0116]

[0117] Design the communication topology node set as Node Set Divided into two subsets, the subsets are defined to satisfy

[0118] Definition i =y i -x, exists ∥ e i (t)∥≤τ, for t≥T 1max ,have Because for Both in Therefore, for have τ is a definition parameter.

[0119] set up If for For example, existence For The following settings are available: i =2. If exist For The following settings are available: i = -2. If none of the above conditions are met, set d i = 0. Therefore, for exist:

[0120]

[0121] when For all agents i,j, if

[0122] and a2(1-k)>β,

[0123] In a fixed time, estimate y ij can converge to a point with real action x j In the neighborhood of centered. where ζ∈(0,1) and

[0124] because for have to

[0125]

[0126] x * is the state of the participants after reaching equilibrium; so we can get Converge to a neighborhood centered on the true action.

[0127] Step 5: Iteratively update the equation according to the proposed distributed Nash equilibrium search method (Formula 2a and Formula 2b) to achieve fixed-time Nash equilibrium in a fixed time. The pseudo code of the search method is as follows:

[0128]

[0129] The present invention can ensure that the player's estimate converges to a small neighborhood around the actual action within a fixed time, and this time can be predefined only by design parameters. The present invention uses an additional internal dynamic variable to further reduce energy consumption, and is activated only when there is a need to update the estimated dynamics. By introducing dynamic variables, the number of triggers can be reduced, and the number of unnecessary communications can be effectively reduced, thereby optimizing the use of network resources. Under the same conditions, the present invention can save the size of the memory for data storage, while reducing the number of communications, and can reduce the heating problem of the aircraft's energy consumption, busy calculations and other problems. In terms of energy allocation, dynamic allocation of resources can be quickly achieved, effectively solving the problem of high-precision control in complex environments.

[0130] The present application also provides a computer storage medium, which is used to store program data. When the program data is executed by a processor, it is used to implement the accelerated distributed Nash equilibrium search method based on event-triggered control as described in the above embodiment.

[0131] The present application also provides a computer program product, wherein the computer program product includes a computer program, and the computer program is operable to enable a computer to execute the accelerated distributed Nash equilibrium search method based on event-triggered control as described in the embodiment of the present application. The computer program product can be a software installation package.

[0132] The accelerated distributed Nash equilibrium search method based on event-triggered control described in the above embodiment of the present application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0133] In a specific embodiment, the energy management problem in an intelligent transportation system is simulated, such as Figure 2 As shown, consider four agents and follow the Figure 2 The cost function of agent i is expressed as follows:

[0134]

[0135] where q i ,w>0 represents a constant related to the user, and w, is a constant. λ is calculated min (H -1 )=0.4, and λ max (H -1 )=0.4762, λ min (H) = 2.1. Let the relevant design coefficient in the cost function be w1=0.1,w=10,q i =1, The only Nash equilibrium point obtained by calculation is x * =[6.86,11.62,16.38,21.14] T .

[0136] Set the initial state to x(0) = [22, 22, 22, 22] T , y1(0)=-[10,10,10,10] T , y2(0)=-[5,5,5,5] T, y3(0)=[0,0,0,0] T , y4(0)=[5,5,5,5] T . Set the relevant design parameters a1 = 10, a2 = 11, a3 = 10, a4 = 2, a5 = 0.6, a6 = 0.1, ω = 2, k = 0.1, δ = 0.5, β=9,η=10e7,χ ij (0) = 20 For all agents i and j, this holds true. Setting the value of κ to 0.99, it can be calculated that the fixed time from the estimated value to the actual action is less than 1.76 seconds.

[0137] like Figure 3 Shown is a schematic diagram of changes in the behavior of an intelligent agent using the present invention; Figure 4 The error indicating that the actual behavior of the participants deviates from the Nash equilibrium is obvious. The players converge to the Nash equilibrium point within 1.8 seconds. Figure 4 This indicates that the participants’ behavior quickly approaches the Nash equilibrium point within 1.8 seconds. Figure 5 It shows that participants' predictions of behavior were well below the theoretical maximum time limit for prediction, with these predictions closely aligned with actual behavior to within 0.25 seconds. Figure 6 shows that before the estimated and actual behavior converge, participants The specific moment when the estimation is triggered. At the beginning, the error |E ij (t)| will rise rapidly because there are variables with powers greater than 1. Due to the effect of the hyperbolic tangent function, as the estimated value approaches the actual action of the player, |E ij (t)| increases rapidly, triggering the activation of the trigger mechanism. In theory, once consensus is reached, there will be no further triggers. However, in actual projects, since the system is discrete, γ ij (t) oscillates around the origin, so it still triggers. This reveals that the player has a fixed convergence time of 0.25 seconds at the appropriate trigger moment.

Claims

1. An accelerated distributed Nash equilibrium search method based on event-triggered control, characterized in that: The following steps are involved: S1: Use undirected graphs to build communication between nodes in a multi-agent system to achieve information exchange; S2: Build a gradient descent model; S3: Determine whether the triggering condition for communication is met through the condition detector, and optimize the communication network in the gradient descent model; S4: Satisfy the convergence requirements through Lyapunov consensus and calculate the upper bound of fixed-time convergence; S5: Iteratively update equations through distributed neural dynamics methods to achieve fixed-time Nash equilibrium for controlling distributed multi-agent systems.

2. The method for accelerating distributed Nash equilibrium search based on event-triggered control according to claim 1, characterized in that: In S1, an undirected graph is used to construct a communication method between nodes. If there is at least one path between any two nodes, the undirected graph is connected. If there are connected edges between nodes, agents i and j can communicate with each other. Agents can obtain strategy information of adjacent agents through the undirected graph.

3. The method for accelerating distributed Nash equilibrium search based on event-triggered control according to claim 1, characterized in that: In S2, the gradient descent model is: The cost function of each agent is defined as: Cost function f i The gradient of (x) satisfies: and This results in the following defined parameters: Where, the agent set N = {1, 2, ..., N}; x is the action set of all agents; is the action strategy of the ith agent, Used to estimate the entire action set x, y ij is the approximate estimate of agent i’s action on agent j, is an N-dimensional real Euclidean space; H i The transpose of γ=[γ 11 ,γ 12 ,…,γ 1N ,γ 21 ,…,γ N1 ,…,γ NN ] T and B is a diagonal matrix with the initial connected graph as the diagonal. is the Laplace matrix, 1 N is an N-order unit matrix; is the moment when the trigger event occurs; η>>1, s>1 is the ratio of two non-negative odd numbers, and the step size of gradient descent is adjusted by parameters β, a1, a2, a3.

4. The method for accelerating distributed Nash equilibrium search based on event-triggered control according to claim 1, characterized in that: In S3, the calculation expression of the trigger condition is: When the above trigger conditions are met, the event is triggered. The function is: The restrictions are ω>0 and κ∈(0,1); in, In the formula, χ ij (t) is an internal dynamic variable, and its derivative satisfies: ， a1, a2, a3, a4, a5, a6 are constants, a1, a2, a3, a4, a5, a6>0; δ∈(0,1), ω, κ are design parameters; E ij (t) is the measurement error.

5. The method for accelerating distributed Nash equilibrium search based on event-triggered control according to claim 1, characterized in that: In S4, for all agents, if the following conditions are met: and a2(1-κ)>β, In a fixed time, the estimates of all agents can converge to a neighborhood centered on the true action; The fixed time convergence upper bound is: T 1max +T2(x(T 1max )), in, ρ∈(0,1), and λ min ,λ max are the minimum and maximum values ​​of the variable λ with parameters defined respectively; 6. The method for accelerating distributed Nash equilibrium search based on event-triggered control according to claim 1, characterized in that: In S3, the distributed neural dynamics method comprises the following steps: Input and initialize parameters: including communication topology G, cost function f i (x), initial value x i , initial prediction value And parameter a i ,ω,β,η,s,δ; when hour, calculate The cost function f i Assign (x) to H(x) and calculate the following expression; Return x k , and finally output x * .

Images