An Accelerated Distributed Nash Equilibrium Search Method Based on Event-Triggered Control

Abstract

The present invention discloses an accelerated distributed Nash equilibrium search method based on event-triggered control, comprising the following steps: S1: In a multi-agent system, an undirected graph is adopted to construct communication between nodes to achieve information intercommunication; S2: A gradient descent model is constructed; S3: A condition detector is used to determine whether the triggering condition for communication is reached, and the communication network in the gradient descent model is optimized; S4: The Lyapunov consensus is used to meet the convergence requirements, and the upper bound of fixed-time convergence is calculated; S5: The equation is iteratively updated by a distributed neurodynamics method to achieve a fixed-time Nash equilibrium for controlling a distributed multi-agent system. The present invention can effectively reduce unnecessary communication times, thereby optimizing the use of network resources, and for specific engineering application scenarios, a strategy for controlling the input of participants is formulated to cope with factors not considered in the model.

Description

Technical Field

[0001] The present invention relates to the technical fields of distributed control and event-triggered control, and particularly relates to an accelerated distributed Nash equilibrium search method based on event-triggered control. Background Art

[0002] The Nash equilibrium point represents the optimal decision-making behavior of participants within the framework of game theory. When reaching the Nash equilibrium, each participant has made the best response based on the expected actions of the other party, and cannot further optimize its objective function or reduce the cost function by individually adjusting its behavioral strategy. The Nash equilibrium not only reflects a dynamic equilibrium state in the game, but also is a strategy combination jointly achieved by all players when pursuing the maximization of their own interests. The distributed algorithm does not rely on centralized information processing and can converge to the Nash equilibrium point only through the mutual communication between nodes, significantly improving the security and communication efficiency of the system, reducing energy consumption, and effectively protecting the information security of each node and the privacy of the objective function.

[0003] In recent years, as the engineering complexity has gradually increased, a single intelligent system has begun to be difficult to meet the requirements. Inspired by the swarm phenomenon in nature and combined with complex network analysis methods and distributed ideas, scholars have designed a large number of distributed algorithms to solve the multi-agent game Nash equilibrium problem. For example, the zero-sum game algorithm under undirected and weighted balanced graphs, the zero-sum game Nash equilibrium search algorithm under non-smooth cost functions, and the Nash equilibrium search algorithm considering communication delays and the zero-sum game Nash equilibrium search algorithm with a common constraint set under two time-varying multi-agent subnetworks. In addition, the generalized Nash equilibrium algorithm has also been widely studied to handle the constraint conditions in practical applications. The distributed Nash equilibrium search strategy has been widely studied and applied to game analysis in environments with limited partial decision-making information, including various discrete-time and continuous-time strategies due to its high communication efficiency, flexibility, and strong reliability.

[0004] Among many distributed computing technologies, consensus algorithms are particularly important. Such methods ensure that each intelligent agent with execution ability can reach a common state. Especially in optimization problems, there is a close connection between consensus algorithms and optimization problems. Distributed control of networked uncertain Euler-Lagrange systems in the presence of stochastic disturbances: a prescribed performance approach published in Nonlinear Dynamics proposed a distributed nonlinear non-affine consensus algorithm for multi-agent systems. Finite-time consensus problems for networks of dynamic agents published in IEEE Transactions on Automatic Control developed an innovative cooperative adaptive neural network control strategy according to the least learning parameter technique and established performance criteria. This control strategy ensures that all closed-loop network signals cooperate locally and uniformly to remain in a ultimately bounded state, and at any time, the tracking error is restricted within a preset bound. To further improve the convergence speed and overcome the problem of inaccurate estimation of the upper bound of the convergence time for the initial state in the finite-time control method, the fixed-time control method was proposed. Such methods can ensure that the system reaches a consensus state within a predefined fixed time regardless of the initial state, which was first proposed in Finite-time consensus and finite-time H∞

[0005] consensus of multi-agent systems under directed topology published in IEEE Transactions on Network Science and Engineering when studying the stability of homogeneous vector fields. Under specific conditions, such as the existence of directed communication topology and input constraints, the research was further extended to the finite-time consensus problem and the finite-time consensus for multi-agent systems with double integrators under input saturation conditions.

[0006]

[0007] ​Although there have been a large number of relevant research results, the research on the cluster game problem of nonlinear systems or hybrid heterogeneous systems composed of linear and nonlinear systems, as well as the Nash equilibrium search algorithm considering communication topology changes and communication losses, is still limited. At the same time, such strategies need to rely on centralized control gains, which in turn depend on the understanding of the game scale, the monotonicity of the player gradients, and the Lipschitz conditions. However, obtaining these global information in a distributed environment is often costly or even impossible, resulting in difficulties in implementing strategies that rely on centralized gains. In some cases, traditional algorithms have been difficult to meet the needs of modern applications due to problems such as insufficient robustness, low efficiency, and limited practicality. Summary of the Invention

[0008] The object of the present invention is to provide an accelerated distributed Nash equilibrium search method based on event-triggered control, which can ensure that the estimates of the players converge to a small neighborhood around the actual actions within a fixed time, and this time can be predefined only through design parameters.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] An accelerated distributed Nash equilibrium search method based on event-triggered control includes the following steps:

[0011] S1: In a multi-agent system, an undirected graph is used to construct the communication between nodes to achieve information exchange;

[0012] S2: Using the gradient descent method, adjust the step size towards the optimal solution to construct a gradient descent model;

[0013] S3: Determine whether the gradient descent model reaches the communication trigger condition through a condition detector, and optimize the communication network in the gradient descent model;

[0014] S4: Calculate the upper bound of fixed-time convergence through Lyapunov consensus to meet the convergence requirements;

[0015] S5: Iteratively update the equation through a distributed neurodynamics method to achieve a fixed-time Nash equilibrium, which is used to control a distributed multi-agent system to reduce unnecessary communication times and thus optimize network resources.

[0016] Preferably, in S1, the method of using an undirected graph to construct the communication between nodes is as follows: Since adjacent agents can communicate information through an undirected connected graph, if there is at least one path between any two nodes, the undirected graph is connected. If there is a connected edge between nodes, agents i and j can communicate with each other, and agents can obtain the strategy information of adjacent agents through the undirected graph;

[0017] Preferably, in S2, the gradient descent model is as follows:

[0018]

[0019]

[0020] The cost function of each agent is defined as:

[0021]

[0022] The cost function f i (x) has a gradient that satisfies and From this, the defined parameters are obtained:

[0023]

[0024] In the formula, the agent set N = {1, 2,..., N}; x is the action set of all agents; is the action strategy of the i-th agent, used to estimate the entire action set x, y ij is the approximate estimate of the action of agent i on agent j, is the N-dimensional real Euclidean space; is the transpose of H i , γ = [γ 11 , γ 12 , …, γ 1N , γ 21 , …, γ N1 , …, γ NN T and

[0025]

[0026] B is a diagonal matrix with the initial connected graph as the diagonal, is the Laplacian matrix, 1 N is the N-order identity matrix; is the time when the triggering event occurs; ξ i is the set parameter, η >> 1, s > 1 is the ratio of two non-negative odd numbers, and through the design of parameters β, a 1 , a 2 , a 3 the step size of the gradient descent can be adjusted to achieve the best optimization effect.

[0027] Preferably, in S3, the calculation expression of the triggering condition is:

[0028] ​

[0029] When the above trigger conditions are met, an event trigger is performed, and the function is:

[0030] The limiting conditions are ω > 0 and κ ∈ (0, 1);

[0031] Among them,

[0032]

[0033] In the formula, χ ij (t) is an internal dynamic variable, and its derivative satisfies:

[0034]

[0035] a 1 , a 2 , a 3 , a 4 , a 5 , a 6 are constants, a 1 , a 2 , a 3 , a 4 , a 5 , a 6 > 0; δ ∈ (0, 1), ω, κ are design parameters; E ij (t) is the measurement error.

[0036] Preferably, in S4, for all agents, if the following conditions are met:

[0037] And a 2 (1 - κ) > β,

[0038] Within a fixed time, the estimates of all agents can converge to a neighborhood centered on the true action;

[0039] The upper bound of fixed-time convergence is:

[0040]

[0041] Among them,

[0042]

[0043] ρ ∈ (0, 1), And ζ ∈ (0, 1); λ min , λ max are the minimum and maximum values of the defined variable λ containing parameters,

[0044] Preferably, in S5, the distributed neural dynamics method includes the following steps:

[0045] Input and initialize parameters: including communication topology G, cost function f i (x), initial value x i , initial prediction value and parameter a i , ω, β, η, s, δ;

[0046] Assign the initial time as 0, and assign x i , the initial value of the participant, y i , the initial estimate of the participant, to ψ ij

[0047] When it is

[0048] Calculate

[0049] Calculate H(x) from the cost function f i (x), and calculate the following expressions;

[0050]

[0051] Return x k , and finally output x * .

[0052] A computer storage medium is used to store program data. When the program data is executed by a processor, it is used to implement the above-mentioned accelerated distributed Nash equilibrium search method based on event-triggered control.

[0053] The present application also provides a computer program product. Among them, the above computer program product includes a computer program, and the above computer program can operate to make a computer execute the above-mentioned accelerated distributed Nash equilibrium search method based on event-triggered control. This computer program product can be a software installation package.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention can ensure that the player's estimation converges to a small neighborhood around the actual action within a fixed time, which can be predefined only through design parameters. The present invention uses an additional internal dynamic variable to further reduce energy consumption, which is activated only when there is a necessity to update the estimation dynamics. By introducing the dynamic variable, the number of trigger times is reduced, effectively reducing the number of unnecessary communications, thereby optimizing the use of network resources. For specific engineering application scenarios, a strategy for controlling the input of participants is formulated to address factors not considered in the model. Under the same conditions, the present invention can save the memory size for data storage, reduce the number of communications, and reduce problems such as heat generation and heavy computing load of the aircraft. In terms of the energy allocation problem, dynamic allocation of resources can be quickly achieved, effectively solving the problem of high-precision control in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a schematic flowchart of the present invention;

[0057] Figure 2 is a schematic diagram of the communication network topology in an embodiment of the present invention;

[0058] Figure 3 is a schematic diagram of the evolution of the player's actions in an embodiment of the present invention;

[0059] Figure 4 is a schematic diagram of the error between the player's behavior and the Nash equilibrium in an embodiment of the present invention;

[0060] Figure 5 is a schematic diagram of the player's estimation of other participants in an embodiment of the present invention;

[0061] Figure 6 is a schematic diagram of the trigger communication moment in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0063] Embodiment 1:

[0064] Please refer to Figures 1-6 , a method for accelerating distributed Nash equilibrium search based on event-triggered control, including the following steps:

[0065] The first step: In a multi-agent system, an undirected graph is used to construct the communication between nodes to achieve information exchange.

[0066] In a non - cooperative game with N players, the set of participants is denoted as N = {1, 2, …, N}. Each participant i corresponds to a cost function The set x, which is composed of ordered arrays of all N real numbers, is defined as the set of action profiles. Among them, is the N - dimensional Euclidean space, N is a positive integer, and the set x = [x 1 , x 2 , …, x N T , represents the action strategy of the i - th participant. is an estimate of the entire action set x, where y ij is the approximate estimate of the action of participant j by participant i. Information can be communicated between adjacent agents through an undirected connected graph. An undirected graph is connected if there is at least one path between any two nodes. If there is a connected edge between nodes, agent i and agent j can communicate with each other, and agents can obtain the strategy information of adjacent agents through the undirected graph;

[0067] Step 2: By the gradient - descent method, adjust the appropriate step size and establish a gradient - descent model.

[0068] Establish the following dynamic system:

[0069]

[0070] Among them, is the N - dimensional Euclidean space; g(x, t) is a non - linear function that changes with time t and satisfies g(0 N , t) = 0; t is time.

[0071] Add the previous information to the iteration formula to introduce a momentum term, with

[0072] Consider the following strategy for seeking Nash equilibrium, that is, participants update their own strategies through this strategy:

[0073]

[0074] Among them, the set of agents N = {1, 2, …, N}; x is the action set of all agents; is the action strategy of the i - th agent, is used to estimate the entire action set x, where y ij is the approximate estimate of the action of agent j by agent i, is the N - dimensional real Euclidean space.

[0075] ​The cost function of each agent is defined as:

[0076]

[0077] Define the cost function f i (x) such that its gradient satisfies:

[0078] And

[0079] From this, the defined parameters can be obtained:

[0080]

[0081] Is the transpose of H i ,

[0082] γ = [γ 11 , γ 12 , …, γ 1N , γ 21 , …, γ N1 , …, γ NN T And

[0083]

[0084] Is B is a diagonal matrix with the initial connected graph as the diagonal, Is the Laplacian matrix, 1 N Is the N - order identity matrix; Is the time when the triggering event occurs; ξ i Is a set parameter, η >> 1, s > 1 is the ratio of two non - negative odd numbers. Through the design of parameters β, a 1 , a 2 , a 3 The step size of gradient descent can be adjusted to achieve the best optimization effect.

[0085] Step 3: Introduce a condition detector to determine whether the communication trigger condition is reached and optimize the communication network in the model.

[0086] The calculation expression of the said trigger condition is:

[0087]

[0088] Among them, the event - triggering function when the above conditions are met is:

[0089] The constraint conditions are ω > 0 and κ ∈ (0, 1); ​

[0090] Measurement error E ij is defined as:

[0091]

[0092] where χ ij (t) is an internal dynamic variable, and its derivative satisfies

[0093] is a constant, a 1 , a 2 , a 3 , a 4 , a 5 , a 6 > 0; δ ∈ (0, 1), ω, κ are design parameters.

[0094] In this embodiment, a consensus-based estimation method is adopted to correct the strategies of each participant. In formula (4), it plays an accelerating role in making the estimated value close to the true value, it plays an accelerating role when there is a significant difference between the estimated value and the true value, it plays an accelerating role when achieving exponential fast convergence.

[0095] Fourth step: Meet the convergence requirement through Lyapunov consensus and calculate the upper bound of fixed-time convergence.

[0096] Construct the Lyapunov function V 1 (t). For the convenience of expression, this function is divided into two parts Y 1 (t) and Y 2 (t) for proof. V 1 (t) = Y 1 (t) + Y 2 (t), and the definitions of Y 1 (t) and Y 2 (t) are as follows;

[0097] where the defined function is:

[0098]

[0099] λ min is the minimum value of a defined variable λ containing parameters, satisfying:

[0100]

[0101] Based on -γ ij E ij (t) ≤ |γ ij ||E ij | and for any number There is an inequality holds, where β >> 1 and

[0102] depends on the identity. For any number F 1 and F 2 , it satisfies and for any real number sequence γ 1 , γ 2 , …, γ N , the inequality holds, where

[0103] Similarly, for it also holds.

[0104] Differentiate V 1 (t) with respect to time t:

[0105]

[0106] where

[0107]

[0108] The upper bound of the convergence time can be calculated as satisfies:

[0109]

[0110] where ρ ∈ (0, 1), there exists

[0111]

[0112] Under the above conditions, the ratio of the left limit of the measurement error corresponding to the next trigger to the upper bound of its change rate is less than the interval between execution times, that is:

[0113]

[0114] The numerator is strictly non - negative and the denominator is bounded above, and the Zeno phenomenon does not exist.

[0115] Let By calculation, the time derivative of V 2 is:

[0116]

[0117] Design the node set of the communication topology graph as The node set is divided into two subsets, and the defined subsets satisfy

[0118] Define \(e\) i \(=y\) i \(-x\), there exists \(\parallel e\parallel\) i (t)\(\parallel\leqslant\tau\), for \(t\geqslant T\) 1max , there is Since for all have where Therefore for there is \(\tau\) is a defined parameter.

[0119] Let If for it is said that there exists Then for there is the following setting: \(d\) i \(=2\). If for there exists Then for there is the following setting: \(d\) i \(=-2\). If neither of the above situations is satisfied, then set \(d\) i \(=0\). Therefore, for there exists:

[0120]

[0121] When For all agents \(i, j\), if it satisfies

[0122] And \(a\) 2 (1 - k)\(>\beta\),

[0123] Within a fixed time, estimate \(y\) ij can converge to a neighborhood centered on the true action \(x\) j . There is where \(\zeta\in(0,1)\) and

[0124] Since For get

[0125]

[0126] \(x\) * is the state of the participants after reaching equilibrium; thus, it can be obtained that when converges to a neighborhood centered on the true action.

[0127] Step 5: Iteratively update the equations according to the proposed distributed Nash equilibrium search method (Equations 2a and 2b) to achieve a fixed-time Nash equilibrium within a fixed time. The pseudocode of the search method is as follows:

[0128]

[0129] The present invention can ensure that the player's estimate converges to a small neighborhood around the actual action within a fixed time, and this time can be predefined only through design parameters. The present invention uses an additional internal dynamic variable to further reduce energy consumption. It is activated only when there is a necessity to update the estimation dynamics. By introducing the dynamic variable, the number of trigger times is reduced, effectively reducing unnecessary communication times, thereby optimizing the use of network resources. Under the same conditions, the present invention can save the memory size of data storage, reduce the number of communications, and reduce problems such as heat generation and busy operation of the aircraft. In the energy allocation problem, it can quickly achieve dynamic allocation of resources and effectively solve the problem of high-precision control in complex environments.

[0130] The present application also provides a computer storage medium for storing program data. When the program data is executed by a processor, it is used to implement the accelerated distributed Nash equilibrium search method based on event-triggered control as described in the above embodiments.

[0131] The present application also provides a computer program product. The computer program product includes a computer program that can be operated to cause a computer to execute the accelerated distributed Nash equilibrium search method based on event-triggered control as described in the embodiments of the present application. The computer program product can be a software installation package.

[0132] When the accelerated distributed Nash equilibrium search method based on event-triggered control described in the above embodiments of the present application exists in the form of a software functional unit and is sold or used as an independent product, it can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., which can store program codes.

[0133] In a specific embodiment, the energy management problem in the intelligent transportation system is simulated, such as Figure 2 shown. Considering four agents, and communicating as Figure 2 shown. The cost function of agent i is expressed as follows:

[0134]

[0135] where q i , w>0 represents a constant related to the user, and w, is a constant. It is calculated that λ min (H -1 ) = 0.4, and λ max (H -1 ) = 0.4762, λ min (H) = 2.1. Let the relevant design coefficients in the cost function be w 1 = 0.1, w = 10, q i = 1, By calculation, the unique Nash equilibrium point is x * = [6.86, 11.62, 16.38, 21.14] T .

[0136] Set the initial state as x(0) = [22, 22, 22, 22] T , y 1 (0) = -[10, 10, 10, 10] T , y 2 (0) = -[5, 5, 5, 5] T , y 3 (0) = [0, 0, 0, 0] T , y 4 (0) = [5, 5, 5, 5] T . Set the relevant design parameters a 1 = 10, a 2 = 11, a 3 = 10, a 4 = 2, a 5 = 0.6, a 6 = 0.1, ω = 2, k = 0.1, δ = 0.5, β = 9, η = 10e7, χ ij (0) = 20 for all agents i, j. Set the value of κ to 0.99, and it can be calculated that the fixed time for the estimated value to reach the actual action is less than 1.76 seconds.

[0137] As Figure 3The figure shows a schematic diagram of the change in the behavior of the agent using the present invention; Figure 4 It shows that the error of the actual behavior of the participant deviating from the Nash equilibrium is obvious, and the player converges to the Nash equilibrium point within 1.8 seconds. Figure 4 It shows that the behavior of the participant quickly approaches the Nash equilibrium point within 1.8 seconds. Figure 5 It shows the predictions of the participants' behavior, which are far lower than the predicted theoretical maximum time limit, and these predictions are closely aligned with the actual behavior within 0.25 seconds. Figure 6 Then it shows that before the estimate approaches the actual behavior, the participants The specific moment triggered by the estimate of. At the beginning, the error |E ij (t)| will rise rapidly because there are variables with powers greater than 1. Due to the effect of the hyperbolic tangent function, as the estimated value gradually approaches the actual action of the player, |E ij (t)| increases rapidly, triggering the activation of the trigger mechanism. Theoretically, once an agreement is reached, there will be no further triggers. However, in actual projects, since the system is discrete, γ ij (t) oscillates around the origin, so it will still cause triggers. It reveals that the fixed convergence time of the player at the appropriate trigger moment is 0.25 seconds.

Claims

1. An accelerated distributed Nash equilibrium search method based on event-triggered control, characterized in that: The following steps are involved: S1: Use undirected graphs to build communication between nodes in a multi-agent system to achieve information exchange; S2: Build a gradient descent model: (2a)； The cost function of each agent is defined as: Cost function f i The gradient of (x) satisfies: and This results in the following defined parameters: Where, the agent set N = {1, 2, ..., N}; x is the action set of all agents; is the action strategy of the ith agent, Used to estimate the entire action set x, y ij is the approximate estimate of agent i’s action on agent j, is an N-dimensional real Euclidean space; H i The transpose of γ=[γ 11 ,γ 12 ,…,γ 1N ,γ 21 ,…,γ N1 ,…,γ NN ] T and B is a diagonal matrix with the initial connected graph as the diagonal. is the Laplace matrix, 1 N is an N-order unit matrix; is the time when the trigger event occurs; η>>1, s>1 is the ratio of two non-negative odd numbers, and the step size of gradient descent is adjusted by parameters β, a1, a2, a3; S3: Determine whether the triggering condition for communication is met through the condition detector, and optimize the communication network in the gradient descent model; S4: Satisfy the convergence requirements through Lyapunov consensus and calculate the upper bound of fixed-time convergence; S5: Iteratively update equations through distributed neural dynamics methods to achieve fixed-time Nash equilibrium for controlling distributed multi-agent systems.

2. The method for accelerating distributed Nash equilibrium search based on event-triggered control according to claim 1, characterized in that: In S1, an undirected graph is used to construct a communication method between nodes. If there is at least one path between any two nodes, the undirected graph is connected. If there are connected edges between nodes, agents i and j can communicate with each other. Agents can obtain strategy information of adjacent agents through the undirected graph.

3. The method for accelerating distributed Nash equilibrium search based on event-triggered control according to claim 1, characterized in that: In S3, the calculation expression of the trigger condition is: When the above trigger conditions are met, the event is triggered. The function is: The restrictions are ω>0 and κ∈(0,1); in, In the formula, χ ij (t) is an internal dynamic variable, and its derivative satisfies: a1, a2, a3, a4, a5, a6 are constants, a1, a2, a3, a4, a5, a6>0; δ∈(0,1), ω, κ are design parameters; E ij (t) is the measurement error.

4. The method for accelerating distributed Nash equilibrium search based on event-triggered control according to claim 1, characterized in that: In S4, for all agents, if the following conditions are met: and a2(1-κ)>β, In a fixed time, the estimates of all agents can converge to a neighborhood centered on the true action; The fixed time convergence upper bound is: T 1max +T2(x(T 1max )), in, ρ∈(0,1), And ζ∈(0,1);λ min ,λ max are the minimum and maximum values ​​of the variable λ with parameters defined respectively; 5. The method for accelerating distributed Nash equilibrium search based on event-triggered control according to claim 1, characterized in that: In S3, the distributed neural dynamics method comprises the following steps: Input and initialize parameters: including communication topology G, cost function f i (x), initial value x i , initial prediction value And parameter a i ,ω,β,η,s,δ; when hour, calculate The cost function f i Assign (x) to H(x) and calculate the following expression; Return x k , and finally output x * .

Images