Convergence control method and system of random multi-agent system under matrix weighted topology

Abstract

The invention relates to a convergence control method and system of a random multi-agent system under matrix weighted topology, and belongs to the technical field of distributed cooperative control of multi-agent systems. The method comprises the following steps: (1) selecting estimation duration and state updating times, constructing a communication topological graph among multiple agents, and determining a weight matrix; (2) establishing a dynamic model of each agent; (3) establishing a Kalman filtering estimator of each agent in a dual-time scale process; (4) designing a convergence control protocol; and (5) if the number of state updating times of each agent reaches a preset value, the MASs reach convergence, otherwise, the step (4) is repeated. According to the method, convergence of a multi-agent system influenced by communication noise under matrix weighted topology is realized, the robustness of the system is improved, and control input items of neighbor agents in a Kalman filtering estimator are removed.

Description

Technical Field

[0001] The present invention relates to a convergence control method and system for a stochastic multi-agent system under matrix weighted topology, and belongs to the technical field of distributed cooperative control of multi-agent systems. Background Art

[0002] With the continuous development of science and technology, artificial intelligence technology is gradually becoming the core driving force for promoting social transformation and economic innovation. Among the many applications of artificial intelligence technology, multi-agent systems (MASs) as an important research direction have been widely used in many fields such as smart grids, unmanned driving, and robot cooperation. The so-called multi-agent refers to those systems that are independent of each other but can work together. Multiple independent but cooperative agents form MASs through cooperation.

[0003] Convergence means that in MASs, agents design distributed control protocols by using the interaction information of neighbor agents, so that the agents in the system reach an agreement on some variables of interest (such as attitude, position, speed, voltage, etc.). Due to its wide application in the fields of distributed optimization, swarm control, and sensor networks, the convergence problem of MASs has been studied and concerned by many scholars.

[0004] At present, researchers usually study the convergence problem of MASs in scalar weighted topology, and rarely study the convergence problem of MASs under matrix weighted topology. However, in some actual scenarios, compared with scalar weighted topology, matrix weighted topology is more suitable for describing the relationship between high-dimensional states of agents. For example, the study of social network dynamics, the synchronization of coupled LC oscillators, the distributed control of spacecraft formations, the bearing stiffness theory and its applications, etc. Matrix weighted topology extends the edge weights in scalar weighted topology to matrices, that is, the communication weights can be positive definite matrices, semi-positive definite matrices or zero matrices, and the corresponding edge connection methods are full connection, semi-connection or no connection. When the weight matrix in matrix weighted topology contains only one element, the traditional scalar weighted topology can be regarded as a special case of matrix weighted topology.

[0005] In practical applications, the neighbor interaction information obtained by multi-agent systems through the network is inevitably affected by communication noise. The existence of communication noise may significantly affect the performance of the system and even undermine its stability. Using the Kalman filter to estimate the states of multi-agent systems and thereby weaken the impact of communication noise is an effective and commonly used method. As a classic state estimation algorithm, the Kalman filter uses Bayesian methods for state estimation under the assumptions that the observed system is linear and the measurement noise is Gaussian noise. In the case of a single agent, it can achieve the minimum mean square estimation error performance. Due to its well-known optimal properties and numerical efficiency, the Kalman filter has been widely applied, such as in sensor networks, autonomous vehicles, satellite orbit prediction, etc.

[0006] Currently, there are two basic challenges in the research on the convergence problem of MASs under matrix-weighted topologies: On the one hand, in practical engineering, communication noise is inevitable, and agents cannot receive accurate interaction information from their neighbor agents. It is necessary to consider how to design a convergence protocol for MASs using the Kalman filter. On the other hand, when using the Kalman filter to design a convergence protocol, the Kalman filter estimator of each agent contains the control input terms of neighbor agents. In reality, agents cannot accurately obtain this information, and it is necessary to consider how to design a Kalman-filter-based convergence protocol under a double time scale. Summary of the Invention

[0007] Aiming at the deficiencies of the prior art, the present invention provides a convergence control method and system for a stochastic multi-agent system under a matrix-weighted topology, which realizes the convergence of a multi-agent system affected by communication noise under a matrix-weighted topology, improves the robustness of the system, and also removes the control input terms of neighbor agents in the Kalman filter estimator.

[0008] The technical solution of the present invention is as follows:

[0009] A convergence control method for a stochastic multi-agent system under a matrix-weighted topology, the steps are as follows:

[0010] (1) Select the estimation duration and the number of state updates, construct a communication topology graph among multi-agents and determine the weight matrix;

[0011] (2) Establish the dynamic model of each agent;

[0012] (3) Establish the Kalman filter estimator of each agent under a double time scale process;

[0013] (4) Design a consensus control protocol. In the estimation stage, each agent has no control input and uses a Kalman filter estimator to estimate the states of its neighbors. After the estimation duration ends, it enters the control stage. In the control stage, the estimated values of the Kalman filter estimator remain unchanged. Each agent designs a consensus control protocol using the estimated values and its own state to apply control to itself, records the number of times its own state is updated, and then enters the estimation stage.

[0014] (5) If the number of times each agent's state is updated reaches a preset value, the MASs achieve consensus; otherwise, repeat step (4).

[0015] Preferably according to the present invention, in step (1), the specific steps are as follows:

[0016] Consider a MASs composed of N agents, and the communication topology is represented by an undirected graph where and represent the sets of agents, communication edges, and matrix weights respectively; e ij =(i, j) is an ordered array used to represent the edge in the undirected graph ; if there exists e ij =(i, j) in the set ε, it means there is a communication behavior between agent i and agent j. According to e ij =(i, j), find the corresponding edge weight matrix in the matrix weight set ; represents an n×n-dimensional real matrix;

[0017] Denote as the neighbor set of the i-th agent. If there is no communication behavior between agent i and j, then A ij is a zero matrix; if there is a communication behavior between agent i and j, then A ij is a positive definite matrix or a semi-positive definite matrix; the Laplacian matrix is used to describe the properties of the undirected graph , where and l uj =-A ij , i≠j; represents an nN×nN-dimensional real matrix;

[0018] Let be the initial state of the i-th agent, be the initial estimate of the i-th agent for the states of its neighbors, and R>0 be the covariance matrix of the communication noise;

[0019] Select the weight matrix of each edge according to the communication topology graph

[0020] Select the estimated duration Denote the set of positive integers; for the selection of the estimated duration, it can be determined according to different application scenarios, such as L k = C is applicable to scenarios with relatively small environmental changes in multi-agent consensus control, that is, the communication topology and system dynamics are relatively stable; L k = (t k + m) α is applicable to scenarios where the task complexity changes with time, such as multi-agent games or multi-agent reinforcement learning, where C, m, α are integers; according to the estimated duration, set the state update threshold of each agent to β.

[0021] According to the preference of the present invention, in step (2), the communication topology graph the dynamic model of each agent in is:

[0022]

[0023] wherein, and respectively represent the state and control input of the i-th agent, is the observation of the i-th agent on the states of its neighbors, is independent zero-mean Gaussian white noise with covariance matrix R>0.

[0024] According to the preference of the present invention, in step (3), specifically:

[0025] The i-th agent uses a Kalman filter estimator to estimate the states of neighboring agents from the observation information y ji (t):

[0026]

[0027] wherein, the initial value is arbitrarily selected, represents the estimator at the left limit of time t k+1 , and the matrix P(t) satisfies the following Riccati equation:

[0028]

[0029] The initial value P(0)>0. If the computing performance of each agent is limited, the matrix P(t) can be calculated in advance and stored in the memory; for example, after determining the estimated duration L of each agent kAfter that, it is possible to choose whether to calculate P(t) in advance according to the performance of the agent. If the performance of the agent is poor, the matrix P(t) can be calculated on a computer with strong performance, and the pre-calculated value of P(t) can be written into the memory of each agent when initializing the startup parameters of each agent.

[0030] According to a preferred embodiment of the present invention, in step (4), a consensus control protocol is designed: the i-th agent updates its own state using the following consensus control protocol according to the estimated neighbor states as follows:

[0031]

[0032] Repeat the state estimation and consensus control until the MASs reach consensus.

[0033] A consensus control system for a stochastic multi-agent system under matrix-weighted topology, comprising:

[0034] A parameter configuration module, configured to: select the estimation duration and the number of state updates, construct a communication topology graph among multi-agents and determine the weight matrix;

[0035] A dynamics module, configured to: establish a dynamics model for each agent;

[0036] A Kalman filtering module, configured to: establish a Kalman filter estimator for each agent in a two-time-scale process;

[0037] A state update module, configured to: design a consensus control protocol. In the estimation stage, each agent uses the Kalman filter estimator to estimate the states of its neighbors; in the control stage, the estimated values of the Kalman filter estimator remain unchanged, and each agent designs a consensus control protocol using the estimated values and its own state;

[0038] A control module, configured to: judge the number of state updates of the agent. If the number of state updates of each agent reaches a preset value, the MASs reach consensus, otherwise, repeat the state update.

[0039] The beneficial effects of the present invention are as follows:

[0040] 1. The present invention studies the consensus problem of MASs affected by communication noise under matrix-weighted topology, combines the two-time-scale process and the Kalman filtering algorithm, realizes the consensus of MASs, and improves the robustness of the system.

[0041] 2. Compared with the general consensus protocol based on Kalman filtering, the present invention separates the system estimation and control, removing the control input term of the neighboring agents in the Kalman filter estimator. This approach makes the consensus algorithm of the present invention more practical for engineering applications.

[0042] 3. The initial value of the traditional Kalman filter estimator follows a Gaussian distribution, while the initial value of the Kalman filter estimator in the present invention is arbitrarily selected, which can reduce the engineering workload when implementing the consensus algorithm of the present invention in engineering. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic diagram of the double-time-scale process used in the present invention;

[0044] Figure 2 is a schematic diagram of the flow of the consensus control method for a stochastic multi-agent system under the matrix-weighted topology of the present invention;

[0045] Figure 3 is a schematic diagram of the matrix-weighted topological structure of the first-order multi-agent system established in the embodiment of the present invention;

[0046] Figure 4 is a schematic diagram of the trajectory change of the Kalman filter estimation error value of each agent in the matrix-weighted topological structure diagram of the first-order multi-agent system in the embodiment of the present invention; in the figure, the curve fluctuates within a small range as time t changes, indicating that the estimator based on Kalman filtering designed by the present invention can accurately estimate the state of the agent;

[0047] Figure 5 is a schematic diagram of the trajectory change of the first component of the state of each agent in the matrix-weighted topological structure diagram of the first-order multi-agent system in the embodiment of the present invention;

[0048] Figure 6 is a schematic diagram of the trajectory change of the second component of the state of each agent in the matrix-weighted topological structure diagram of the first-order multi-agent system in the embodiment of the present invention;

[0049] Figure 7 is a schematic diagram of the trajectory change of the third component of the state of each agent in the matrix-weighted topological structure diagram of the first-order multi-agent system in the embodiment of the present invention;

[0050] Figure 8 is a schematic diagram of the trajectory change of the state error value of each agent in the matrix-weighted topological structure diagram of the first-order multi-agent system in the embodiment of the present invention;

[0051] Figure 5 , Figure 6 and Figure 7The curves in it respectively represent the change trajectories of the first, second, and third components of the agent state. As time t changes, the value of each component tends to a fixed value, indicating that the convergence protocol designed by the present invention can enable multi-agents to achieve convergence.

[0052] Figure 8 The curve in it represents the change trajectory of the multi-agent convergence error. As time t changes, the convergence error gradually tends to zero, meaning that the multi-agents achieve convergence. Specific implementation manners

[0053] The present invention will be further described below by way of examples in conjunction with the accompanying drawings, but is not limited thereto.

[0054] Example 1:

[0055] As Figures 1-3 shown, this example provides a convergence control method for a stochastic multi-agent system under matrix weighted topology, and the steps are as follows:

[0056] (1) Select the estimation duration and the number of state updates, construct the communication topology graph among multi-agents and determine the weight matrix; consider a MASs composed of N agents, and the communication topology is represented by an undirected graph where and respectively represent the sets of agents, communication edges, and matrix weights; e ij =(i, j) is an ordered array used to represent the edge in the undirected graph ; if there exists e ij =(i, j) in the set ε, it means that there is a communication behavior between agent i and agent j. According to e ij =(i, j), find the corresponding edge weight matrix in the matrix weight set ; represents an n×n-dimensional real matrix;

[0057] Denote as the neighbor set of the i-th agent. If there is no communication behavior between agent i and j, then A ij is a zero matrix; if there is a communication behavior between agent i and j, then A ij is a positive definite matrix or a semi-positive definite matrix; the Laplacian matrix is used to describe the properties of the undirected graph where and l ij =-A ij , i≠j; represents an nN×nN-dimensional real matrix;

[0058] Let is the initial state of the \(i\)-th agent, is the initial estimate of the \(i\)-th agent about the states of its neighbors, and \(R>0\) is the covariance matrix of the communication noise;

[0059] According to the communication topology graph Select the weight matrix for each edge

[0060] Select the estimation duration denotes the set of positive integers; for the selection of the estimation duration, it can be determined according to different application scenarios. For example, \(L\) k \(=C\) is applicable to scenarios with relatively small environmental changes in multi-agent consensus control, that is, the communication topology and system dynamics are relatively stable; \(L\) k \(=(t\) k \(+m)\) α is applicable to scenarios where the task complexity changes with time, such as multi-agent games or multi-agent reinforcement learning, where \(C\), \(m\), \(\alpha\) are integers; according to the estimation duration, set the state update threshold of each agent to \(\beta\).

[0061] (2) Communication topology graph The dynamic model of each agent in it is:

[0062]

[0063] where, and respectively represent the state and control input of the \(i\)-th agent, is the observation of the \(i\)-th agent about the states of its neighbors, is independent zero-mean Gaussian white noise with covariance matrix \(R>0\).

[0064] (3) Establish the Kalman filter estimator for each agent under the double-time-scale process. Specifically:

[0065] The \(i\)-th agent uses the Kalman filter estimator to estimate the states of neighboring agents from the observation information \(y\) ji (t):

[0066]

[0067] where, the initial value is arbitrarily selected, represents the left limit of the estimator at time \(t\) k+1 , and the matrix \(P(t)\) satisfies the following Riccati equation:

[0068]

[0069] The initial value P(0) > 0. If the computing performance of each agent is limited, the matrix P(t) can be calculated in advance and stored in the memory. For example, after determining the estimation duration L of each agent k After that, it is possible to choose whether to calculate P(t) in advance according to the performance of the agent. If the performance of the agent is poor, the matrix P(t) can be calculated on a computer with strong performance, and the pre-calculated value of P(t) can be written into the memory of each agent when initializing the startup parameters of each agent.

[0070] (4) Design a consensus control protocol. In the estimation stage, each agent has no control input and uses a Kalman filter estimator to estimate the states of its neighbors. After the estimation duration ends, it enters the control stage. In the control stage, the estimated value of the Kalman filter estimator remains unchanged. Each agent designs a consensus control protocol using the estimated value and its own state to exert control on itself, records the number of times its own state is updated, and then enters the estimation stage.

[0071] Design a consensus control protocol: The i-th agent updates its own state according to the estimated neighbor states using the following consensus control protocol:

[0072]

[0073] The two-time-scale process of this embodiment is divided into an estimation stage and a control stage. As Figure 1 shown, in the estimation stage each agent in the system uses a Kalman filter estimator to estimate the states of neighboring agents; in the control stage the estimator of each agent remains, and the estimated states of neighboring agents are used to update its own state. This method removes the control input term of neighboring agents in the Kalman filter estimator and effectively solves the consensus problem of MASs affected by communication noise under matrix-weighted topologies.

[0074] (5) If the number of times each agent's state is updated reaches a preset value, then the MASs achieve consensus; otherwise, repeat step (4).

[0075] Repeat state estimation and consensus control until the MASs achieve consensus.

[0076] In the specific engineering application of this embodiment, simulation can be carried out in advance to facilitate the adjustment of relevant parameters.

[0077] If the engineering staff is not sure how to choose the estimation duration L k , it is possible to simulate the curve in Figure 4 and, according to whether more accurate state estimation is required, refer to Figure 4Select the estimation duration L based on the change in the estimation error k .

[0078] If the engineer is unsure of the state value when the agent reaches consensus, they can simulate Figure 5 , Figure 6 and Figure 7 the curves in to determine the final consensus value of the agent, in order to better arrange the engineering tasks after the agent reaches consensus.

[0079] If the engineer is unsure of the number of state updates of the agent, they can simulate Figure 8 the curve in, and select the moment that meets the engineering data accuracy in the change trajectory of the consensus error according to the requirements of data accuracy in the project to determine the number of state updates of each agent.

[0080] Embodiment 2:

[0081] This embodiment provides a consensus control system for a stochastic multi-agent system under matrix weighted topology, including:

[0082] A parameter configuration module, configured to: select the estimation duration and the number of state updates, construct a communication topology graph between multi-agents, and determine the weight matrix;

[0083] A dynamics module, configured to: establish a dynamics model for each agent;

[0084] A Kalman filter module, configured to: establish a Kalman filter estimator for each agent in a two-time scale process;

[0085] A state update module, configured to: design a consensus control protocol. In the estimation stage, each agent uses the Kalman filter estimator to estimate the state of its neighbors; in the control stage, the estimated value of the Kalman filter estimator remains unchanged, and each agent uses the estimated value and its own state to design a consensus control protocol;

[0086] A control module, configured to: judge the number of state updates of the agent. If the number of state updates of each agent reaches a preset value, the MASs reach consensus, otherwise, repeat the state update.

[0087] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made by those skilled in the art without creative efforts within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A convergence control method for a random multi-agent system under a matrix-weighted topology, characterized in that: Here are the steps: (1) Select the estimated duration and number of state updates, construct the communication topology graph between multiple agents, and determine the weight matrix; (2) Establish a dynamic model for each agent; (3) Establish a Kalman filter estimator for each agent in a dual time scale process; (4) Design a convergence control protocol. In the estimation phase, each agent does not have any control input and uses a Kalman filter estimator to estimate the state of its neighbors. After the estimation period is over, it enters the control phase. In the control phase, the estimated value of the Kalman filter estimator remains unchanged. Each agent uses the estimated value and its own state to design a convergent control protocol to control itself, record the number of times its own state is updated, and then enters the estimation phase; (5) If the number of state updates of each agent reaches a preset value, the MASs reach convergence, otherwise repeat step (4).

2. The convergence control method of a random multi-agent system under a matrix-weighted topology as claimed in claim 1, characterized in that: In step (1), the specific steps are as follows: Consider a MAS composed of N agents, and the communication topology uses an undirected graph Indicates that and Represent the set of agents, communication edges and matrix weights respectively; e ij =(i,j) is an ordered array used to represent an undirected graph If there is an edge in the set ε ij =(i,j), which means that there is communication between agents i and j. ij =(i,j) in the matrix weight set Find the corresponding edge weight matrix in; represents a real matrix of n×n dimensions; remember is the neighbor set of the ith agent. If agents i and j do not have any communication behavior, then A jj is a zero matrix; If agents i and j have communication behavior, then A ij is a positive definite matrix or a semi-positive definite matrix; the Laplace matrix Used to describe undirected graphs of properties, among which And l ij =-A ij ,i≠j; represents a real matrix of nN×nN dimensions; set up is the initial state of the ith agent, is the initial estimate of the i-th agent on the state of its neighbors, R>0 is the covariance matrix of the communication noise; According to the communication topology diagram Choose the weight matrix for each edge Select estimated duration Represents a set of positive integers; according to the estimated duration, the state update threshold of each agent is set to β.

3. The convergence control method of random multi-agent system under matrix weighted topology as claimed in claim 2, characterized in that: In step (2), the communication topology diagram The dynamic model of each agent in is: in, and denote the state and control input of the ith agent, respectively. is the observation of the i-th agent on the state of its neighbors, is an independent zero-mean Gaussian white noise with a covariance matrix of R>

0.

4. The convergence control method of a random multi-agent system under a matrix-weighted topology as claimed in claim 3, characterized in that: In step (3), specifically: The i-th agent uses the Kalman filter estimator to estimate the observed information y ji (t) estimates the state of the neighboring agents: Among them, the initial value Choose any one, Representation Estimator In t k+1 The left limit of time, the matrix P(t) satisfies the following Riccati equation: The initial value P(0)>0. If the computing performance of each agent is limited, the matrix P(t) is calculated in advance and stored in the memory.

5. The convergence control method of random multi-agent system under matrix weighted topology as claimed in claim 4, characterized in that: In step (4), a convergence control protocol is designed: the i-th agent estimates the state of its neighbors Use the following convergence control protocol to update its own status: The state estimation and convergence control are repeated until the MASs reach convergence.

6. A convergence control system for a random multi-agent system under a matrix-weighted topology, characterized in that: include: The parameter configuration module is configured to: select the estimated duration and the number of state updates, construct the communication topology diagram between multiple agents and determine the weight matrix; The dynamics module is configured to: build a dynamics model for each agent; The Kalman filter module is configured to: establish a Kalman filter estimator for each agent under a dual time scale process; The state update module,is configured to: design a convergence control protocol, in the estimation phase, each agent estimates the state of its neighbors using a Kalman filter estimator; In the control phase, the estimated value of the Kalman filter estimator remains unchanged, and each agent uses the estimated value and its own state to design a convergent control protocol; The control module is configured to: determine the number of state updates of the agent, if the number of state updates of each agent reaches a preset value, the MASs reach convergence, otherwise the state update is repeated.

Images