Multi-agent system control method, system and device and storage medium
By designing an adaptive optimal consistency controller in a multi-agent system, using fuzzy integral reinforcement learning-driven adaptive parameter updates, the calculation burden caused by the increase in the system order is solved, and the control accuracy and efficiency are improved.
Patent Information
- Application Number
- CN202411238632.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-12-10
AI Technical Summary
As the order of multiple agent systems increases, the computing burden of the controller in the prior art increases, reducing the control accuracy, and multiple networks are required for each step, resulting in excessive computing burden.
By establishing dynamic models of followers and leaders, defining local error variables and optimizing objective functions, designing an adaptive optimal consistency controller, and using fuzzy integral reinforcement to learn Bellman's residual-driven adaptive parameter update law to reduce control The number of networks of the device improves the computing efficiency.
Without losing control performance, the computing burden of the controller is reduced, the convergence speed of the multi-agent system is accelerated, the control accuracy is improved, and working time is effectively saved.
Smart Images
Figure CN119105280A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of multi-agent adaptive optimal consistency control, and in particular to a multi-agent system control method, system, device and storage medium. Background Art
[0002] A multi-agent system is a complex system composed of several agents that communicate only through limited communication. It has advantages in terms of task execution capability, reliability, scalability, and maintainability, and is widely used in the fields of multi-unmanned systems, satellite systems, and multi-robot systems. In the field of multi-agent collaborative control, consistency is a basic core issue. Consistency refers to the state or output of each agent in a multi-agent system becoming the same after a period of time. At present, distributed control is an effective means to achieve consistency in multi-agent systems.
[0003] In recent years, intelligent control technologies based on fuzzy logic systems and neural networks have played an important role in multi-agent consistency control and solved the consistency control problem in uncertain dynamic situations. However, with the continuous deepening of research, especially under the premise of practical demand drive, researchers are paying more and more attention to the computational burden and optimality of multi-agent system control algorithms in actual deployment. On the one hand, optimal consistency control provides a powerful means to solve the problem of energy consumption. At present, relevant technical means have been used to achieve optimal consistency control of multi-agents. For high-order systems, the optimal backstepping method based on reinforcement learning solves the above problems. In order to reduce the computational burden, the network structure in the controller is trimmed to reduce the number of networks in the controller without losing control performance.
[0004] However, the current approach requires a network at each step in the design process, which increases the computational burden of the controller and reduces the control accuracy of the multi-agent system as the system order increases. Summary of the invention
[0005] In view of the deficiency of the prior art that the computational burden of the controller increases with the increase of the system order, the present invention proposes a multi-agent system control method, system, device and storage medium, which solves the problems existing in the prior art by establishing a dynamic model of the follower agent, establishing a dynamic model of the leader, establishing a follower local error variable, defining the follower local optimization objective function and its corresponding value function, establishing an adaptive optimal consistency controller for the follower, and selecting appropriate controller parameters.
[0006] A multi-agent system control method, wherein the multi-agent system includes a leader and multiple followers, and the control method includes the following steps:
[0007] Establish dynamic models of followers and leaders, analyze each dynamic model, and obtain the state variables of followers and leaders;
[0008] According to the state variables of the followers and leaders, an adaptive optimal consistency controller for the followers is established;
[0009] By applying the output of the follower's adaptive optimal consistency controller to the corresponding follower, the follower can track the leader signal based on local information exchange only;
[0010] The establishment of the adaptive optimal consistency controller of the follower includes the following steps:
[0011] According to the state variables of the follower and the leader, the output signal of the leader, the output signal of the follower and the output signal of the follower's neighbor are obtained;
[0012] According to the output signal of the leader, the output signal of the follower and the output signal of the follower's neighbors, the optimal backstepping method is used to obtain the optimal virtual control input of the follower at a certain step, and the local error variable of the follower is obtained by subtracting the state variable of the follower from the optimal virtual control input;
[0013] Establish the follower's local optimization objective function and its corresponding value function according to the follower's local error variable;
[0014] Based on the follower's local optimization objective function and its corresponding value function, an adaptive optimal consistency controller for the follower is established.
[0015] Furthermore, the dynamic models of the follower and the leader are respectively expressed as:
[0016]
[0017] 1≤p≤n i -1
[0018]
[0019] y i =η i ,1
[0020] Where i = 1, 2, ..., N, N is the number of follower agents, η i，p ∈R is the state variable, is the state vector, u i ∈R and y i ∈R are the input and output of the ith follower, is an unknown nonlinear function, n i represents the order of the i-th agent;
[0021]
[0022] y l =η l
[0023] Among them, η l ∈R and y l ∈R represent the state variables and output signals of the leader respectively, represents the time derivative of the state variable.
[0024] Furthermore, according to the output signal of the leader, the output signal of the follower and the output signal of the follower's neighbor, the optimal backstepping method is used to obtain the optimal virtual control input of the follower at a certain step, and the state variable of the follower is subtracted from the optimal virtual control input to obtain the local error variable of the follower. The local error variable of the follower is expressed as:
[0025]
[0026] Where p = 2, 3, ..., n i , i = 1, 2, ..., N; is the optimal virtual control input of the designed ith follower at step p; y j represents the output of the jth neighbor of the ith follower; a i，j ≥0 indicates the connection strength between the ith follower and its neighbor nodes; b i = 0 means that the ith follower is not directly connected to the leader, b i =1 means that the i-th follower is directly connected to the leader.
[0027] Furthermore, the local optimization objective function of the follower and its corresponding value function are established according to the local error variable of the follower, which are respectively expressed as:
[0028]
[0029] in, Defined as T is the matrix transpose; n i is the system order of the ith follower; represents the cost function; it is defined as Q i >0 and R i >0 is a positive definite matrix; the corresponding value function is expressed as:
[0030]
[0031] Furthermore, the adaptive optimal consistency controller for establishing the follower is expressed as:
[0032]
[0033] Among them, K i >0,
[0034] Furthermore, the update law of the adaptive parameters adopts fuzzy integral reinforcement learning Bellman residual drive, which is expressed as:
[0035] The update law of the adaptive parameters in the adaptive optimal consistency controller of the i-th follower is designed as
[0036]
[0037] Among them, l i >0 is the learning rate, Ψ i (ω i ) is defined as:
[0038]
[0039] is a design parameter. λ Defined as "·" λ =|·| λ sign(·);
[0040] The Bellman residual in the adaptive update law is:
[0041]
[0042] in,
[0043]
[0044] It is a design parameter that represents the length of the integration time.
[0045] Furthermore, the method of estimating unknown values and their gradients in the learning process using a fuzzy logic system specifically includes the following steps:
[0046] The fuzzy logic system is used to estimate the unknown value function and its gradient as
[0047]
[0048] in, is the weight of the fuzzy logic system of the i-th follower, is the activation function of the fuzzy logic system of the i-th follower;
[0049] The activation function of the fuzzy logic system is chosen as:
[0050]
[0051] in, represents the number of fuzzy rules in the fuzzy logic system, M represents the number of input vectors in the fuzzy logic system, Represents the parameters of the Gaussian function.
[0052] The present invention also includes a multi-agent system control system, wherein the multi-agent system includes a leader and a plurality of followers, and the control system includes:
[0053] The dynamic model building module is used to build the dynamic models of followers and leaders, analyze each dynamic model, and obtain the state variables of followers and leaders;
[0054] A controller building module is used to build an adaptive optimal consistency controller for followers based on the state variables of followers and leaders;
[0055] A control module, used for applying the output of the follower's adaptive optimal consistency controller to the corresponding follower, so that the follower can track the leader signal based on only local information exchange;
[0056] Wherein, the controller building module includes:
[0057] An output signal acquisition unit, used to obtain the output signal of the leader, the output signal of the follower and the output signal of the follower's neighbor according to the state variables of the follower and the leader;
[0058] The error variable calculation unit is used to obtain the optimal virtual control input of the follower at a certain step by using the optimal backstepping method according to the output signal of the leader, the output signal of the follower and the output signal of the follower's neighbors, and obtain the local error variable of the follower by subtracting the state variable of the follower from the optimal virtual control input;
[0059] A function building unit, used for building a local optimization objective function of the follower and a value function corresponding thereto according to a local error variable of the follower;
[0060] The controller establishment unit is used to establish an adaptive optimal consistency controller of the follower based on the local optimization objective function of the follower and its corresponding value function.
[0061] The present invention also includes a multi-agent system control computer device, including: a memory, a processor, and a computer program stored in the memory, and the processor implements the steps of the multi-agent system control method when executing the computer program.
[0062] The present invention also includes a readable storage medium, which stores a computer program. The computer program includes program instructions, and when the program instructions are executed by a processor, they are used to execute the steps of the multi-agent system control method.
[0063] The present invention provides a multi-agent system control method, system, device and storage medium, which have the following beneficial effects:
[0064] The present invention establishes a dynamic model of followers and leaders, obtains the local error variable of the follower according to the state variable of the follower and the optimal virtual control input, defines the follower's local optimization objective function and its corresponding value function, and thus establishes an adaptive optimal consistency controller for the follower. On the basis of not losing control performance, the number of networks in the controller is reduced, and the calculation burden of the controller is reduced. At the same time, the output signal of the controller is applied to the corresponding follower. The output signal of the follower can track the leader signal based on local information exchange between intelligent agents, and only one adaptive parameter is required in the adaptive controller. By adopting this control method, in the control process of the multi-agent system, the convergence speed is accelerated while the control accuracy is improved, which can effectively save working time for the multi-agent system. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] Figure 1 This is a flow chart of the fuzzy integral reinforcement learning multi-agent adaptive optimal consistency method of the present invention;
[0066] Figure 2 A schematic diagram of the connection relationship between intelligent agents in an embodiment of the present invention;
[0067] Figure 3 A schematic diagram of the state trajectory of each follower in an embodiment of the present invention;
[0068] Figure 4 Schematic diagram of the trajectory of the adaptive parameter norm of each intelligent agent in the embodiment of the present invention. DETAILED DESCRIPTION
[0069] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0070] The present invention proposes a multi-agent system control method for realizing adaptive optimal consistency tracking control of a leader-follower multi-agent system. The method designs a corresponding distributed adaptive optimal consistency controller for each follower agent, specifically comprising the following steps:
[0071] S1. Establish the dynamic model of the i-th follower agent as
[0072]
[0073] 1≤p≤n i -1
[0074]
[0075] y i =η i，1
[0076] Where i = 1, 2, ..., N, N is the number of follower agents, η i,p ∈R is the state variable, is the state vector, u i ∈R and g i ∈R are the input and output of the ith follower respectively. is an unknown nonlinear function.
[0077] S2. Establish the dynamic model of leaders as follows:
[0078]
[0079] y i =η l
[0080] Among them, η l ∈R and y l ∈R represent the state variables and output signals of the leader respectively.
[0081] S3. Establish the local error variable of the i-th follower as
[0082]
[0083] Where p = 2, 3, ..., n i , i = 1, 2, ..., N, is the optimal virtual control input of the designed i-th follower at step p, y j represents the output of the jth neighbor of the ith follower, a i，j ≥0 indicates the connection strength between the ith follower and its neighbor nodes, b i = 0 means that the ith follower is not directly connected to the leader, b i =1 means that the i-th follower is directly connected to the leader.
[0084] S4. Define the local optimization objective function of the i-th follower and its corresponding value function as:
[0085]
[0086] in, Defined as n i is the system order of the ith follower, represents the cost function, defined as Q i >0 and R i >0 is a positive definite matrix. The corresponding value function is described as
[0087]
[0088] S5. Establish the adaptive optimal consistency controller of the i-th follower as:
[0089]
[0090] Among them, K i >0. in,
[0091] The update law of the adaptive parameters in the adaptive optimal consistency controller of the i-th follower is designed as
[0092]
[0093] Among them, l i >0 is the learning rate, Ψ i (ω i ) is defined as
[0094]
[0095] is a design parameter. λ Defined as "·" λ =|·| λ sign(·).
[0096] The Bellman residual in the adaptive update law is
[0097]
[0098] in,
[0099]
[0100] It is a design parameter that represents the length of the integration time.
[0101] The learning process uses a fuzzy logic system to estimate the unknown value function and its gradient as:
[0102]
[0103] in, is the weight of the fuzzy logic system of the i-th follower, is the activation function of the fuzzy logic system of the i-th follower.
[0104] The activation function of the fuzzy logic system is chosen as:
[0105]
[0106] in, represents the number of fuzzy rules in the fuzzy logic system, and M represents the number of input vectors in the fuzzy logic system. Represents the parameters of the Gaussian function.
[0107] A specific example in this implementation example is a multi-agent system consisting of four followers and one leader. The follower dynamics are described as
[0108]
[0109] The output signal of the leader is described as y l = sin(t). The control parameters in this implementation example are selected as T=0.03,l i =1, K i =10I 2×2 The parameters of the fuzzy logic system are chosen as The connection relationship between agents is as follows Figure 2 As shown, 1-4 are followers and 5 is the leader. The state track of each follower in the implementation example is shown in Figure 3 , the trajectory of the adaptive parameter norm of each follower is shown in Figure 4 .
[0110] Based on the same inventive concept, the present invention also proposes a multi-agent system control system, comprising:
[0111] The dynamic model building module is used to build dynamic models of followers and leaders.
[0112] The error variable determination module is used to determine the local error variable of the follower according to the output signals of the dynamic models of the follower and the leader.
[0113] The function establishment module is used to establish the follower's local optimization objective function and its corresponding value function according to the follower's local error variable.
[0114] A controller establishment module is used to establish an adaptive optimal consistency controller of the follower based on the local optimization objective function of the follower and its corresponding value function; the controller includes an adaptive parameter; the update law of the adaptive parameter adopts fuzzy integral reinforcement learning Bellman residual drive, and adopts a fuzzy logic system to estimate unknown values and their gradients in the learning process.
[0115] The control module is used for applying the output of the controller to the corresponding follower so that the output signal of the follower tracks the output signal of the leader based on exchanging information only with its neighboring nodes.
[0116] Based on the same inventive concept, the present invention also proposes a multi-agent system control computer device, including: a memory, a processor and a computer program stored in the memory, and the processor implements the steps of the multi-agent system control method when executing the computer program.
[0117] Based on the same inventive concept, the present invention also proposes a readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, they are used to execute the steps of the multi-agent system control method.
[0118] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A multi-agent system control method, It is characterized in that The multi-agent system includes a leader and multiple followers, and the control method includes the following steps: Establish dynamic models of followers and leaders, analyze each dynamic model, and obtain the state variables of followers and leaders; According to the state variables of the followers and leaders, an adaptive optimal consistency controller for the followers is established; By applying the output of the follower's adaptive optimal consistency controller to the corresponding follower, the follower can track the leader signal based on local information exchange only; The establishment of the adaptive optimal consistency controller of the follower includes the following steps: According to the state variables of the follower and the leader, the output signal of the leader, the output signal of the follower and the output signal of the follower's neighbor are obtained; According to the output signal of the leader, the output signal of the follower and the output signal of the follower's neighbors, the optimal backstepping method is used to obtain the optimal virtual control input of the follower at a certain step, and the local error variable of the follower is obtained by subtracting the state variable of the follower from the optimal virtual control input; Establish the follower's local optimization objective function and its corresponding value function according to the follower's local error variable; Based on the follower's local optimization objective function and its corresponding value function, an adaptive optimal consistency controller for the follower is established.
2. A multi-agent system control method according to claim 1, It is characterized in that The dynamic models of the follower and the leader are expressed as: 1≤p≤n i -1 y i =the i，1 Where i = 1, 2, ..., N, N is the number of follower agents, η i，p ∈R is the state variable, is the state vector, u i ∈R and y i ∈R are the input and output of the ith follower, is an unknown nonlinear function, n i represents the order of the i-th follower; y i =the i Among them, η i ∈R and g i ∈R represent the state variables and output signals of the leader respectively, represents the time derivative of the state variable.
3. A multi-agent system control method according to claim 2, It is characterized in that The local error variable of the follower is expressed as: Among them, p=2,3,...,ni,i=1,2,...,N; is the optimal virtual control input of the ith follower at step p; y j represents the output of the jth neighbor of the ith follower; a i，j ≥0 indicates the connection strength between the ith follower and its neighbor nodes; b i = 0 means that the ith follower is not directly connected to the leader, b i =1 means that the i-th follower is directly connected to the leader.
4. A multi-agent system control method according to claim 3, It is characterized in that The local optimization objective function of the follower is expressed as: in, Defined as T is the matrix transpose; n i is the system order of the ith follower; represents the cost function; it is defined as Q i >0 and R i >0 is a positive definite matrix; The value function corresponding to the follower's local optimization objective function is expressed as:
5. A multi-agent system control method according to claim 4, It is characterized in that The adaptive optimal consistency controller of the follower is expressed as: Among them, K i >0, 6. A multi-agent system control method according to claim 5, It is characterized in that The update law of the adaptive parameters in the adaptive optimal consistency controller of the follower is driven by the fuzzy integral reinforcement learning Bellman residual, where the update law of the adaptive parameters in the adaptive optimal consistency controller of the i-th follower is It is expressed as: in, is the learning rate, Ψ i (ω i ) is defined as: is a design parameter, Defined as Then the update law of the adaptive parameters driven by the Bellman residual of fuzzy integral reinforcement learning is expressed as: in, It is a design parameter that represents the length of the integration time.
7. A multi-agent system control method according to claim 6, It is characterized in that The fuzzy logic system is used to estimate the unknown value function in the fuzzy integral reinforcement learning process. and its gradient Respectively expressed as: in, is the weight of the fuzzy logic system of the i-th follower, is the activation function of the fuzzy logic system of the i-th follower; the activation function of the fuzzy logic system is selected as: in, represents the number of fuzzy rules in the fuzzy logic system, M represents the number of input vectors in the fuzzy logic system, represents the parameters of the Gaussian function, x i Represents the elements in the input vector of the fuzzy logic system.
8. A multi-agent system control system, the multi-agent system comprising a leader and a plurality of followers, It is characterized in that The control system includes: The dynamic model building module is used to build the dynamic models of followers and leaders, analyze each dynamic model, and obtain the state variables of followers and leaders; A controller building module is used to build an adaptive optimal consistency controller for followers based on the state variables of followers and leaders; A control module, used for applying the output of the follower's adaptive optimal consistency controller to the corresponding follower, so that the follower can track the leader signal based on only local information exchange; Wherein, the controller building module includes: An output signal acquisition unit, used to obtain the output signal of the leader, the output signal of the follower and the output signal of the follower's neighbor according to the state variables of the follower and the leader; The error variable calculation unit is used to obtain the optimal virtual control input of the follower at a certain step by using the optimal backstepping method according to the output signal of the leader, the output signal of the follower and the output signal of the follower's neighbors, and obtain the local error variable of the follower by subtracting the state variable of the follower from the optimal virtual control input; A function building unit, used for building a local optimization objective function of the follower and a value function corresponding thereto according to a local error variable of the follower; The controller establishment unit is used to establish an adaptive optimal consistency controller of the follower based on the local optimization objective function of the follower and its corresponding value function.
9. A multi-agent system controls computer equipment. It is characterized in that include: A memory, a processor and a computer program stored in the memory, wherein the processor implements the steps of the multi-agent system control method according to any one of claims 1 to 7 when executing the computer program.
10. A readable storage medium, It is characterized in that The readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, they are used to execute the steps of the multi-agent system control method described in any one of claims 1-7.