A stable control method for multi-controller system based on event triggering
Through non-zero sum differential game model and reinforcement learning technology, combined with event triggering mechanism, the problems of controller optimization performance and communication efficiency in multi-controller systems are solved, and efficient stable control and robustness improvement are achieved.
Patent Information
- Application Number
- CN202210910134.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-29
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-07-29
AI Technical Summary
The control method of traditional multi-controller systems is difficult to take into account the optimization performance of all controllers, and the optimal control model based on event triggering is usually difficult to solve, resulting in waste of communication efficiency and computing resources.
Using the non-zero sum differential game model combined with reinforcement learning and event triggering mechanism, the optimal control strategy is calculated through the Bellmann optimal method and the cost function of the controller, and an evaluation neural network and Lyapunov function are constructed to establish adaptive event triggering rules for optimization. Control decision making.
It improves control decision efficiency, improves communication efficiency, enhances the robustness of the system, and effectively solves the stable control problem of multi-controller systems.
Smart Images

Figure CN115309047B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of event triggering and intelligent control, and more specifically, to a multi-controller system stable control method based on event triggering. Background Art
[0002] Multi-controller systems have received extensive attention and application. They are characterized by multiple mutually coupled controllers and control loops, such as aircraft autopilot systems. The main goal of a multi-controller system is to maintain state variables near nominal or stable values using multiple control inputs with independent optimization requirements. Traditional control methods are time-triggered, that is, the state is periodically transmitted to each controller or actuator, which leads to communication waste and unnecessary computing power consumption. To solve this problem, event-triggered mechanisms with non-periodic sampling have gradually become popular for developing event-triggered control strategies.
[0003] On the other hand, it is difficult for general control methods to take into account the optimization performance of all controllers. Recently, the study of multi-controller systems based on non-zero-sum differential games has received widespread attention in the control community. By deeply exploring the game behavior and optimization relationship between controllers, the overall performance of the system can be improved.
[0004] Therefore, the multi-controller system can be characterized by non-zero-sum differential games, and event triggering can be used to improve performance. However, the optimal control model established in this way is usually difficult to solve. Fortunately, the recent development of intelligent technologies such as reinforcement learning has provided a good idea for solving this problem. Therefore, how to combine the above technologies to develop an intelligent control method and event triggering method is very necessary to improve control effects and communication efficiency. Summary of the invention
[0005] The present invention provides a method for stable control of a multi-controller system based on event triggering. The method solves the problem of stable control of a multi-controller system. The present invention uses differential game to ensure the optimality of the control strategy, uses reinforcement learning to improve the autonomous decision-making ability, and uses the event triggering mechanism to improve the communication efficiency. See the following description for details:
[0006] The present invention is implemented by adopting the following technical scheme:
[0007] A multi-controller system stability control method based on event triggering, the method comprising the following steps:
[0008] Establishing an optimal control model for multi-controller systems using non-zero-sum differential games
[0009] Through the Bellman optimal method and the controller cost function V i(x) Calculate and output the optimal control strategy for the optimal control model
[0010] Construct an evaluation neural network and use experience replay to evaluate the event triggering time of the optimal control strategy Control output event-based control strategy
[0011] Construct a Lyapunov function based on the local cost function and establish the adaptive event triggering rules of the controller;
[0012] The control strategy of the event and the adaptive event triggering rule are applied to a multi-controller system.
[0013] Furthermore, the optimal control strategy output process:
[0014] 101. Establish the optimal control model of non-zero-sum differential game for multi-control systems according to the following formula:
[0015]
[0016] Where: f(x(t)) is the internal dynamic information of the system, N represents the number of players, i.e., controllers; for the i-th player, g i (x(t)) is the control coupling matrix, u i (t) is the control strategy, i.e., the output of the i-th controller;
[0017] 102. Set the quadratic cost function V of the control cost for each controller according to the following formula: i (x)
[0018]
[0019] Among them, Q i ,R ij is a positive and symmetric constant matrix, which is used to adjust the components of system operation cost and control cost in the cost function. It can be seen that this cost function includes the control cost of all controllers. For the i-th controller, its own control cost is
[0020] 103. Based on the optimal cost function, according to the Nash equilibrium condition and Bellman optimality principle, the optimal control strategy of the player is as follows:
[0021]
[0022] in: is the optimal cost function V i *The optimal control strategy of the constructed non-zero-sum differential game model is
[0023] Further, the event-based control strategy output process:
[0024] 201. Use a three-layer feedforward neural network to construct an evaluation neural network to approximate the optimal cost function online and evaluate the output value of the network Will be used to calculate the control strategy, the output value is determined by the evaluation weight And the activation function Characterization is as follows:
[0025]
[0026]
[0027] in: is the partial derivative of the approximate cost function with respect to x;
[0028] 202. Given a control strategy The Newton-Leibniz formula is used to calculate the reward accumulation of the controller in a fixed interval to obtain the reinforcement signal as follows:
[0029]
[0030] Where: T>0 is a fixed integration interval; at the same time, based on the difference of the activation function at different integration times, the regression vector is calculated as follows:
[0031]
[0032] 203. Establish data stack area Store empirical data, where L represents the length of the data stack. The empirical data mainly includes the enhanced signal and the regression vector at different times t q The calculated values are expressed as follows:
[0033]
[0034]
[0035] The standards for stacking empirical data are as follows:
[0036]
[0037] Where: θ i >0 is a small constant;
[0038] 204. Using Evaluation Weights and optimal control strategy At the time of event triggering Compute outputs for event-based control strategies as follows:
[0039]
[0040] Further, the adaptive event triggering rule establishment process:
[0041] 301. Calculate the local sampling error for each controller using the following formula:
[0042]
[0043] 302. Establish the following Lyapunov function based on the local cost function;
[0044]
[0045] in: is the weight error of the evaluation network. This function L i (t) is a criterion that combines stability, convergence, and event triggering;
[0046] 303. Calculate the stability condition based on the Lyapunov function and obtain the following adaptive event triggering rules:
[0047]
[0048] in: are two constant parameters. The trigger rule describes when to generate the required event trigger time, that is, at the current trigger time Get the next trigger time based on The initial triggering moment can be considered as The determination of the next triggering time depends on the system operation cost x T Q i x. Control strategy at the current triggering moment Sampling Error and the evaluation weight at the current moment
[0049] The present invention also provides a non-transitory computer-readable storage medium on which a computer program and control instructions are stored. When the program and instructions are executed by each controller, the multi-controller system can implement the method described in the present disclosure.
[0050] The beneficial effects of the technical solution provided by the present invention are:
[0051] The event-triggered multi-controller system stable control method disclosed in the present invention regards the multi-controller system as a non-zero-sum differential game model, and designs an event-triggered intelligent control method in combination with reinforcement learning, which improves the control decision-making efficiency, effectively improves the communication restricted conditions, and improves the robustness of the system through distributed event triggering.
[0052] The present invention aims at the stable control problem of multi-controller system, and uses differential game, event triggering and reinforcement learning technology to study the event-triggered learning control method under resource-constrained conditions, which is in line with the application requirements and development trends of related technologies. No similar technical solutions were found through the search of existing literature and technologies. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 is a flow chart of a multi-controller system stability control method based on event triggering according to an embodiment;
[0054] Figure 2 is a flow chart of step S10 according to an embodiment;
[0055] Figure 3 is a flow chart of step S20 according to an embodiment;
[0056] Figure 4 is a flow chart of step S30 according to an embodiment;
[0057] Figure 5 is a flow chart of step S40 according to an embodiment;
[0058] Figure 6 1 is a main state variable and control actuator of a typical multi-controller system according to an embodiment;
[0059] Figure 7 A state trajectory and event-based control signal of a typical multi-controller system according to an embodiment are shown; DETAILED DESCRIPTION
[0060] In order to make the purpose, technical scheme and advantages of the present invention clearer, the embodiments of the present invention are further described in detail below. It should be noted that unless otherwise specified, the steps, mathematical expressions and numerical values described in these embodiments do not limit the scope of the present disclosure. At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.
[0061] The present invention provides a multi-controller system stable control method based on event triggering, which is an event-triggered learning control method based on saving communication and computing resources, and specifically comprises the following steps:
[0062] First, an optimal control model based on non-zero-sum differential game is established. A multi-controller system is a typical complex network system, in which each controller and the system form an independent control loop, and all controllers work together on the system to maintain the stability of related state variables. This characteristic can be described by non-zero-sum differential game. In one embodiment, a multi-person non-zero-sum differential game model is established. Each independent controller is considered to be a player. In order to characterize the optimization needs of the player, the cost function of the player is designed by comprehensively considering the system operation cost and the control cost of all players. Each player makes a control decision by minimizing the cost function, and the optimal control strategy of the non-zero-sum differential game model is obtained according to the Nash equilibrium condition and the Bellman optimality principle.
[0063] According to the stable control requirements of the multi-controller system, the present invention can select a quadratic energy cost function to obtain the required optimal control strategy. For different control requirements, the cost function needs to be specifically changed. For example, for the shortest time control requirement, the cost function should mainly reflect the time variable.
[0064] The present invention constructs an evaluation neural network and uses experience playback to obtain an event-based control strategy. For a non-zero-sum differential game model, a multi-layer feedforward neural network is used to construct an evaluation neural network to approximate the optimal cost function online, and the output value of the evaluation network will be used to calculate the control strategy; at the same time, for a given control strategy, the Newton-Leibniz formula is used to calculate the reward accumulation of the player in a fixed interval to obtain a reinforcement signal and a regression vector. In order to make full use of the data generated by learning and improve decision-making efficiency, a data stack area is established to store experience data; the experience data mainly includes state information, reinforcement signals and regression vectors generated by previous iterations; and at the same time, the stacking standard and replacement method of the experience data are designed. Combining real-time data and experience data, the gradient descent adaptive method is used to obtain the adaptive update rule of the evaluation network, which characterizes the update iteration method of the evaluation weight. Combining the evaluation weight and the optimal control strategy, an event-based control strategy is obtained at the event triggering moment, and the control strategy is the actual decision result of each player, that is, the actual output of the controller.
[0065] The reinforcement learning of the present invention can be implemented using a behavior-evaluation network structure, in which the control strategy is directly given by the behavior network rather than indirectly calculated by the evaluation network; in comparison, a single evaluation network structure is easier to implement and has a smaller computational burden.
[0066] The present invention obtains an adaptive event triggering rule based on distributed triggering. Under distributed triggering, each player is equipped with a local event trigger, independently performs state sampling and data transmission, and calculates the local sampling error according to the sampling state recorded by the local event trigger. In order to ensure the optimization and stability of the triggering moment, a Lyapunov function based on the local cost function is established. The stability condition is derived from the Lyapunov function, and the adaptive event triggering rule is obtained by reverse deduction based on the stability condition. The triggering rule describes when to generate the required event triggering moment.
[0067] The event triggering mode of the present invention can be distributed triggering or centralized triggering. The former emphasizes the personalized needs of the players, while the latter emphasizes global synchronization and needs to consider the cost functions of all players.
[0068] The present invention applies the obtained control strategy and event triggering rules to a multi-controller system. The control strategy and triggering rules are loaded into the multi-controller system, wherein each controller updates the control strategy according to the events generated by the local event trigger, and each controller uses the evaluation network to perform a distributed reinforcement learning process to realize event-triggered learning control.
[0069] The present invention also provides a non-transitory computer-readable storage medium on which a computer program and control instructions are stored. When the program and instructions are executed by each controller, the multi-controller system can implement the method described in the present disclosure.
[0070] Example 1
[0071] The embodiment of the present invention aims at the stability control problem of the multi-controller system, based on differential game and reinforcement learning technology, see Figure 1 , the method comprises the following steps:
[0072] S10: Establish an optimal control model based on non-zero-sum differential game;
[0073] Among them, the multi-controller system is a typical complex network system, in which each controller and the system form an independent control loop, and all controllers work together on the system to ensure the stability of related state variables. This characteristic can be described by non-zero-sum differential game. In one embodiment, considering the possible nonlinear characteristics, a multi-person non-zero-sum differential game model is established. Each independent controller is considered as a player, and its control strategy is u i (t), the number of players is N, N ≥ 2. In order to characterize the optimization needs of the players, the system operation cost and the control cost of all players are comprehensively considered, and a quadratic cost function V is designed for each player. i(x). By minimizing this cost function, each player can drive the state to a stable value while optimizing the control cost; the optimal cost function V i * (x) corresponds to the Nash equilibrium solution of the game problem. According to the Nash equilibrium condition and Bellman optimality principle, the optimal control strategy of the non-zero-sum differential game model is obtained.
[0074] S20: Construct an evaluation neural network and use experience replay to obtain an event-based control strategy;
[0075] Among them, for the above non-zero-sum differential game model, a multi-layer feedforward neural network is used to construct an evaluation neural network to approximate the optimal cost function online and evaluate the output value of the network. Will be used to calculate the control strategy, the output value is determined by the evaluation weight And the activation function characterization; at the same time, for a given control strategy Use the Newton-Leibniz formula to calculate the cumulative reward of the player in a fixed interval and obtain the reinforcement signal and the regression vector In order to make full use of the data generated by learning and improve decision-making efficiency, a data stack area is established Store experience data; experience data mainly includes state information x(t) generated by previous iterations, reinforcement signals and the regression vector At the same time, the stacking standard and replacement method of empirical data are designed. The stacking standard determines what kind of learning data can be stored in the stack area, and the replacement method determines how the new and old data are replaced. Combining real-time data and empirical data, the gradient descent adaptive method is used to obtain the adaptive update rule of the evaluation network, which represents the update iterative method of the evaluation weight. and optimal control strategy In the form of, at the time of event triggering Get event-based control strategy The control strategy is the actual decision result of each player, that is, the actual output of the controller.
[0076] S30: Obtaining an adaptive event triggering rule based on distributed triggering;
[0077] Under distributed triggering, each player is equipped with a local event trigger to independently perform state sampling and data transmission. Each player records the sampled state according to the local event trigger. Calculate the local sampling error In order to ensure the optimization and stability of the triggering time, a Lyapunov function L based on the local cost function is established.i (t), this function L i (t) Comprehensively considers the impact of cost function, weight error, and event triggering, and is a criterion that combines stability, convergence, and event triggering. The stability condition is derived by derivation of the constructed Lyapunov function, which describes the derivative According to the stability condition, the adaptive event triggering rule is derived. This triggering rule determines how to trigger the event at the current triggering moment. Get the next trigger time based on
[0078] S40: applying the obtained control strategy and event triggering rules to the multi-controller system;
[0079] The control strategy and trigger rules are loaded into the multi-controller system. Each controller generates an event based on the local event trigger at the event triggering time. Update control strategy Under the distributed trigger setting, the learning of each learning loop presents independent characteristics, and the control strategy is no longer updated synchronously. Each controller uses the evaluation network to execute a distributed reinforcement learning process to achieve event-triggered learning control, and ultimately jointly stabilize the entire system.
[0080] Example 2
[0081] In combination with the specific calculation formula, Example 2 further introduces the solution in Example 1, as shown in detail below:
[0082] First, through Figure 1 In step S10, an optimal control model is established.
[0083] S10: Establish an optimal control model based on non-zero-sum differential game;
[0084] In this embodiment, Figure 2 The optimal control model is established by the steps in step S10, wherein step S10 mainly includes:
[0085] Step S101, establish a multi-person non-zero-sum differential game model. In the stability control problem of a multi-controller system, multiple actuators participate in the stability system together. The actuators can be considered as controllers, and no special distinction will be made in the following text. At this time, there are multiple control loops in the multi-controller system. In order to better exert the control effect of all actuators, a non-zero-sum differential game is used to describe it, in which each controller will be regarded as an independent player; at the same time, considering the nonlinear characteristics of the aircraft itself, a multi-person non-zero-sum differential game model is established as follows:
[0086]
[0087] where f(x(t)) is the internal dynamic information of the system, N represents the number of players; for the i-th player, g i (x(t)) is the control coupling matrix, u i (t) is the control strategy, i.e., the output of the i-th controller. Note that in the subsequent technical descriptions, the time variable t will be omitted for convenience.
[0088] Step S102, designing the cost function of the player. In order to characterize the optimization requirements of the player, the system operation cost and the control cost of all players are comprehensively considered, and a quadratic cost function V is designed for each player. i (x):
[0089]
[0090] Among them, Q i ,R ij is a positive and symmetric constant matrix, which is used to adjust the components of system operation cost and control cost in the cost function. It can be seen that this cost function includes the control cost of all players. For the i-th player, its own control cost is Therefore, by minimizing this cost function, each player can drive the state x(t) to a stable value while optimizing the control cost.
[0091] Step S103, obtain the optimal control strategy. Each player hopes to obtain the optimal cost function V i * (x) to make the whole system reach Nash equilibrium, that is, the best control result in theory. Therefore, based on the optimal cost function, according to the Nash equilibrium condition and Bellman optimality principle, the optimal control strategy of the player is as follows:
[0092]
[0093] in is the optimal cost function V i * The optimal control strategy of the constructed non-zero-sum differential game model is
[0094] After obtaining the optimal control model, we can continue to Figure 1 Step S20 in the above example constructs an evaluation neural network to achieve the optimal cost function V i * (x) is an approximation of
[0095] S20: Construct an evaluation neural network and use experience replay to obtain an event-based control strategy;
[0096] In this embodiment, Figure 3 The steps in construct the evaluation neural network and obtain the event-based control strategy, such as Figure 3 As shown, step S20 mainly includes:
[0097] Step S201, constructing an evaluation neural network. In this embodiment, a three-layer feedforward neural network is used to construct an evaluation neural network to approximate the optimal cost function online. The output value of the evaluation network Will be used to calculate the control strategy, the output value is determined by the evaluation weight And the activation function Characterization is as follows:
[0098]
[0099] in is the partial derivative of the approximate cost function with respect to x. Then a control strategy is given Use the Newton-Leibniz formula to calculate the cumulative reward of the player in a fixed interval and obtain the reinforcement signal as follows:
[0100]
[0101] Where T>0 is a fixed integration interval. At the same time, based on the difference of the activation function at different integration times, the regression vector is calculated as follows:
[0102]
[0103] Step S202: Establish a data stack area and obtain empirical data. In order to make full use of the data generated by learning and improve decision-making efficiency, establish a data stack area Store empirical data, where L represents the length of the data stack. The empirical data mainly includes the enhanced signal and the regression vector at different times t q The calculated values are expressed as follows:
[0104]
[0105]
[0106] It can be seen that these data are actually generated by the system and will be used many times. However, not all data can be stored. The design experience data stacking standards are as follows:
[0107]
[0108] where θ i>0 is a small constant. Obviously, new data will be stored in the stack only when the data is very different. The replacement method of new and old data is designed as a "push stack" method, that is, the data in the stack is stored in chronological order, and the new data will squeeze out the old data and enter the stack, and the cycle will continue.
[0109] Step S203, obtain the adaptive update rule of the evaluation network. After obtaining the empirical data, combined with the real-time data, the update rule of the evaluation network is obtained by using the gradient descent adaptive method as follows:
[0110]
[0111] Among them l i >0 is the learning rate of the i-th evaluation network. As can be seen from formula (9), the evaluation network can realize the adaptive update of the evaluation weight by using real-time data and experience data; once the new evaluation weight is obtained, the cost function can be updated by formula (4).
[0112] Step S204: Get the event-based control strategy. And the optimal control strategy shown in formula (3) At the time of event triggering Calculate event-based control strategies as follows:
[0113]
[0114] This control strategy is the actual decision result of each player under the event trigger mechanism, and is also the actual output of the controller.
[0115] The above obtained event-based control strategy, but the event triggering time Still can't get it, so continue through Figure 1 In step S30, an adaptive event triggering rule is obtained, and the rule is used to calculate the event triggering time.
[0116] S30: Obtaining an adaptive event triggering rule based on distributed triggering;
[0117] In this embodiment, Figure 4 The steps in get the adaptive event triggering rules, such as Figure 4 As shown, step S30 mainly includes:
[0118] Step S301, calculate the local sampling error. In order to cater to the individual needs of the players, the embodiment of the present disclosure adopts a distributed event trigger mechanism. Under the distributed trigger, each player needs to be equipped with a local event trigger to independently perform state sampling and data transmission. Assume that the event triggering time generated by the i-th local event trigger is Each player samples the state at the corresponding triggering moment. And the sampling state sent to the controller to calculate event-based control strategies Then, according to the recorded sampling status Calculate the local sampling error
[0119]
[0120] This local sampling error will be used to design event triggering rules.
[0121] Step S302: Establish a Lyapunov function based on a local cost function. In order to ensure the optimization and stability of the triggering time, establish a Lyapunov function L based on a local cost function. i (t) are as follows:
[0122]
[0123] in is the weight error of the evaluation network. This function L i (t) is a criterion that combines stability, convergence, and event triggering. To derive this function, according to the Lyapunov stability theory, the following stability conditions must be met:
[0124]
[0125] where ρ i ∈(0,1) is a conservative parameter.
[0126] Step S303, obtain the adaptive event triggering rule. According to the above stability conditions, reverse design is performed to obtain the adaptive event triggering rule as follows:
[0127]
[0128] in are two constant parameters of the design. The trigger rule describes when to generate the required event trigger moment, that is, at the current trigger moment Get the next trigger time based on The initial triggering moment can be considered as Obviously, the determination of the next triggering time depends on the system operation cost x T Q i x. Control strategy at the current triggering moment Sampling Error and the evaluation weight at the current moment
[0129] After obtaining the control strategy and trigger rules, you can continue to Figure 1 Step S40 in the above method applies them to the autopilot system.
[0130] S40: applying the obtained control strategy and event triggering rules to the multi-controller system;
[0131] In this embodiment, Figure 5 The steps in the application of control strategies and triggering rules to multi-controller systems are as follows: Figure 5 As shown, step S40 mainly includes:
[0132] Step S401, loading the control strategy and triggering rules into the multi-controller system. In some embodiments, the control strategy and triggering rules can be loaded into the multi-controller system in the form of computer programs and mathematical symbols. At this time, each controller Update control strategy Complete a new round of update iteration and learning calculation.
[0133] Step S402: Implement event-triggered learning control. Under the distributed trigger setting, the learning of each control loop presents independent characteristics, and the control strategy is no longer updated synchronously. At this time, each controller uses local triggers and evaluation networks to perform distributed reinforcement learning processes, realize event-triggered learning control, and finally jointly stabilize the entire system.
[0134] Example 3
[0135] The feasibility of the solutions in Embodiments 1 and 2 is verified below in combination with specific experimental data and examples. In this embodiment, a typical multi-controller system, namely, an aircraft autopilot system, is considered, as described below:
[0136] first, Figure 6 The main state variables and actuators of the autopilot system in this embodiment are shown. In the stable cruising process of this autopilot system, there are mainly two actuators that play a role: the rudder and the aileron. There are three main state variables involved in lateral motion: the roll angle φ, the yaw angle ψ and the sideslip angle β. During the cruising process, the sideslip angle β(t) (unit: degree, deg), the roll angle φ(t) (unit: degree, deg), the roll rate R φ(t) (unit: degree / second, deg / s), yaw rate R ψ (t) (unit: degree / second, deg / s). Then, according to Figure 1 The steps S10-S40 shown are continued.
[0137] According to step S10, the rudder and aileron are regarded as two independent players, and the output signals of the two actuators are the rudder deflection angle δ R (t) (unit: degree / second, deg / s) and aileron deflection angle δ A (t) (unit: degree / second, deg / s), that is, the control strategy of the player is u1(t) = δ R (t),u2(t)=δ A (t); At the same time, the state of this two-person non-zero-sum differential game model is expressed as x(t) = [x1, x2, x3, x4] T ; x1=β, x2=φ, x3=R φ ,x4=R ψ ; For two players, the cost function is configured as Q1 = diag{10,1,1,1}, Q2 = diag{1,5,1,1}, R 11 =1,R 12 =0.25,R 21 =1,R 22 =0.25.
[0138] According to step S20, in this embodiment, 10 nodes are configured in the hidden layer of the evaluation network, and the activation function is set to
[0139]
[0140] To calculate the reinforcement signal, the integration interval is T = 0.05s; the parameter configuration of the experience stack is θ1 = θ2 = 0.2, L = 12; in addition, the evaluation network learning rate is l1 = l2 = 0.01.
[0141] According to step S30, the two players are equipped with two independent event triggers, and the parameters of the corresponding adaptive event triggering rules are configured as ρ1=ρ2=0.3, δ1=1, δ2=0.25,
[0142] According to step S40, the autopilot system calculates the event-based control strategy according to formula (10) and calculates the new event triggering time according to formula (14). Each player implements distributed reinforcement learning through evaluation network and distributed triggering, and then implements event-triggered learning control.
[0143] Figure 7It is the state trajectory and event-based control signal of the aircraft autopilot system shown in this embodiment. It can be found that the four state variables of the aircraft are completely stable in about 50 seconds after the initial disturbance, and the control signal is "step-shaped", indicating that the control strategy is updated non-periodically. In this process, controller 1 needs to be updated 179 times and controller 2 only needs to be updated 83 times. Compared with 1000 times of traditional time-triggered control, the controllers in the disclosed method reduce data transmission and control calculations by approximately 82% and 90%, respectively, effectively saving communication resources and thus improving communication-restricted conditions.
[0144] Example 4
[0145] Next, based on Example 3, a classic event-triggered control method is considered, and the robustness of the event-triggered learning control method for saving communication and computing resources disclosed in the present invention is verified by comparison, as described below for details:
[0146] In this embodiment, the inventive method is implemented according to the steps in Embodiment 3. In addition, a classic event trigger control method is considered, which is recorded as "classic method". In this embodiment, it is assumed that the second event trigger has a communication failure within [10, 20] s, during which player 2 cannot receive the sampled data triggered by the event.
[0147] Due to the use of distributed triggering, the invented method maintains a certain margin, that is, the event trigger of player 1 is still working normally at this time, so the system state will not produce a large divergence; in contrast, the classical method will cause a large state divergence behavior during the failure of the event trigger, and this excessive divergence may endanger the safety of the aircraft system.
[0148] In view of the results of Example 3 and Example 4, the beneficial effect of the event-triggered learning control method for saving communication and computing resources disclosed in the embodiment of the present invention is true.
[0149] Example 5
[0150] This embodiment provides a non-transitory computer-readable storage medium (including but not limited to a disk storage, an optical storage, etc.), which stores:
[0151] The computer program includes an adaptive updating rule for evaluating the network, an adaptive event triggering rule based on distributed triggering, and related computer codes for implementing the method.
[0152] Control instructions, mainly the control strategy of each controller.
[0153] When the program and instructions are executed by each controller, the multi-controller system can implement the method described in the first aspect of the present disclosure, which will not be elaborated here.
[0154] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as methods, prototype systems or computer program products. Therefore, the present disclosure can take the form of a complete software embodiment or a combination of software and hardware embodiments.
[0155] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A multi-controller system stability control method based on event triggering, characterized in that: The method comprises the following steps: Establishing an optimal control model for multi-controller systems using non-zero-sum differential games Through the Bellman optimal method and the controller cost function V i (x) Calculate and output the optimal control strategy for the optimal control model Wherein: The optimal control strategy output process:
101. Establish the optimal control model of non-zero-sum differential game for multi-control systems according to the following formula: Where: f(x(t)) is the internal dynamic information of the system, N represents the number of players, i.e., controllers; for the i-th player, g i (x(t)) is the control coupling matrix, u i (t) is the control strategy, i.e., the output of the i-th controller; 102. Set the quadratic cost function V of the control cost for each controller according to the following formula: i (x) Among them, Q i ,R ij is a positive and symmetric constant matrix, which is used to adjust the components of system operation cost and control cost in the cost function. It can be seen that this cost function includes the control cost of all controllers. For the i-th controller, its own control cost is 103. Based on the optimal cost function, according to the Nash equilibrium condition and Bellman optimality principle, the optimal control strategy of the player is as follows: in: is the optimal cost function V i * The optimal control strategy of the constructed non-zero-sum differential game model is Construct an evaluation neural network and use experience replay to evaluate the event triggering time of the optimal control strategy Control output event-based control strategy Construct a Lyapunov function based on the local cost function and establish the adaptive event triggering rules of the controller; The control strategy of the event and the adaptive event triggering rule are applied to a multi-controller system.
2. The event-triggered multi-controller system stability control method according to claim 1, characterized in that: The event-based control strategy output process:
201. Use a three-layer feedforward neural network to construct an evaluation neural network to approximate the optimal cost function online and evaluate the output value of the network Will be used to calculate the control strategy, the output value is determined by the evaluation weight And the activation function Characterization is as follows: in: is an approximate cost function Partial derivatives with respect to x; 202. Given a control strategy The Newton-Leibniz formula is used to calculate the reward accumulation of the controller in a fixed interval to obtain the reinforcement signal as follows: Where: T>0 is a fixed integration interval; at the same time, based on the difference of the activation function at different integration times, the regression vector is calculated as follows:
203. Establish data stack area Store empirical data, where L represents the length of the data stack. The empirical data mainly includes the enhanced signal and the regression vector at different times t q The calculated values are expressed as follows: The standards for stacking empirical data are as follows: Where: θ i >0 is a small constant; 204. Using Evaluation Weights and optimal control strategy At the time of event triggering Compute outputs for event-based control strategies as follows:
3. The event-triggered multi-controller system stability control method according to claim 1, characterized in that: The adaptive event triggering rule establishment process:
301. Calculate the local sampling error for each controller using the following formula:
302. Establish the following Lyapunov function based on the local cost function; in: is the weight error of the evaluation network. This function L i (t) is a criterion that combines stability, convergence, and event triggering; 303. Calculate the stability condition based on the Lyapunov function and obtain the following adaptive event triggering rules: in: are two constant parameters. The trigger rule describes when to generate the required event trigger time, that is, at the current trigger time Get the next trigger time based on The initial triggering moment can be considered as The determination of the next triggering time depends on the system operation cost x T Q i x. Control strategy at the current triggering moment Sampling Error and the evaluation weight at the current moment 4. A non-transitory computer-readable storage medium storing computer programs and control instructions, which enable a multi-controller system to implement the method of claims 1 to 3 when the programs and instructions are executed by each controller.
Citation Information
Patent Citations
Optimized control method for event trigger of discrete time nonlinear system
CN111812973A
Differential game collaborative guidance method based on event triggering
CN113325866A
Event-triggered distributed self-learning cooperative game interception guidance method
CN113341727A
Autonomous trolley cruise control method and device
CN114756025A
Adaptive event-triggered control method for permanent magnet synchronous generator coupled chaotic network system
GB202111727D0
