Engineering system fault-tolerant tracking control method based on data driving

Abstract

The invention discloses an engineering system fault-tolerant tracking control method based on data driving, and belongs to the technical field of engineering control methods. Comprising the following steps: S1, establishing an engineering system equivalence model under a fault, and defining a performance index function and a corresponding value function so as to convert an optimal control problem into a zero-sum game problem; s2, aiming at the problem that the state of the engineering system is unmeasurable, reconstructing the state by utilizing historical input and output data so as to design and solve an optimal control strategy; and S3, introducing and defining a Q function, designing a controller gain and a disturbance gain # imgabs0 #, and optimizing the # imgabs1 # in combination with a designed model-free non-strategy Q-learning output feedback algorithm to obtain an optimal controller. Through an injection molding industrial process and a depth control simulation experiment of an autonomous underwater vehicle (AUV), convergence of the algorithm and an excellent tracking control effect of a designed controller are verified.

Description

Technical Field

[0001] The present invention relates to an advanced control method, and in particular to a data-driven engineering system fault-tolerant tracking control method for problems such as unmeasurable states, external interference and multiple faults. Background Art

[0002] In modern industrial and automation systems, there is an increasing demand for precise and reliable control. However, existing technologies are unable to cope with some key challenges. Specifically, these challenges include the unmeasurability of system states, external disturbances that are difficult to predict and control, and multiple failures that may occur in the system, which seriously affect the stability and performance of the system, especially in complex environments such as underwater operations, chemical industry, and resource mining. First, the unmeasurable state makes it impossible to directly obtain accurate information about the internal state of the system, which is a major obstacle for control strategies that rely on accurate state feedback; second, external disturbances, such as environmental changes or unexpected changes in operating conditions, may cause system performance to degrade or even cause safety accidents; finally, multiple failure problems, including failures of sensors and actuators, further increase the instability of the system, because these failures may lead to the loss or error of control signals, triggering a chain reaction.

[0003] The existing technology still has many deficiencies in solving the above problems, specifically the following points:

[0004] (1) It is difficult to obtain an accurate system model in practical applications;

[0005] (2) Model-based state feedback fault-tolerant control is not very applicable;

[0006] (3) Traditional policy learning algorithms still have great deficiencies in terms of convergence. At the same time, although the existing non-policy Q-learning algorithms based on input-output data can alleviate the impact of monitoring noise to a certain extent, they often ignore the impact of equipment failures on system performance. In particular, when actuators and sensors fail, these algorithms may not be able to provide effective control strategies. Summary of the invention

[0007] In view of the above shortcomings, the research objective of the present invention is to propose a data-driven fault-tolerant tracking control method for engineering systems, which has the following advantages: (1) It can handle external interference and multiple fault problems without relying on the measurement of the state of the engineering system; (2) Compared with the traditional model-based method, the data-driven fault-tolerant control method proposed in the present invention has better applicability, flexibility and robustness, especially in dealing with complex engineering systems, coping with uncertainties and multiple faults in engineering systems. It is more effective; (3) When a fault occurs in the engineering system, the controller in the present invention will quickly reorganize or reconstruct the existing control law according to the type of fault, thereby buying a lot of time for fault-tolerant control, and then controlling a wider range of faults, ensuring the stability of the system and good tracking performance.

[0008] To achieve this object, the present invention adopts the following technical solutions:

[0009] The engineering system fault-tolerant tracking control method based on input-output data reconstruction drive includes the following steps:

[0010] S1. Establish an equivalent model of the engineering system under fault conditions, define the performance index function and the corresponding value function, and thus transform the optimal control problem into a zero-sum game problem;

[0011] S2. For the problem that the state of the engineering system is unmeasurable, the state is reconstructed using historical input and output data to design and solve the optimal control strategy;

[0012] S3. Introduce and define the Q function, and calculate the controller gain and disturbance gain. Design and combine the designed model-free non-strategy Q-learning output feedback algorithm to Optimize and obtain the optimal controller.

[0013] Furthermore, in S1, an engineering system fault equivalence model is established and the problem is transformed. The specific steps are as follows:

[0014] S11. For the engineering system model with multiple fault effects, its equivalent system model is established as follows:

[0015] First, establish the following linear discrete engineering system:

[0016]

[0017] in, and are the system state, control input, disturbance input signal and system output at time k respectively. {A, B, C, D} is the system matrix of suitable dimension, When the actuator and sensor fail during the production process, the actual input and output signals can be expressed as: in and Different values ​​of correspond to different operating conditions of actuators and sensors: ① hour, At this time, the equipment is operating normally; ② and ③ When the equipment fails partially; When , the equipment fails completely. Therefore, the engineering system failure model can be obtained as follows:

[0018]

[0019] For the above engineering system failure model, the following variables are defined:

[0020]

[0021] Where, Δs k+1 ,Δd k , They are the state error, disturbance error and output tracking error of the engineering system under fault conditions, y r To set the expected output tracking value, is the iterative update law designed at time k. Combining (2) and (3), we can get the equivalent model of the engineering system under fault:

[0022]

[0023] in, {A * ,B * ,C * ,D *} is a suitable-dimensional matrix. At this time, are the system state, control input, disturbance input and system output of the engineering system at time k under fault. At the same time, hour,

[0024] S12. Define the performance index function and the corresponding value function according to the optimal control theory:

[0025]

[0026] When the optimal strategy exists, the system will have an L2 gain less than or equal to γ ​​(γ ≥ 0), that is, it satisfies in, Q, R are the positive matrices to be determined, representing the output Control input g k The weight matrix of .

[0027] S13, for external interference Δd k Uncontrollable problems, combined with Minimax theory, convert the problem into a linear quadratic zero-sum game problem:

[0028] Put the control input g k and external disturbance Δd k As players 1 and 2 respectively, the H∞ problem can be regarded as solving the optimal control input that minimizes the value function. and the worst external perturbation that maximizes the value function The problem of finding saddle points So that:

[0029]

[0030] Then the value function has the following quadratic form:

[0031]

[0032] Here, P is a positive definite matrix.

[0033] Furthermore, in S2, the state is reconstructed using historical input and output data, and an optimal control strategy is designed. The specific steps are as follows:

[0034] S21. To solve the problem that the state of the engineering system is unmeasurable, the state is reconstructed using historical input and output data:

[0035] When A is non-singular and (A, C) is observable, the output of the equivalent model (4) of the engineering system under fault is It can be expressed as:

[0036]

[0037] It can be expressed as follows:

[0038]

[0039] in

[0040]

[0041] Let L = [(C * (A * ) -N ) T … (C * (A * ) -2 ) T (C * (A * ) -1 ) T ]T ,but Let Z = [L + L + E g L + E Δd ], then for The original state variables of the equivalent model (4) of the engineering system under fault are reconstructed into the following form:

[0042]

[0043] in,

[0044]

[0045] S22. For the equivalent model of the engineering system under fault (4), combined with the above state reconstruction form (7), the following optimal control input and worst disturbance input are designed based on data drive:

[0046]

[0047] in

[0048]

[0049] Furthermore, in S3, the Q function is introduced and defined for Design and combine the designed model-free non-strategy Q-learning output feedback algorithm to Optimize and get the optimal controller, as follows:

[0050] S31. Combined with the performance index (5), the Q-function is introduced and defined as follows:

[0051]

[0052] but:

[0053] S32, controller gain and disturbance gain The design is as follows:

[0054] For stable strategies The Q-function has the following quadratic form:

[0055]

[0056] in

[0057]

[0058] Combined with (8), we can know that:

[0059]

[0060] As can be seen from the above formula, the data-driven model-free non-strategy Q-learning algorithm actually calculates the controller gain and disturbance gain by solving the parameters of the matrix L. Design.

[0061] S33: In order to solve the contradiction between exploring data and using data in Q-learning, we introduce the target strategy and The control strategy is continuously optimized as follows:

[0062] First, combined with the state variable reconstruction formula (7), the equivalent model of the engineering system under fault (4) is equivalent to the following expression:

[0063]

[0064] in, Therefore, combining (6) and (9), the following equation holds:

[0065]

[0066] in:

[0067]

[0068] Therefore, combined with (10a), the above formula (12) can be transformed into:

[0069]

[0070] Using the Kronecker product, the above formula can be rewritten as:

[0071]

[0072] Then, the controller gain element can be solved by the least squares method:

[0073]

[0074] At this time S j+1 can be expressed as:

[0075]

[0076] in,

[0077]

[0078] Therefore, combining formula (11) we can get:

[0079]

[0080] In the result of formula (13), the model-free non-strategy Q-learning output feedback algorithm is used to Optimize and get the optimal controller. At the same time, the controller gain and disturbance gain The target strategy is obtained based on the input and output data, which is different from the traditional model-based state feedback fault-tolerant control. and After that, the data generated by the behavior strategy is continuously optimized and learned. Under the action of these two strategies, the comprehensiveness of the data and the global optimality of the learning can be guaranteed. At the same time, by introducing detection noise into the behavior strategy of the non-strategy Q-learning algorithm, and adding detection noise, the Q function solution obtained is the same as that without adding detection noise, which proves that the algorithm is bias-free for detection noise, ensuring that under different initial values ​​and detection noise, the controller solved by the algorithm can eventually converge to the optimal value and maintain stable tracking performance.

[0081] Beneficial effects of the present invention:

[0082] 1. The core of the present invention is to develop a data-driven fault-tolerant tracking control method for engineering systems, which can not only deal with the problem that the state of the engineering system cannot be measured, but also effectively deal with external interference and equipment failures, ensuring that the engineering system can maintain stable operation and good performance even under these adverse conditions. In addition, the optimal controller obtained by this method is bias-free for detection noise, which makes it more reliable and effective than traditional policy learning algorithms in practical applications.

[0083] 2. The present invention cleverly introduces variables such as tracking error, constructs an equivalent model of the fault system, and combines the Minimax theory to transform the problem into a zero-sum game problem. Based on this, it makes full use of historical input and output data to reconstruct the difficult-to-measure system state. Then, the key Q function is introduced, and a model-free non-strategy Q-learning algorithm is innovatively proposed to effectively solve the optimal controller. At the same time, the algorithm's unbiasedness in detecting noise is explained, overcoming many shortcomings of strategy learning, while ensuring the continuity and stability of the injection molding process and autonomous underwater robot (AUV).

[0084] 3. The method of the present invention does not rely on the precise mathematical model of the system, has a simple and clear structure, is easy to operate, and has good control performance. Through this method, not only the reliability and robustness of the engineering system are improved, but also the progress of industrial automation and marine resource development technology is further promoted. This has important practical significance for coping with the uncertainty of complex industrial environment and marine environment, and ensuring the safety and efficiency of industrial production and underwater operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0086] Figure 2 is the system output tracking trajectory under five situations;

[0087] Figure 3 For Case 1 The convergence curve of

[0088] Figure 4 For Case 2 The convergence curve of

[0089] Figure 5 For Case 2 Parameter convergence curve of ;

[0090] Figure 6 For Case 2 Parameter convergence curve of ;

[0091] Figure 7 For Case 3 and Case 4 under different Noises Convergence curve comparison;

[0092] Figure 8 is the value of Case 5 under different initial values Convergence curve;

[0093] Fig. 9 Schematic diagram of AUV diving depth control for different Cases;

[0094] Fig.10 For Case B and Case C The convergence curve of

[0095] Fig.11 For Case D and Case E The convergence curve of

[0096] Fig.12 Schematic diagram of the depth control error of AUV under four fault conditions; DETAILED DESCRIPTION

[0097] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0098] The data-driven engineering system fault-tolerant tracking control method includes the following steps:

[0099] S1. Establish an equivalent model of the engineering system under fault conditions, define the performance index function and the corresponding value function, and thus transform the optimal control problem into a zero-sum game problem;

[0100] S2. For the problem that the state of the engineering system is unmeasurable, the state is reconstructed using historical input and output data to design and solve the optimal control strategy;

[0101] S3. Introduce and define the Q function, and calculate the controller gain and disturbance gain. Design and combine the designed model-free non-strategy Q-learning output feedback algorithm to Optimize and obtain the optimal controller.

[0102] The present invention proposes a data-driven fault-tolerant tracking control method for engineering systems. The core of the method is to cleverly introduce variables such as tracking errors to construct an equivalent model of the engineering system under faults in view of the complex situations such as unmeasurable state, external interference and multiple faults in the engineering system. At the same time, the problem is transformed into a zero-sum game problem in combination with the Minimax theory. On this basis, the historical input and output data are fully utilized to reconstruct the difficult-to-measure system state. Then the key Q function is introduced, and a model-free non-strategy Q-learning algorithm is innovatively proposed to effectively solve the optimal controller, and the algorithm is explained in terms of the unbiasedness in detecting noise, overcoming many shortcomings of strategy learning. The method does not rely on the state of the engineering system and has significant advantages. Through the simulation experiments of injection speed control in the injection molding industry and depth control of autonomous underwater robots, the convergence of the algorithm and the excellent tracking control effect of the designed controller are effectively verified, thereby ensuring that the engineering system can still maintain stable operation in the face of various faults and achieve good tracking performance, providing a valuable solution to solve the control problems in the actual production process.

[0103] In S1, an engineering system fault equivalence model is established and the problem is transformed. The specific steps are as follows:

[0104] S11. For the engineering system model with multiple fault effects, its equivalent system model is established as follows:

[0105] First, establish the following linear discrete engineering system:

[0106]

[0107] in, and are the system state, control input, disturbance input signal and system output at time k respectively. {A, B, C, D} is the system matrix of suitable dimension, When the actuator and sensor fail during the production process, the actual input and output signals can be expressed as: in and Different values ​​of correspond to different operating conditions of actuators and sensors: ① hour, At this time, the equipment is operating normally; ② and ③ When the equipment fails partially; When , the equipment fails completely. Therefore, the engineering system failure model can be obtained as follows:

[0108]

[0109] For the above engineering system failure model, the following variables are defined:

[0110]

[0111] Where, Δs k+1 ,Δd k , They are the state error, disturbance error and output tracking error of the engineering system under fault conditions, y r To set the expected output tracking value, is the iterative update law designed at time k. Combining (2) and (3), we can get the equivalent model of the engineering system under fault:

[0112]

[0113] in, {A * ,B * ,C * ,D *} is a suitable-dimensional matrix. At this time, are the system state, control input, disturbance input and system output of the engineering system at time k under fault. At the same time, hour,

[0114] S12. Define the performance index function and the corresponding value function according to the optimal control theory:

[0115]

[0116] When the optimal strategy exists, the system will have an L2 gain less than or equal to γ ​​(γ ≥ 0), that is, it satisfies in, Q, R are the positive matrices to be determined, representing the output Control input g k The weight matrix of .

[0117] S13, for external interference Δd k Uncontrollable problems, combined with Minimax theory, convert the problem into a linear quadratic zero-sum game problem:

[0118] Put the control input g k and external disturbance Δd k As players 1 and 2 respectively, the H∞ problem can be regarded as solving the optimal control input that minimizes the value function. and the worst external perturbation that maximizes the value function The problem of finding saddle points So that:

[0119]

[0120] Then the value function has the following quadratic form:

[0121]

[0122] Here, P is a positive definite matrix.

[0123] For actuator and sensor failures, the engineering system model under fault is established, the iterative learning control law is determined, and after defining multiple errors, the equivalent model of the engineering system under fault is constructed. Then, by defining the performance indicator function and the value function, it is finally transformed into a zero-sum game problem in combination with the Minimax theory. The advantages of this design are that it fully considers the failure situation of the engineering system to make the model more realistic; it is constructed and transformed based on a sufficient theoretical basis to ensure a rigorous and scientific process; the precise error definition and performance indicator quantification facilitate the evaluation and optimization of the engineering system; and the transformation into a zero-sum game problem provides an effective idea and method for the subsequent solution of the optimal controller, which helps to improve the stability and control performance of the engineering system.

[0124] Among them, in S2, the state is reconstructed using historical input and output data, and the optimal control strategy is designed. The specific steps are as follows:

[0125] S21. To solve the problem that the state of the engineering system is unmeasurable, the state is reconstructed using historical input and output data:

[0126] When A is non-singular and (A, C) is observable, the output of the equivalent model (4) of the engineering system under fault is It can be expressed as:

[0127]

[0128] It can be expressed as follows:

[0129]

[0130] in

[0131]

[0132] Let L = [(C * (A * ) -N ) T … (C * (A * ) -2 ) T (C * (A * ) -1 ) T ] T ,but Let Z = [L + L + E g L + E Δd ], then for The original state variables of the equivalent model (4) of the engineering system under fault are reconstructed into the following form:

[0133]

[0134] in,

[0135]

[0136] S22. For the equivalent model of the engineering system under fault (4), combined with the above state reconstruction form (7), the following optimal control input and worst disturbance input are designed based on data drive:

[0137]

[0138] in

[0139]

[0140] By using historical input and output data to reconstruct unpredictable states, the problem of obtaining system states is solved, providing a key foundation for the implementation of subsequent control strategies. The designed optimal control input and worst-case disturbance input help improve the system's ability to respond to faults and enhance the system's stability and robustness. This method has strong logic and theoretical basis.

[0141] Among them, in S3, the Q function is introduced and defined for Design and combine the designed model-free non-strategy Q-learning output feedback algorithm to Optimize and obtain the optimal controller.

[0142] The details are as follows:

[0143] S31. Combined with the performance index (5), the Q-function is introduced and defined as follows:

[0144]

[0145] but:

[0146] S32, controller gain and disturbance gain The design is as follows:

[0147] For stable strategies The Q-function has the following quadratic form:

[0148]

[0149] in

[0150]

[0151] Combined with (8), we can know that:

[0152]

[0153] As can be seen from the above formula, the data-driven model-free non-strategy Q-learning algorithm actually calculates the controller gain and disturbance gain by solving the parameters of the matrix L. Design.

[0154] S33: In order to solve the contradiction between exploring data and using data in Q-learning, we introduce the target strategy and The control strategy is continuously optimized as follows:

[0155] First, combined with the state variable reconstruction formula (7), the equivalent model of the engineering system under fault (4) is equivalent to the following expression:

[0156]

[0157] in, Therefore, combining (6) and (9), the following equation holds:

[0158]

[0159] in:

[0160]

[0161] Therefore, combined with (10a), the above formula (12) can be transformed into:

[0162]

[0163] Using the Kronecker product, the above formula can be rewritten as:

[0164]

[0165] Then, the controller gain element can be solved by the least squares method:

[0166]

[0167] At this time S j+1 can be expressed as:

[0168]

[0169] in,

[0170]

[0171]

[0172] Therefore, combining formula (11) we can get:

[0173]

[0174] By introducing the Q function, a model-free non-strategy Q-learning algorithm is proposed to solve the optimal controller and solve the system control strategy optimization problem. At the same time, the target strategy is introduced and The contradiction between exploring and utilizing data is balanced, and the data generated by the behavioral strategy is continuously optimized and learned. Under the action of these two strategies, the comprehensiveness of data and the global optimality of learning are ensured; at the same time, by introducing detection noise into the behavioral strategy of the non-strategy Q-learning algorithm, and adding detection noise, the Q function solution obtained is the same as that without adding detection noise, which proves that the algorithm is bias-free to detection noise, ensuring that under the influence of different initial values ​​and detection noise, the controller solved by the algorithm can eventually converge to the optimal value and maintain stable tracking performance. Its advantage is that compared with the traditional model-based method, the controller design based on data-driven is highly innovative, breaking through the limitations of traditional strategy learning, and realizing the optimization solution of the controller through rigorous theoretical derivation and algorithm design, enhancing the adaptability and stability of the system, and effectively improving the control performance of the system in the face of complex environments and uncertainties, ensuring the reliable operation of the system, and having higher reliability and effectiveness in practical applications.

[0175] The present invention will be further described below in conjunction with specific examples.

[0176] In the process of injection molding industry and autonomous underwater vehicle (AUV) exploration, the controller's fault control effect directly affects its work efficiency. The performance of the controller is crucial to the operation of the system. When the system fails, such as the actuator measurement value is too high or too low, the thruster fails, etc., a corresponding controller needs to be designed to compensate for the fault.

[0177] Taking the injection speed control in the injection molding industrial process and the depth control simulation experiment of the autonomous underwater vehicle (AUV) as examples, the feasibility and effectiveness of the model-free non-strategy Q-learning algorithm designed according to historical input and output data are verified.

[0178] The following is a specific implementation process of injection speed control in the injection molding industry:

[0179] This section uses the injection speed control simulation experiment in the injection molding industrial process to verify the effectiveness of the designed method. This section will illustrate the effectiveness of the proposed method through numerical experiments in the following five cases:

[0180] Case 1: When , the corresponding actuator and sensor operate normally;

[0181] Case 2: When , the value measured by the actuator is higher than the actual value, and the value measured by the sensor is lower than the actual value;

[0182] Case 3: When , the value measured by the actuator is lower than the actual value, and the value measured by the sensor is higher than the actual value;

[0183] Case 4: When , the values ​​measured by the actuator and sensor are higher than the actual values;

[0184] Case 5: When , the values ​​measured by the actuator and sensor are lower than the actual values.

[0185] According to step S1, the state space model of the injection molding process can be established as:

[0186]

[0187] in, is a real number within a certain range, and the expected output tracking value y in (9) is set r =10, the relevant parameters in performance index (6) are: γ = 2.2, Q = diag (5, 0, 1), R = 0.01.

[0188] According to step S2, different strategy initial values ​​are given In the designed algorithm, the above five situations After sampling and learning the input and output data at historical moments, they can all converge to the corresponding optimal value.

[0189] Case 1:

[0190]

[0191] Case 2:

[0192]

[0193] Case 3:

[0194]

[0195]

[0196] Case 4:

[0197]

[0198] Case 5:

[0199]

[0200] Figure 2 The system output tracking trajectory diagram under 5 different situations is shown. It can be seen from the figure that in the face of different equipment operation situations, as the data is generated and collected, the controller solved by the designed method can make the system output track to the set value, indicating that the proposed algorithm has good tracking control performance.

[0201] (1) For Case 1 and Case 2, set the random initial strategy:

[0202]

[0203] Figure 3-4 The control strategy for each iteration is The norm of the difference from the optimal policy convergence curve is intended to show the convergence of the controller gains.

[0204] As can be seen from the figure, since the initial strategy is randomly generated, the difference norm of the initial iteration is large. Figure 3 As shown in , under normal circumstances, the control strategy of the system can converge to the optimal value after the third iteration, and the convergence speed is fast; Figure 4As shown in Figure 2, under fault scenario Case 2, the system can gradually converge to the optimal value after the 6th iteration. Figure 5-6 It is known that although Although a jump occurred in the previous iteration cycle, each component can still converge to the optimal value in the sixth iteration cycle. It can be seen that the proposed algorithm effectively enables the control strategy to converge to the optimal value quickly, reflecting its superior control performance.

[0205] (2) Next, we add different detection noises to the behavior strategies of Case 3 and Case 4, respectively, and discuss the corresponding The convergence of the system is shown to illustrate that the solution of the optimal strategy is not affected by the detection noise.

[0206] set up The initial value is a random number in [-5,5]:

[0207]

[0208] ①Set the detection noise Noise 1 to:

[0209] e 1k =0.8*e -0.01k (7sin 2 (k)+sin(2k)+sin(10k)-sin(2k))

[0210] e 2k =100*e -0.01k (sin(k)+sin(0.6k)-sin(1.3k))

[0211] ②Set the detection noise Noise 2 to:

[0212] e 1k =e -0.01k (7sin 2 (k)+sin(2k)+sin(10k)-sin(2k))

[0213] e 2k =110*e -0.01k (sin(k)+sin(0.6k)-sin(1.3k))

[0214] Figure 7 (a) and (b) show the system after adding detection noise Noise 1 and Noise 2 in Case 3. As shown in the figure, the norm convergence curve of the difference from the optimal strategy fluctuates greatly in the first four iterations, but starting from the third iteration, the convergence norm of the difference decreases significantly and finally converges to the optimal value in the sixth iteration. Figure 7 (c) and (d) show the controller gain after adding different detection noises in Case 4. As shown in the figure, as the detection noise increases, the convergence curve has large fluctuations in the first four iterations, but as the iteration period increases, the curve can converge to 0 around the 7th iteration, and the strategy solved at this time is the optimal strategy. It can be seen that the detection noise does not affect the solution of the optimal strategy, proving that the designed algorithm is unbiased with respect to the detection noise.

[0215] (3) Then, under the same setting of detection noise Noise 2, different control strategy initial values ​​are set for Case 5 to verify that the solution of the optimal controller is not affected by the strategy initial value.

[0216] Noise 2:

[0217] e 1k =e -0.01k (7sin 2 (k)+sin(2k)+sin(10k)-sin(2k))

[0218] e 2k =110*e -0.01k (sin(k)+sin(0.6k)-sin(1.3k))

[0219] ① Set the initial value of the strategy for Case 5 to the zero vector:

[0220]

[0221] ② Set the initial value of the strategy for Case 5 to a randomly generated vector:

[0222]

[0223]

[0224] Figure 8 .(a),(b), Figure 8 (c) and (d) are the convergence of the control strategy of Case 5 under two different initial values. As shown in the figure, under the same fault degree, the initial value of the control strategy affects the convergence speed, resulting in the influence of several iteration cycles, but has no effect on its convergence result, and finally converges to the optimal value.

[0225] In summary, the control strategy designed by the proposed algorithm can eventually converge to the optimal solution under the influence of the initial value of the control strategy and the detection noise, indicating that it has good control performance.

[0226] The following is a specific implementation process of depth control in conjunction with an autonomous underwater vehicle (AUV):

[0227] Autonomous underwater vehicle (AUV) is a robot that can perform tasks underwater autonomously without real-time human operation. It is widely used in maritime investigation, security and protection, underwater archaeology, rescue and salvage, hydrographic survey and topographic exploration. At the same time, its structure is equipped with many actuators (thrusters, steering gears and mechanical arms, etc.) and sensors (depth sensors, inertial navigation systems and sonar, etc.). These devices will inevitably fail, but it is unrealistic to replace these devices before completing the task. Therefore, the algorithm designed in this paper uses the input and output data stored in these devices to reconstruct the system state and perform fault-tolerant depth control on the robot to verify the effectiveness and applicability of the proposed algorithm.

[0228] Depth control plays a vital role in ensuring the stability of autonomous underwater vehicles (AUVs) and the successful execution of missions. For example, when detecting the impact of industrial wastewater on marine life, different species of organisms have different living layers. Therefore, autonomous underwater robots need to dive to precise depths to perform marine pollution detection tasks - collecting water samples, measuring parameters such as temperature and salinity, etc.

[0229] This section uses the depth control simulation experiment of the autonomous underwater vehicle (AUV) to verify the effectiveness of the designed method. Therefore, for the autonomous underwater vehicle (AUV) with the following state space, the corresponding depth control problem can be discussed:

[0230] First, according to step 1, the state space model can be established as:

[0231]

[0232] in, is a real number within a certain range, and the expected output tracking value in (9) is set to (unit: m)y r =20, the relevant parameters in performance index (6) are: γ = 5.2, Q = diag (6, 0, 0, 1), R = 0.01, and the forward speed of the autonomous underwater robot (AUV) is set to 2.0 m / s.

[0233] This section will also illustrate the effectiveness of the proposed method through numerical experiments in the following five cases:

[0234] Define the situation where both the actuator and the sensor are operating normally: Case A:

[0235] (1) The actuator fails (for example, the thruster cannot move or can only move to a limited extent, which will cause the robot to be unable to reach the target area), and the sensor is functioning normally Situation:

[0236] Case B: The value measured by the actuator is lower than the actual value;

[0237] Case C: The value measured by the actuator is higher than the actual value;

[0238] (2) The sensor fails (for example, the depth measured by the depth sensor is significantly different from the actual value, which may cause damage to the robot structure), and the actuator operates normally Situation:

[0239] Case D: The value measured by the sensor is lower than the actual value;

[0240] Case E: The sensor measures a value that is higher than the actual value.

[0241] Fig. 9 The simulation diagram of the diving depth of the autonomous underwater robot (AUV) under the control of the designed controller under different fault situations is shown, corresponding to the diving depth trajectory of Case A-Case E. It can be seen from the figure that: through the generation and collection of initial moment data, the AUV starts to dive from the 4th moment. As time goes by, the AUV can dive to the expected depth of 20m under different fault conditions, indicating that the designed controller has excellent control effect.

[0242] Next, the norm convergence of the difference between the controller gain and the theoretical optimal value under different fault conditions and normal conditions will be compared to further illustrate the effectiveness of the designed controller.

[0243] Fig.10 (a) and (b) are actuator failure Case B (take ) controller gain Theoretical optimal value The norm convergence curve of the difference between the two. It can be seen from the figure that: since the measured data of the actuator is lower than the actual value, the fault system There are larger fluctuations than normal, which is also Fig. 9 The reason why Case B took a long time to dive to the desired depth (about 25 seconds). Fig.10 (c) and (d) show Case C (take )of The convergence is better, with a faster convergence speed and a smaller error value, which also enables the depth y3 in this case to reach the expected value in about 12 seconds, close to the depth control performance of the normal case Case A.

[0244] Fig.11 Shows the controller gains for two scenarios when a sensor fails Theoretical optimal value Norm convergence curve of the difference between them. Fig.11 .(a),(b) corresponds to Case D (take ), as shown in the figure, compared with the normal situation, the convergence curve fluctuates more widely at this time, but as time goes by and the iteration proceeds, it can reach a stable state before the 10th iteration cycle, and, combined with Fig.11 (b) It can be seen that the control depth y4 can reach the set value in about 15 seconds. Fig.11 .(c) and (d) show that Case E (taking ) The corresponding convergence curve experienced multiple jumps before becoming stable, but it was finally able to reach a stable value after 15 iterations, which made Figure 8 The corresponding system depth y5 in (b) reaches the set value at around 17 seconds.

[0245] Fig.12 is the control error diagram of the designed AUV depth controller under the above four fault conditions, Fig.12 (a)-(b) correspond to the fault case BE. As can be seen from the figure, after the data of the initial moment is generated and collected, the depth error of the four situations starts the dive command from the 4th moment. The depth controller makes the tracking error decrease at a faster speed and finally converge to 0.

[0246] In summary, the proposed algorithm uses historical input and output data to solve the optimal control problem, which not only enables the injection molding process to quickly achieve the desired output value under multiple faults, but also ensures that the AUV can effectively track the desired depth under various fault conditions, indicating that the proposed algorithm has excellent tracking control performance and strong applicability.

Claims

1. A data-driven fault-tolerant tracking control method for engineering systems, characterized in that: The steps include: S1. Establish an equivalent model of the engineering system under fault conditions, define the performance index function and the corresponding value function, and thus transform the optimal control problem into a zero-sum game problem; S2. For the problem that the state of the engineering system is unmeasurable, the state is reconstructed using historical input and output data to design and solve the optimal control strategy; S3. Introduce and define the Q function, and calculate the controller gain and disturbance gain. Design and combine the designed model-free non-strategy Q-learning output feedback algorithm to Optimize and obtain the optimal controller.

2. The data-driven engineering system fault-tolerant tracking control method according to claim 1 is characterized in that: In step S1, an engineering system fault equivalence model is established and the problem is transformed. The specific steps are as follows: S11. For the engineering system model with multiple fault effects, the equivalent system model is established as follows: First, establish the following linear discrete engineering system: Among them, s k ∈R n ,p k ∈R m ,d k ∈R r and k ∈R l are the system state, control input, disturbance input signal and system output at time k respectively, {A, B, C, D} is the system matrix of suitable dimension, A∈R n×n ,B∈R n×m ,C∈R l×n ,D∈R n×r , when the actuator and sensor fail during the production process, the actual input and output signals are expressed as: in, and Different values ​​of correspond to different operating conditions of actuators and sensors: ① hour, At this time, the equipment is operating normally; ② and ③ When the equipment fails partially; When , the equipment fails completely; therefore, the engineering system failure model is as follows: For the above engineering system failure model, the following variables are defined: Where, Δs k+1 ,Δd k , They are the state error, disturbance error and output tracking error of the engineering system under fault conditions, y r is the expected output tracking value set, g k ∈R m The iterative update law designed for time k. Combining (2) and (3), we get the equivalent model of the engineering system under fault: in, {A * ,B * ,C * ,D * } is a suitable-dimensional matrix. At this time, g k ,Δd k , are the system state, control input, disturbance input and system output of the engineering system at time k under fault. At the same time, hour, S12. Define the performance index function and the corresponding value function according to the optimal control theory: When the optimal strategy exists, the system will have an L2 gain less than or equal to γ ​​(γ ≥ 0), that is, it satisfies in, Q, R are the positive matrices to be determined, representing the output Control input g k The weight matrix of S13, for external interference Δd k Uncontrollable problems, combined with Minimax theory, convert the problem into a linear quadratic zero-sum game problem: Input control to g k and external disturbance Δd k As players 1 and 2 respectively, the H∞ problem is to solve the optimal control input that minimizes the value function and the worst external perturbation that maximizes the value function The problem of finding saddle points So that: Then the value function has the following quadratic form: Among them, P is a positive definite matrix.

3. The data-driven engineering system fault-tolerant tracking control method according to claim 1 is characterized in that: In step S2, the state is reconstructed using historical input and output data, and an optimal control strategy is designed. The specific steps are as follows: S21. To solve the problem that the state of the engineering system is unmeasurable, the state is reconstructed using historical input and output data: When A is non-singular and (A, C) is observable, the output of the equivalent model (4) of the engineering system under fault is It is expressed as: Expressed as follows: in, Let L = [(C * (A * ) -N ) T … (C * (A * ) -2 ) T (C * (A * ) -1 ) T ] T ,but Let Z = [L + L + E g L + E Δd ], then for ρ∈N, the original state variables of the equivalent model (4) of the engineering system under fault are reconstructed into the following form: in S22. For the equivalent model of the engineering system under fault (4), combined with the above state reconstruction form (7), the following optimal control input and worst disturbance input are designed based on data drive: in 4. The data-driven engineering system fault-tolerant tracking control method according to claim 1 is characterized in that: In step S3, the Q function is introduced and defined. Design and combine the designed model-free non-strategy Q-learning output feedback algorithm to Optimize and get the optimal controller, as follows: S31. Combined with the performance index (5), the Q-function is introduced and defined as follows: but: S32, controller gain and disturbance gain The design is as follows: For stable strategies The Q-function has the following quadratic form: in Combining formula (8), we can get: From the above formula, we can see that the data-driven model-free non-strategy Q-learning algorithm actually solves the parameters of the matrix L to calculate the controller gain and disturbance gain. Design; S33: In order to solve the contradiction between exploring data and using data in Q-learning, we introduce the target strategy and The control strategy is continuously optimized as follows: First, combined with the state variable reconstruction formula (7), the equivalent model of the engineering system under fault (4) is equivalent to the following expression: in, Combining (6) and (9), the following equation holds: in: Combined with (10a), the above formula (12) is transformed into: Using Kronecker product, the above formula can be rewritten as: The controller gain elements are solved by the least squares method: At this time S j+1 is expressed as: in, Combining formula (11), we can get: In the result of formula (13), the model-free non-strategy Q-learning output feedback algorithm is used to Optimize and obtain the optimal controller.

Images