A controller parameter online setting method based on control performance assessment and reliability domain bayesian optimization

Abstract

The application discloses a controller parameter online setting method based on control performance evaluation and reliability domain Bayesian optimization. The method comprises the following steps: performing control performance evaluation on a control loop in time according to online running data, and performing controller parameter optimization when a performance index is lower than an expected value; establishing a local GPR model based on a reliability domain, which is used for representing the relationship between the controller parameter and the control performance; maximizing a designed acquisition function in the reliability domain to select a new controller parameter; performing control performance evaluation on the selected controller parameter; if an optimization stop condition is not met, adaptively updating the reliability domain to perform a next iteration optimization process, otherwise, stopping the optimization. The method of the application avoids additional excitation experiments on the control system, and can safely and quickly optimize the controller parameter under the normal operation of a production process, so that the control system performance can be kept at a good level for a long time.

Description

Technical Field

[0001] The present invention belongs to the field of process industry control engineering, and in particular relates to an online controller parameter tuning method based on control performance evaluation and trust region Bayesian optimization. Background Art

[0002] Maintaining optimal control system performance is crucial in industrial processes. Control performance directly impacts product quality, production efficiency, and resource utilization. A well-functioning control system accurately controls and regulates the controlled variables in the production process, ensuring stable and safe operation across all production links.

[0003] However, in actual industrial processes, degraded control performance is a common phenomenon, primarily caused by inappropriate controller parameters. Most controller parameters are determined based on system characteristics during the controller design phase and are not actively adjusted thereafter. However, over time, due to changes in the production environment, equipment aging, and varying operating conditions, system characteristics change, and the original controller parameters no longer align with the new system characteristics, leading to degraded control system performance. Only when system performance degrades significantly, or even oscillations occur, do personnel consider retuning controller parameters. Traditional controller parameter tuning methods typically require additional excitation experiments on the control system to obtain system characteristics or identify models. However, this interferes with the normal operation of the production process, is prohibitively expensive, and is unacceptable in many scenarios.

[0004] Therefore, how to optimize controller parameters in a timely and online manner to keep system control performance within the expected range for a long time has become an urgent need in today's control industry. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention proposes an online tuning method for controller parameters based on control performance evaluation and trust region Bayesian optimization.

[0006] The method comprises the following steps:

[0007] Step (1): Based on the operating data of the closed-loop control system, the closed-loop impulse response (IR) model of the closed-loop control system at the current moment is calculated and compared with the closed-loop impulse response model of the reference control system to obtain the regulation performance index (RPI); if the RPI is less than the set threshold, step (2) is performed; otherwise, the controller continues to operate with the current parameters and repeats step (1) at the next moment;

[0008] Step (2): Use the regulation performance index to construct the objective function, further calculate the overshoot and response time as constraints, and construct the optimization problem; use the trust region Bayesian optimization method to perform online iteration of the controller parameters in the closed-loop control system to solve the optimization problem, and apply the obtained optimal controller parameters to the closed-loop control system to complete the tuning;

[0009] The use of the trust region Bayesian optimization method to perform online iteration of controller parameters to solve the optimization problem specifically includes:

[0010] (2.1) Initializing the data sample and trust region, wherein the initial data set is constructed by performing Latin hypercube sampling within a set range near the current controller parameters and calculating the objective function value and constraint value; the initial trust region is constructed by taking the current controller parameters as the center and the set range as the side length;

[0011] (2.2) Establish a local Gaussian process regression (GPR) model based on the trust region to characterize the relationship between controller parameters and control performance;

[0012] (2.3) Maximize the expected improvement acquisition function considering the constraints and select the controller parameters that are most likely to improve the control performance;

[0013] (2.4) If the optimization stop condition is met, exit the optimization phase; if not, proceed to step (2.5);

[0014] (2.5) Update the trust region based on the performance evaluation results of the selected controller parameters and return to step (2.2).

[0015] Furthermore, the optimization problem of step (2) is expressed as:

[0016]

[0017] stc1(x)≤threshold1

[0018] c2(x)≤threshold2

[0019] Where x is the d-dimensional controller parameter to be tuned, x * The objective function J(x) is the adjustment performance index, which is a function of the controller parameter x. The constraints c1(x) and c2(x) are the overshoot and recovery time, respectively. Threshold1 and threshold2 are the set overshoot threshold and recovery time threshold, respectively. Ω is the range of controller parameters that can be adjusted, i.e., the trust region range.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention only requires daily loop operation data to perform control performance evaluation to iteratively optimize controller parameters, avoiding additional excitation experiments on the production device and facilitating the normal operation of the production process.

[0022] The present invention enables timely and proactive automatic tuning of controller parameters after control performance deteriorates, ensuring that the control loop performance is maintained at a desired level for a long period of time.

[0023] The present invention can quickly optimize controller parameters while ensuring safety, only requires a small number of parameter experiments, and has a low cost for parameter optimization, which meets the needs of industrial sites. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flowchart of the controller parameter online tuning method based on control performance evaluation and trust region Bayesian optimization;

[0025] Figure 2 It is a block diagram of the closed-loop control loop structure of an industrial process;

[0026] Figure 3 It is a schematic diagram of control performance evaluation;

[0027] Figure 4 It is a diagram of trust region Bayesian optimization;

[0028] Figure 5 It is a comparison chart of the controller parameter tuning process regulation performance index;

[0029] Figure 6 This is a comparison chart of overshoot during the controller parameter tuning process;

[0030] Figure 7 This is a comparison chart of the response time during the controller parameter tuning process;

[0031] Figure 8 It is a comparison chart of closed-loop impulse response of controller parameter tuning results. DETAILED DESCRIPTION

[0032] The present invention will be further described and illustrated below in conjunction with specific embodiments. The technical features of each embodiment of the present invention can be combined accordingly without conflict.

[0033] The present invention's online controller parameter tuning method, based on control performance evaluation and trust-region Bayesian optimization, is primarily applicable to industrial process control systems, enabling the active adjustment of controller parameters. This allows the control system to address performance degradation caused by changes in the production environment, equipment aging, and varying operating conditions. Furthermore, the proposed method can perform online iterative optimization of controller parameters by evaluating control performance based on daily operating data, avoiding additional stimulus testing of production equipment and facilitating the normal operation of the production process.

[0034] like Figure 1 As shown, the implementation steps of this method are as follows:

[0035] Regularly evaluate control performance using daily operating data

[0036] The performance of a closed-loop control system can be quantified using a closed-loop impulse response model. Using the system's real-time operating data, the closed-loop impulse response model of the current system is calculated at set intervals (four hours in this example). This model is then compared with the closed-loop impulse response model of a reference control system to generate a regulation performance index. The reference control system's closed-loop impulse response model represents a benchmark control system model expected by users in real-world industrial scenarios.

[0037] Figure 2 The block diagram of the closed-loop control system commonly used in industrial processes is shown. Usually, the closed-loop control system can be described using an autoregressive (AR) model:

[0038] y t +a1y t-1 +…+a n y t-n =e t

[0039] Among them, y t ,y t-1 ,...,y t-n They represent the output values ​​of the control system at time t, time t-1, ..., time tn respectively; a1, ..., a n represents the coefficient of the AR model; e t Represents the white noise value at time t; the closed-loop control system model can be equivalently transformed into an IR model:

[0040] y t =(f0+f1q -1 +…+f d-1 q -d+1 +f d q -d +…+f i q -i+…)e t

[0041] Among them, f i is the coefficient of the IR model, q is the unit forward operator, and d is the lag time of the system. The output variance of the closed-loop control system is as follows:

[0042]

[0043] in is the output variance of white noise. Assume that the IR model of the reference control system is:

[0044]

[0045] Where α=exp(-t s / τ ref ) is the attenuation factor, τ ref is the reference time constant, t s is the data sampling period.

[0046] Therefore, the output variance of the reference control system is:

[0047]

[0048] like Figure 3 As shown, the output variance of the reference control system is compared with the output variance of the current control system to obtain the regulation performance index:

[0049]

[0050] (1) Perform controller parameter optimization steps based on trust region Bayesian optimization

[0051] In the present invention, when the regulation performance index RPI is less than the set threshold (the threshold is selected as 0.6 in this embodiment), it is determined that the current control performance is poor. For systems with poor control performance, the present invention uses Bayesian optimization to perform controller parameter tuning online. In order to ensure the safety and stability of the system, the controller parameters are first optimized progressively based on the trust region strategy to ensure that the controller parameters will not change suddenly. Secondly, the overshoot and recovery time are further calculated based on the closed-loop impulse response model as safety constraints in the optimization process. Figure 3 As shown in the figure, the overshoot is the ratio of the maximum deviation of the controlled quantity to the steady-state value under the set end step input, and the recovery time is the shortest time it takes to return to the steady-state value within 5% and no longer exceed the range.

[0052] The regulation performance index RPI of the present invention will change according to the different controller parameters x. Therefore, the regulation performance index RPI can be understood as a function J(x) of the controller parameter x.

[0053] The controller parameter tuning problem is summarized as follows:

[0054]

[0055] s.t.c1(x)≤threshold1

[0056] c2(x)≤threshold2

[0057] where x is the d-dimensional controller parameter to be tuned, x * is the optimal controller parameter to be solved, the objective function J(x) is the regulation performance index, the constraint conditions c1(x) and c2(x) are the overshoot and recovery time respectively, threshold1 and threshold2 are the set overshoot threshold and recovery time threshold respectively, and Ω is the range of controller parameter allowed to be tuned, i.e. the range of the trust region. The present application adopts the trust region Bayesian optimization method shown in the figure to iteratively solve the optimization problem, and applies the obtained optimal controller parameter to the closed-loop control system to complete the tuning. The iterative solution specifically includes: Figure 4

[0058] (2.1) initialize the data sample and the trust region, wherein the initial data set is constructed by performing Latin hypercube sampling in the range set near the current controller parameter and calculating the objective function value and the constraint value; the initial trust region is constructed by setting the current controller parameter as the center and the range as the edge length;

[0059] (2.2) establish a local GPR model based on the trust region to represent the relationship between the controller parameter and the control performance;

[0060] (2.3) maximize the expected improvement acquisition function considering the constraints to select the controller parameter most likely to improve the control performance;

[0061] (2.4) if the optimization stopping condition is met, exit the optimization link; if not, perform step (2.5);

[0062] (2.5) update the trust region according to the performance evaluation result of the selected controller parameter, and return to step (2.2).

[0063] The present application further describes each step of the iterative solution in detail:

[0064] (2.1) initialize the data set and the trust region

[0065] perform Latin hypercube sampling on n ini points in the range of thirty percent near the current controller parameter, wherein n ini = 2d-1. The controller performance of the sampled controller parameters is evaluated to construct the initial data set.​

[0066] The boundary of the initial trust region is determined by the center C and the radius L, as shown in the following formula:

[0067]

[0068] By initializing the controller parameters x start Initialize the center C and radius L:

[0069] C (1) =x start

[0070] L (1) =30%x start

[0071] (2.2) Establishing a local GPR model based on trust region

[0072] GPR is a commonly used nonparametric probability model. In GPR, the objective function J(X) can be approximated as a Gaussian process (GP), whose properties are completely determined by the mean m(x) and the kernel function k(x,x'), where x and x' are d-dimensional input vectors:

[0073]

[0074] The kernel function uses the Matérn 5 / 2 kernel, which is sufficient to model quadratically differentiable and not very smooth functions:

[0075]

[0076] where γ is the signal standard deviation, l is the length scale parameter, and r(x,x') is the distance between data points.

[0077] In the process of calculating the control performance, due to the model error generated when fitting the AR model and the error of the signal sequence itself, the result will also contain noise. Assume that the obtained regulation performance index h(x) containing noise satisfies h(x)=J(x)+ε, and the noise ε satisfies the independent and identically distributed Gaussian distribution: p(ε)=N(0,σ ε 2 ). The controller parameter data set and the objective function value set in the current trust region are represented as x={x i |i=1,2,...,n} and h={h i |i=1,2,...,n}, where xi is the i-th controller parameter and hi is the noise-containing regulation performance index value corresponding to the i-th controller parameter. For any unevaluated controller parameter x * , and its corresponding function value is J(x *), according to the properties of Gaussian process, we can get:

[0078]

[0079] where I denotes the identity matrix and k(·,·) is the Matérn 5 / 2 kernel function.

[0080] According to the above joint probability distribution, we can further obtain the following posterior distribution of the regulation performance index without evaluating the controller parameters:

[0081]

[0082] cov(J(x * ))=k(x * ,x * )-k(x * ,x)[k(x,x)+σ ε 2 I] -1 k(x,x * )

[0083] in represents J(x * ), cov(J(x*)) represents the predicted mean of J(x * )’s predicted covariance.

[0084] (2.3) Maximizing the acquisition function to select controller parameters

[0085] Similar to the objective function J(X), we use the GPR model to constrain c i (x) Modeling. Based on the posterior distribution of the control performance index of the unevaluated controller parameters obtained by the GPR model, an expected improvement acquisition function considering the constraints is formulated:

[0086]

[0087] where h + is the maximum value of the regulation performance index that meets the constraints among the currently evaluated points, μ i (x) and σ i (x) are the posterior prediction mean and standard deviation of the i-th constraint, Φ(·) is the cumulative distribution function of the standard normal distribution, v i is the constraint threshold of the i-th constraint condition in the trust region. If there is a controller parameter in the current trust region that satisfies the set global constraint threshold, then v i Equal to threshold i , otherwise v iis the minimum value of the existing constraint in the trust region. This is to avoid the acquisition function from failing when all points in the trust region do not meet the global constraint threshold. EI(x|y + ) is the expected improvement function, which represents the expected value of the improvement of the prediction point:

[0088]

[0089] where μ(x) and σ(x) are the posterior predicted mean and standard deviation of the conditioning performance index, respectively, and φ(·) is the probability distribution function of the standard normal distribution.

[0090] At the current k-th iteration, the next set of controller parameters x can be selected by maximizing the expected improvement acquisition function with constraints k+1 :

[0091]

[0092] (2.4) Determine whether the optimization termination condition is met

[0093] Determine whether the number of optimizations reaches the set maximum limit (the maximum limit in this embodiment is set to 40 times). If so, exit the optimization phase; otherwise, continue to update the trust region and enter the next iterative optimization.

[0094] (2.5) Trust Region Update

[0095] The trust region is updated based on the parameter x of the kth iteration k+1 Optimization results, if x k+1 If the adjustment performance index is improved while the constraints are satisfied (i.e., the adjustment performance index increases), the iteration is considered successful, and the trust region updates its center point and expands. If the constraints are violated or the adjustment performance index is not improved, the iteration is considered failed, and the trust region center point remains unchanged and shrinks:

[0096]

[0097] Among them L max and L min are the upper and lower thresholds of the overall trust region size.

[0098] The hardware platform for simulation in the present invention is a PC with Windows 10 system, and the software platform is MATLAB R2022b.

[0099] The present invention is applied to a CSTR model, and its mathematical model can be described by the following formula:

[0100]

[0101] The product concentration C A is the controlled variable. Coolant flow rate qc (t) is the manipulated variable. Other model parameters are shown in Table 1.

[0102] Table 1 CSTR model parameters table

[0103]

[0104]

[0105] The feed temperature T0 has a Gaussian noise disturbance with mean 0 and variance 1. To stabilize the product concentration at a set value, we use a PID controller with filter coefficient N = 10:

[0106]

[0107] When the CSTR product concentration Ca is set to 0.1, the reference time constant is set to 15 s, the overshoot constraint threshold is 5%, and the recovery time constraint threshold is 60 s. When the initial controller parameters (K c , τ I , τ D ) are (1050, 0.15, 0.15), the RPI is detected to be less than 0.6, indicating that the control loop has poor regulation ability, and the controller parameters need to be tuned. In the optimization process, the control performance is evaluated every 4 h using the running data, where the sampling time of the set value, controller output value, and process variable value is set to 5 s. The method is compared with the traditional Bayesian optimization method in terms of optimization performance and control performance improvement before and after tuning. Table 2 shows the comparison of the tuning effect and the optimization performance of the two methods. Figure 5-7 and Figure 8 shows the comparison of the controller parameter tuning process and the controller parameter tuning results.

[0108] Table 2 Tuning effect comparison

[0109]

[0110] As can be seen, compared with the traditional Bayesian optimization method, the method can more quickly realize the tuning of the controller parameters, which is conducive to reducing the cost of tuning the controller parameters in actual industrial production. In addition, even in the case of serious violation of the safety constraints by the initial controller parameters, the method can make the control performance meet the constraint threshold as soon as possible, and does not violate the constraints after meeting the constraint threshold, which can ensure the stability and safety of the control system.

[0111] The above examples are not a limitation of the present application, and the present application is not limited to the above examples, as long as it meets the requirements of the present application, it belongs to the protection scope of the present application.

Claims

1. A controller parameter online tuning method based on control performance evaluation and trust region Bayesian optimization, characterized in that: The method comprises the following steps: Step (1): Based on the operating data of the closed-loop control system, the closed-loop impulse response model of the closed-loop control system at the current moment is calculated and compared with the closed-loop impulse response model of the reference control system to obtain the regulation performance index; if the regulation performance index is less than the set threshold, step (2) is performed; otherwise, the controller continues to operate with the current parameters and repeats step (1) at the next moment; Step (2): Use the regulation performance index to construct the objective function, further calculate the overshoot and response time as constraints, and construct the optimization problem; use the trust region Bayesian optimization method to perform online iteration of the controller parameters in the closed-loop control system to solve the optimization problem, and apply the obtained optimal controller parameters to the closed-loop control system to complete the tuning; The use of the trust region Bayesian optimization method to perform online iteration of controller parameters to solve the optimization problem specifically includes: (2.1) Initializing the data sample and trust region, wherein the initial data set is constructed by performing Latin hypercube sampling within a set range near the current controller parameters and calculating the objective function value and constraint value; the initial trust region is constructed by taking the current controller parameters as the center and the set range as the side length; (2.2) Establish a local GPR model based on the trust region to characterize the relationship between controller parameters and control performance; (2.3) Maximize the expected improvement acquisition function considering the constraints and select the controller parameters that are most likely to improve the control performance; (2.4) If the optimization stop condition is met, exit the optimization phase; if not, proceed to step (2.5); (2.5) Update the trust region based on the performance evaluation results of the selected controller parameters and return to step (2.2).

2. The controller parameter online tuning method based on control performance evaluation and trust region Bayesian optimization according to claim 1, characterized in that: The step (1) is specifically as follows: Use AR model to describe closed-loop control system: and t +a1y t-1 +…+a n and t-n =e t Among them, y t ,y t-1 ,...,y t-n They represent the output values ​​of the control system at time t, time t-1, ..., time tn respectively; a1, ..., a n represents the coefficient of the AR model; e t represents the white noise value at time t; Convert the AR model of the closed-loop control system into a closed-loop impulse response model: y t =(f0+f1q -1 +…+f d-1 q -d+1 +f d q -d +…+f i q -i +…)e t Among them, f i is the coefficient of the closed-loop impulse response model, q is the unit forward operator, and d is the lag time of the system; Then the output variance of the closed-loop control system is as follows: in is the output variance of white noise; The closed-loop impulse response model of the reference control system in step (1) is a control system model benchmark expected by users in actual industrial scenarios; the closed-loop impulse response model of the reference control system is: Where α=exp(-t s / τ ref ) is the attenuation factor, τ ref is the reference time constant, t s is the data sampling period; The output variance of the reference control system is: Comparing the output variance of the reference control system with the output variance of the current control system can obtain the regulation performance index RPI:

3. The controller parameter online tuning method based on control performance evaluation and trust region Bayesian optimization according to claim 1, characterized in that: The optimization problem of step (2) is expressed as: stc1(x)≤threshold1 c2(x)≤threshold2 Where x is the d-dimensional controller parameter to be tuned, x * The objective function J(x) is the adjustment performance index, which is a function of the controller parameter x. The constraints c1(x) and c2(x) are the overshoot and recovery time, respectively. Threshold1 and threshold2 are the set overshoot threshold and recovery time threshold, respectively. Ω is the range of controller parameters that can be adjusted, i.e., the trust region range.

4. The controller parameter online tuning method based on control performance evaluation and trust region Bayesian optimization according to claim 1, characterized in that: The specific method of step (2.1) is as follows: Perform Latin hypercube sampling n within the set range near the current controller parameters ini points, where n ini =2d-1; perform controller performance evaluation on the sampled controller parameters and construct an initial data set; The boundary of the trust region is determined by the center C and the radius L, as shown in the following formula: By initializing the controller parameters x start Initialize the center C and radius L.

5. The controller parameter online tuning method based on control performance evaluation and trust region Bayesian optimization according to claim 4 is characterized in that The specific method of step (2.2) is as follows: A local GPR model based on trust region is established. In GPR, the objective function J(X) is approximated as a Gaussian process, whose properties are completely determined by the mean m(x) and the kernel function k(x,x'), where x and x' are d-dimensional input vectors: The kernel function uses the Matérn 5 / 2 kernel, which is sufficient to model quadratically differentiable and not very smooth functions: Where γ is the signal standard deviation and l is the length scale parameter; In the process of calculating the control performance, the result will contain noise; it is assumed that the obtained regulation performance index h(x) containing noise satisfies h(x)=J(x)+ε, and the noise ε satisfies the independent and identically distributed Gaussian distribution: p(ε)=N(0,σ ε 2 ); The controller parameter data set and the objective function value set in the current trust region are respectively expressed as x={x i |i=1,2,...,n} and h={h i |i=1,2,...,n}, where x i is the i-th controller parameter, h i is the noise-containing regulation performance index value corresponding to the i-th controller parameter; for any unevaluated controller parameter x * , and its corresponding function value is J(x * ), according to the properties of Gaussian process, we get: Where I represents the identity matrix, k(·,·) is the Matérn 5 / 2 kernel function; According to the above joint probability distribution, the following posterior distribution of the regulation performance index without evaluating the controller parameters is further obtained: cov(J(x * ))=k(x * ,x * )-k(x * ,x)[k(x,x)+σ ε 2 I] -1 k(x,x * ) in represents J(x * ), cov(J(x*)) represents the predicted mean of J(x * )’s predicted covariance.

6. The controller parameter online tuning method based on control performance evaluation and trust region Bayesian optimization according to claim 5, characterized in that: The specific method of step (2.3) is as follows: Based on the posterior distribution of the control performance index of the unevaluated controller parameters obtained by the GPR model, an expected improvement acquisition function considering the constraints is formulated: where h + is the maximum value of the regulation performance index that meets the constraints among the currently evaluated points, μ i (x) and σ i (x) are the posterior prediction mean and standard deviation of the i-th constraint, Φ(·) is the cumulative distribution function of the standard normal distribution, v i is the constraint threshold of the i-th constraint condition in the trust region. If there is a controller parameter in the current trust region that satisfies the set global constraint threshold, then v i Equal to threshold i , otherwise v i is the minimum value of the existing constraints in the trust region, EI(x|y + ) is the expected improvement function, which represents the expected value of the improvement of the prediction point: where μ(x) and σ(x) are the posterior predicted mean and standard deviation of the conditioning performance index, respectively, and φ(·) is the probability distribution function of the standard normal distribution; At the current k-th iteration, the next set of controller parameters x is selected by maximizing the expected improvement acquisition function with constraints k+1 :

7. The controller parameter online tuning method based on control performance evaluation and trust region Bayesian optimization according to claim 1 is characterized in that In step (2.4), if the number of optimizations reaches the maximum limit, exit the optimization phase; otherwise, execute step (2.5).

8. The controller parameter online tuning method based on control performance evaluation and trust region Bayesian optimization according to claim 6 is characterized in that The specific method of step (2.5) is as follows: The trust region is updated based on the parameter x of the kth iteration k+1 Optimization results, if x k+1 If the adjustment performance index is improved while satisfying the constraints, the iteration is considered successful, and the trust region updates its center point and expands. If the constraints are violated or the adjustment performance index is not improved, the iteration is considered failed, and the trust region center point remains unchanged and shrinks: Among them L max and L min are the upper and lower thresholds of the overall trust region size.

Images