Optimal control device, optimal control method, and computer program

Abstract

The purpose of the present embodiment is to provide an optimum control device that achieves search for an optimum value for improving control performance while maintaining stability. An optimal control device (100) according to an embodiment comprises: a process measurement value acquisition unit (110) that acquires a measurement value measured in a process to be controlled; an evaluation value calculation unit (120) that calculates an evaluation value of the process to be controlled using the measurement value; a phase difference compensation operation amount calculation unit (130) that receives an operation amount and the evaluation value as inputs, calculates an estimated amount of phase delay from the operation amount to the evaluation value, and adjusts the phase of the operation amount so that the phase difference between the operation amount and the evaluation value is zero; an evaluation function gradient estimation unit (140) for calculating gradient information of an evaluation function, which is the rate of change of the evaluation value with respect to the operation amount, using the post-adjustment operation amount and evaluation value information; a gradient code information estimation unit (150) for extracting code information of the gradient of the evaluation function using the gradient information; and an extreme value search unit (160) for determining the direction and amount of movement of the operation amount from the code information.

Description

Optimal control device, optimal control method, and computer program

[0001] Embodiments of the present invention relate to an optimal control device, an optimal control method, and a computer program.

[0002] In plant control systems such as water treatment plants and environmental plants, the process state changes due to disturbances and environmental conditions. Often, it is necessary to optimize the process parameters to mitigate the effects of these disturbances, or to determine the optimal process parameters under specific environmental conditions.

[0003] For example, in sewage treatment and wastewater treatment processes, it is necessary to adjust blower airflow, pump flow rate, and chemical input in response to disturbances such as the inflow volume and inflow load (= inflow volume × inflow water quality concentration) into the sewage treatment plant to maintain effluent water quality. Furthermore, in biological wastewater treatment, the optimal operating parameters change particularly depending on seasonal annual water temperature fluctuations.

[0004] Furthermore, in water treatment, the amount of chemicals injected to maintain the required water quality changes depending on external disturbances such as turbidity and solar radiation at the water source. In addition, the amount of coagulant and chlorine injected is often greatly affected by annual fluctuations in water temperature.

[0005] Furthermore, in denitrification control at thermal power plants and the like, nitrogen oxides (NOx) emitted from the power plant are treated as a disturbance, and it is necessary to adjust the amount of ammonia injected to treat NOx according to the amount of NOx emitted.

[0006] In petrochemical and steel plants, it is necessary to control the operational parameters (temperature, pressure, flow rate, etc.) required for petroleum refining and steelmaking to optimal values ​​according to the quality and quantity of raw materials (heavy oil, iron ore, etc.).

[0007] As described above, in plant control in many industries, the state of the plant changes due to the effects of disturbances and environmental conditions. In these plants, disturbance suppression control is often used, which changes various manipulated variables to maintain the plant in an optimal state in order to deal with disturbances, or scheduling control is used, which switches control according to environmental conditions, and in some cases, adaptive control that adapts to environmental conditions is employed.

[0008] Extreme Seeking Control (ESC) is gaining attention as a representative control method for maintaining plants in optimal condition, and is beginning to be used in industry. Extreme Seeking Control is a control method that searches for an extreme value by setting an evaluation function for the optimality to be minimized or maximized, such as plant operating costs or power generation, and constantly changing the manipulated variable while measuring the extreme value (local minimum or local maximum) of that evaluation value in real time. Extreme Seeking Control is attracting attention as a practical optimal control technology for processes involving complex phenomena that are difficult to model mathematically (e.g., sewage treatment processes, combustion processes, petrochemical processes, etc.).

[0009] Extreme value control is an adaptive search method that calculates the optimal value of an evaluation function by changing the manipulated variable while maintaining the evaluation function value at its optimal value (minimum or maximum) based on information obtained from online sensors that can directly measure the value of the evaluation function to be optimized. In other words, extreme value control is an attractive method because it searches for the optimal value by changing the manipulated variable without using complex mathematical models, making it easy to implement in controllers such as PLCs (Programmable Logic Controllers).

[0010] On the other hand, extreme value control is generally difficult to improve while maintaining stability (the ability to search for extreme values) and control performance (such as the speed of convergence to extreme values ​​and the error between the final value and the extreme value). Achieving both stability and control performance (convergence speed + optimality (error from the optimal value)) is an essential challenge when applying extreme value control to real-world controlled objects (plants).

[0011] International Publication No. 2020 / 241657, Japanese Patent Publication No. 2019-83030, Japanese Patent Publication No. 2024-17830

[0012] Yamanaka, Osamu, et al. "Extremum Seeking Based on Approximated Sign of Gradient of Unknown Plant Maps." 2020 59th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE)., 2020.B. G. B. Hunnekens, M. A. M. Haring, N. van de Wouw, andH. Nijmeijer, “A dither-free Extremum-seeking control approach using 1st-order least-squares fits for gradient estimation,” in 53rd Conference on Decision and Control, 2014, pp. 2679-2684.Zengin, Nursefa, and Baris Fidan. "Adaptive extremum seeking using recursive least squares." arXiv preprint arXiv:2003.03891 (2020).Onishi, Yuta, et al. "Extremum Seeking Control for Wastewater Treatment Plant with Prioritized Output Constraints." 2020 59th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE)., 2020.Y. Tan, D. Nesic and I.M.Y. Mareels, “On non-local stability properties of extremum seeking control”, Automatica, vol. 42 (2006), pp.889-903.

[0013] Embodiments of the present invention have been made in view of the above circumstances, and aim to provide an optimal control device, an optimal control method, and a computer program that realize the search for an optimal value that improves control performance while maintaining stability.

[0014] The optimal control device according to the embodiment is a device that searches for the optimal value of an evaluation function by manipulating an manipulated variable in real time based on an manipulated variable of a controlled process and an evaluation value of an evaluation function that indicates an index for the optimization of the controlled process based on a control variable that changes according to the manipulated variable, and comprises: a process measurement value acquisition unit that acquires a measurement value measured in the target process; a process evaluation value calculation unit that calculates an evaluation value of the controlled process using the measurement value; a phase difference compensation manipulated variable calculation unit that calculates an estimated amount of phase delay from the manipulated variable to the evaluation value and adjusts the phase of the manipulated variable so that the phase difference between the manipulated variable and the evaluation value becomes zero, using the manipulated variable after phase adjustment and information on the evaluation value to calculate gradient information of the evaluation function which is the rate of change of the evaluation value with respect to the manipulated variable; a gradient sign information estimation unit that extracts sign information of the gradient of the evaluation function using the gradient information; and an extreme value search unit that determines the direction and amount that the manipulated variable should move from the sign information.

[0015] FIG. 1 is a diagram for explaining an example of a control target of an optimal control device, an optimal control method, and a computer program according to an embodiment. FIG. 2 is a diagram for explaining the principle of searching for an extreme value (local optimal value) in extreme value control. FIG. 3 is a block diagram schematically showing a configuration example of the optimal control device according to the first embodiment. FIG. 4 is a diagram schematically showing an example of the phase difference between input data corresponding to an operation amount and output data corresponding to an evaluation amount. FIG. 5 is a diagram for explaining an example of a configuration for calculating an estimated value of the phase difference between an input signal and an output signal. FIG. 6 is a diagram for explaining an example of a configuration for calculating an estimated value of the phase difference between an input signal and an output signal. FIG. 7 is a diagram schematically showing an example of a process evaluation amount and a process operation amount before phase difference adjustment. FIG. 8 is a diagram schematically showing an example of a process evaluation amount and a process operation amount after phase difference adjustment. FIG. 9 is a diagram for explaining an example of extreme value control by an optimal control device of a comparative example. FIG. 10 is a diagram for explaining an example of extreme value control by an optimal control device of a comparative example. FIG. 11 is a diagram for explaining an example of extreme value control by an optimal control device of a comparative example. FIG. 12 is a diagram schematically showing the functions of the optimal control device according to the first embodiment. FIG. 13 is a block diagram schematically showing a configuration example of the optimal control device according to the second embodiment. FIG. 14 is a block diagram schematically showing a configuration example of a part of the optimal control device according to the second embodiment shown in FIG. 13. Embodiment

[0016] Hereinafter, the optimal control device, the optimal control method, and the computer program according to the embodiment will be described in detail with reference to the drawings. FIG. 1 is a diagram for explaining an example of a control target of an optimal control device, an optimal control method, and a computer program according to an embodiment.

[0017] The optimal control device according to the present embodiment is a device that operates the operation amount in real time based on the operation amount of the control target process and the evaluation value of an evaluation function based on the control amount that changes according to the operation amount, and searches for the optimal value of the evaluation value.

[0018] Here, the coagulant (PAC) injection process of a water treatment plant (e.g., a water purification plant) will be described as an example of the control target of the optimal control device of this embodiment. Note that the control target of the optimal control device of the embodiment may be any process and is not limited to the process shown in FIG. 1.

[0019] The water treatment plant includes, for example, a mixing tank, a flocculation tank, a sedimentation tank, a sludge removal tank, a sand filtration tank, and a water purification tank. Coagulant is injected into the raw water flowing into the water treatment plant in the mixing tank, and the raw water is agitated in the flocculation tank to form solids called flocs. A part of the treated water containing flocs is discharged to the sludge removal tank, and a part of it precipitates in the sedimentation tank. The treated water discharged from the flocculation tank is sent to the sand filtration tank, and the filtered treated water is discharged to the water purification tank.

[0020] In the coagulant injection process of the above water treatment plant, flocs are formed by injecting the coagulant (PAC). The formed flocs move by applying an electric field. The optimal control device of this embodiment measures the moving speed (PV) of the flocs by image processing, and adjusts the injection rate (MV) by PI control so that the moving speed follows the target value (SV) near zero.

[0021] Note that the target value SV of the moving speed PV is preferably set near zero, but its optimal value is considered to vary depending on the water temperature, the quality of the influent water, etc. Therefore, for example, when the optimal control device of this embodiment controls the coagulant injection process, the target value SV of the moving speed of the flocs is adjusted by extreme value control.

[0022] The control system shown in Figure 1 has a configuration called cascade control, and is a two-stage cascade configuration such that the target value SV of PI control becomes the manipulated variable MV value of extreme value control. In this case, the evaluation function in the optimal control device of this embodiment is, for example, the sum of the chemical cost of PAC, the sludge disposal cost, and the washing cost, and the evaluation function (operating cost) is set, with constraints on the turbidity of the settled water and filtered water. The constraints can be converted into the evaluation function by a method such as the one shown in Non-Patent Document 5 or a method called a penalty function. For simplicity, below we will assume that the constraints are incorporated into the evaluation function by a penalty function, and the evaluation function that incorporates these water quality constraints (constraints) into the above operating cost will be called the total cost.

[0023] Figure 2 is a diagram illustrating the principle of extremum (local optimal) search in extremum control. In Figure 2, the horizontal axis represents the manipulated variable operated by extremum control, and in this embodiment, it is the target value (SV) of the movement speed. Also in Figure 2, the vertical axis represents the evaluation function (evaluation variable) to be optimized (minimized) by extremum control, and in this embodiment, it is the total cost.

[0024] It is assumed that there is some kind of relationship between the manipulated variable and the evaluation function, and that it has extrema (local minimum). In Figure 2, a downward-convex functional relationship is assumed between the manipulated variable and the evaluation function.

[0025] When actually performing extreme value control, the values ​​of the evaluation variable relative to the manipulated variable at each moment of control execution can be obtained in real time, but the overall shape of the evaluation function cannot be grasped in real time and remains unknown. In such a situation, extreme value control provides a control algorithm for searching for the extreme values ​​(local optima; in the example shown, the function is unimodal and convex downwards, so the global minimum) shown in the figure.

[0026] As shown in the left diagram of Figure 2, when the manipulated variable is driven by a dither signal, which is a periodic signal such as a sine wave, if the manipulated variable is on the right side of the extreme value (the side where the manipulated variable increases), the movement of the manipulated variable driven by the dither signal and the movement of the evaluation variable (evaluation value) acquired in real time move in sync, that is, they move in phase.

[0027] On the other hand, when the manipulated variable is driven by a dither signal, which is a periodic signal such as a sine wave, if the manipulated variable is to the left of the extreme value (on the side where the manipulated variable decreases), the movement of the manipulated variable driven by the dither signal and the movement of the evaluation variable (evaluation value) acquired in real time will be in opposite phases. In other words, to the left of the extreme value, the manipulated variable driven by the dither signal and the evaluation variable (evaluation value) acquired in real time will move in opposite phases.

[0028] The optimal control device can use this information to determine whether the manipulated variable during control operation is to the left or right of the extremum. If the manipulated variable is to the right of the extremum during control operation, the optimal control device decreases the manipulated variable; if it is to the left of the extremum during control operation, it increases the manipulated variable, thereby enabling extremum search.

[0029] Furthermore, analysis using the formulas of a dither signal-driven extreme value control algorithm reveals that the optimal control device actually estimates the average gradient at an operating point such as point A shown in Figure 2. Using this gradient information, it determines whether to decrease or increase the manipulated variable during control operation (the direction in which the manipulated variable should move) and the magnitude of the decrease or increase (the amount by which the manipulated variable should move).

[0030] The optimal control device of this embodiment improves the stability and control performance of the extreme value control described above. Figure 3 is a schematic block diagram showing one example configuration of the optimal control device of the first embodiment. The controlled object 200 is any process in any plant having an manipulated variable input U and a process output Y. In this embodiment, the controlled object 200 is, for example, the coagulant injection process of a water treatment plant shown in Figure 1, and the manipulated variable input U is the target value of the moving speed (SV) in the coagulant injection process. The actual manipulated variable in the coagulant injection process is the PAC injection rate, but as mentioned above, since this control system has a two-stage cascade configuration, the manipulated variable adjusted by extreme value control is the target value of the moving speed.

[0031] Furthermore, the water treatment plant is equipped with various sensors to detect the value of output Y. The output Y of the coagulant injection process is, for example, the injection rate of coagulant (PAC), the sludge concentration in the sludge tank, the sludge flow rate, the washing frequency of the grit tank, the sludge concentration at the outlet of the sedimentation tank, and the sludge concentration at the outlet of the water treatment tank.

[0032] The optimal control device 100 of this embodiment is, for example, an arithmetic unit comprising at least one processor and a memory in which a program executed by the processor is recorded. The memory of the optimal control device may be provided with a storage area (storage unit) for temporarily storing data (for example, time-series data of measured values ​​or evaluation values) while the program is being executed. The optimal control device can realize various functions described below by software, or by a combination of software and hardware.

[0033] The optimal control device 100 includes a process measurement value acquisition unit 110, a phase difference calculation unit 300, a process evaluation value calculation unit 120, a phase difference compensation manipulated variable calculation unit 130, an evaluation function gradient estimation unit 140, a gradient code information estimation unit 150, an extreme value search unit (optimal manipulated variable adaptation adjustment unit) 160, a dither signal generation unit 170, a dither signal switching unit 180, and a manipulated variable output unit 190.

[0034] The process measurement value acquisition unit 110 acquires measurement information necessary for calculating evaluation quantities and constraints. For example, if the sum of chemical costs, sludge disposal costs, and filtration pond cleaning costs is set as the operating cost as the evaluation function, the process measurement value acquisition unit 110 acquires measurement information such as the PAC injection rate, sludge concentration in the sludge tank, sludge flow rate, and cleaning frequency of the grit tank. Furthermore, if constraints are placed on the sludge concentration at the sedimentation tank outlet and the water purification tank outlet, the process measurement value acquisition unit 110 also acquires these sludge concentration measurements at predetermined intervals and saves them as time-series data in a predetermined format.

[0035] The process evaluation value calculation unit 120 uses the information acquired by the process measurement value acquisition unit 110 to calculate the evaluation value of a preset evaluation function in real time. Operating costs are defined as the sum of chemical costs (PAC costs), sludge disposal costs, and filtration pond cleaning costs. The process evaluation value calculation unit 120 can convert chemical costs by multiplying the time-series data of the PAC injection amount by coefficients such as the chemical unit price and dilution ratio, and can obtain this as time-series data.

[0036] The process evaluation value calculation unit 120 can calculate the amount of sludge generated by multiplying the sludge concentration and sludge flow rate in the sludge tank, and obtain time-series data of sludge disposal costs by multiplying the amount of sludge generated by the sludge disposal unit price. Furthermore, cleaning of the filtration tank is usually performed when the filtration resistance exceeds a predetermined threshold. From this, the process evaluation value calculation unit 120 can determine the timing of cleaning from the time-series data of filtration resistance and cleaning history data, and can obtain the cleaning cost at the time of cleaning from the history data. Since cleaning costs can be considered as the costs incurred from the previous cleaning until the next cleaning, the process evaluation value calculation unit 120 can convert the calculated cleaning costs into time-series data by evenly distributing them over that period.

[0037] However, while converting cleaning costs into time-series data is easy for historical data, in real time, the timing of the next cleaning is uncertain. Therefore, it is necessary to monitor the rate of change in filter resistance, estimate the timing of the next cleaning, and then convert the estimated timing into real-time time-series data for cleaning costs. The sum of each cost converted into time-series data as described above represents the operating cost, which can be obtained in real time using the method described above.

[0038] On the other hand, water quality constraints are obtained by acquiring turbidity data at the sedimentation tank outlet and the water purification tank outlet in Figure 1 as time-series data, and incorporating turbidity constraint values, such as Tlim = 0.8 degrees or less. In extreme value control, constraint conditions cannot be directly handled, so they can be converted into evaluation functions, for example, using the method in Non-Patent Document 4. Here, we treat them as evaluation functions using the concept of penalty functions, which is well known in the field of optimization. That is, for example, the conversion is performed as follows.

[0039] Wcost = max(0, a × (exp(T - Tlim) - 1)) ... (1) Here, T is the turbidity measurement value, Tlim is the turbidity upper limit value, and a > 0 is the design parameter. Wcost is the converted evaluation function, and the water quality cost is obtained by applying the conversion of equation (1) to the turbidity at the sedimentation tank outlet and the water treatment tank outlet, and then summing them up.

[0040] The meaning of equation (1) above is that it is 0 when turbidity T ≤ Tlim, and that the cost increases rapidly (exponentially) when it exceeds Tlim, and is a type of so-called penalty function. In addition to the water quality cost considered by this penalty function to the operating costs mentioned above, the total cost is defined as follows: Total cost J = Chemical cost + Sludge disposal cost + Filtration tank cleaning cost + Water quality cost ... (2)

[0041] In the coagulant injection control shown in Figure 1, the above evaluation function was set, but depending on the problem setting, it may not be possible to separate the target plant 200 and the process evaluation value calculation unit 120. For example, in a wind power plant, when applying extreme value control to control the wind turbine blades to move in accordance with the wind direction to maximize power generation, the evaluation function J is the amount of power generated and the manipulated variable U is the rotation angle of the wind turbine blades, but there is no need to specifically define the plant's output Y. In such problem settings, there are cases where the output Y and the evaluation function J are not distinguished, and the output Y is not necessarily defined and measured separately from the evaluation function J.

[0042] On the other hand, as mentioned above, by individually defining the optimization evaluation function, it may be possible to apply extreme value control to certain cases. In any case, by appropriately setting the evaluation function, the process evaluation value calculation unit 120 can obtain evaluation quantities that change moment by moment by measuring and calculating the value of the evaluation function in real time at a predetermined control cycle.

[0043] The phase difference compensation manipulated variable calculation unit 130 first calculates an estimated amount of phase delay between the manipulated variable input to the controlled object 200 and the controlled variable acquired by the process measurement value acquisition unit 110, or between the manipulated variable input to the controlled object 200 and the evaluated variable calculated by the process evaluation value calculation unit 120. In other words, in this embodiment, the estimated amount of phase difference between the manipulated variable and the controlled variable, or between the manipulated variable and the evaluated variable, is calculated online directly without the use of parameters. The advantage of calculating the estimated amount of phase difference as described above is, firstly, that it becomes possible to provide an extreme value control system that is easy to incorporate into a PLC (Programmable Logic Controller).

[0044] For example, when estimating the phase lag, if the parameters of a linear transfer function, or its special case, a first-order lag + dead time system (polynomial coefficients of the transfer function, or time constants and dead time), are to be calculated in advance off-run, or if these parameters are to be calculated online, it becomes necessary to identify the plant's dead time and time constants offline in advance, for example, using step response tests or system identification tests, or to introduce a parameter estimation mechanism to estimate these parameters online. In this case, if the extreme value control function is to be incorporated into the PLC, it is unavoidable that the control system configuration will become somewhat complex. Furthermore, it is necessary to compensate for the phase shift through a linear transfer function using the estimated (identified) parameters.

[0045] However, the function in this embodiment to calculate the estimated phase difference online, as described later, does not require parameter identification testing and can be incorporated into a PLC with a simpler configuration compared to the parameter estimation mechanism. In other words, it can be realized by shifting the manipulated variable and the controlled variable, or the manipulated variable and the evaluated variable, by a phase difference, making it easy to incorporate the function into a PLC.

[0046] The following describes three specific implementation methods. The first implementation method is the most intuitively understandable. First, the phase difference compensation manipulated variable calculation unit 130 calculates the correlation between the input data corresponding to the manipulated variable and the controlled variable, or the correlation coefficient between the input data corresponding to the manipulated variable and the output data corresponding to the evaluated variable, while gradually shifting the input waveform. Alternatively, the phase difference compensation manipulated variable calculation unit 130 calculates the cross-correlation function between the input data and the output data. Then, it finds the shift amount ΔT that maximizes the value of the cross-correlation function as the delay time Lφ corresponding to the phase difference. That is, the phase difference compensation manipulated variable calculation unit 130 finds Lφ = argmax_ΔT {cross-correlation function between input data U and output data Y}. Converting this time Lφ to phase φ would give the phase difference, but since the delay time Lφ itself is actually used, there is no need to convert it to a phase difference.

[0047] The second implementation method utilizes the concept of a so-called phase comparator. A phase comparator is a process incorporated into circuits such as phase-locked loops (PLLs), and an example of its principle is explained below.

[0048] Figure 4 is a schematic diagram illustrating an example of the phase difference between the input data corresponding to the manipulated variable and the output data corresponding to the evaluated variable.

[0049] Figure 4 compares two signals when the input and output data are square waves, for simplicity. The part labeled "phase difference" in the figure represents the phase difference (the time difference corresponding to the phase difference), and it is easy to detect the time difference corresponding to that phase difference from the input and output data. For example, in the case of Figure 4, if the values ​​of the input and output data are normalized to be in the range of 0 to 1, and the absolute value of the difference is taken, the phase difference detection data shown in the lower section can be obtained. Alternatively, the same phase difference data can be obtained by multiplying the normalized signals and inverting 0 and 1. The phase difference compensation operation amount calculation unit 130 can generate phase difference data with simple arithmetic operations as described above.

[0050] The phase difference compensation operation amount calculation unit 130 may generate phase difference data in a similar manner to the exclusive OR (XOR) circuit used in so-called digital PLLs, or using a phase-frequency comparator (PFD). This is the phase comparator part of the digital PLL circuit configuration and operates on almost the same principle as the arithmetic operation method described above.

[0051] For a rectangular digital waveform like that shown in Figure 4, this principle can be used directly. However, when a dither signal (described later) such as a sine wave is applied in a continuous-time system, the phase difference can also be detected using an analog multiplier and a filter. Assume that the manipulated variable data is driven by a sine wave, and the controlled variable (or evaluated variable) data is also operated by a sine wave of the same frequency with a phase delay φ. For example, assume that the input data and output data are driven by the waveforms shown in equations (3) and (4) below.

[0052] U(t) = A1sin(ωt) ... (3) Y(t) = A2sin(ωt - φ) ... (4) Here, the signal Z(t) obtained by multiplying these signals is given by equation (5) below.

[0053] Z(t) = U(t) × Y(t) = A1sin(ωt) × A2sin(ωt-φ) = (A1 × A2) / 2 × (cos(ωt-(ωt-φ)) - cos(ωt+ωt-φ)) = (A1 × A2) / 2 × (cosφ - cos(2ωt-φ)) ... (5) If a low-pass filter that can block frequencies twice the dither frequency ω is inserted after equation (5), we get the following equation (6).

[0054] Z(t) ≈ (A1 × A2) / 2 × cosφ := Zf(t) ... (6) The phase difference compensation operation amount calculation unit 130 can calculate an estimated value of the phase difference φ using the above Z(t). This estimation method is a phase estimation method based on the concept of demodulation, which is often used in extreme value control systems. In practice, instead of directly using the phase difference φ, the delay time L corresponding to the phase difference is used. φ Alternatively, the phase difference φ between the input signal U(t) and the output signal Y(t) may be adjusted.

[0055] Figure 5 illustrates an example of a configuration for calculating an estimated phase difference between an input signal and an output signal. The phase difference compensation manipulated variable calculation unit 130 includes, for example, the circuit shown in Figure 5, and can calculate an estimated time difference between the manipulated variable and the controlled variable. The calculated estimated time difference can be corrected to adjust and synchronize the phase difference between the manipulated variable and the controlled variable. This method is particularly effective when the dither signal is generated using a sine wave signal. This is a basic method used in so-called analog PLL phase comparators, and more advanced phase comparators may also be used.

[0056] The above methods using phase comparators in PLLs are not limited to those described above. For example, a simpler method is the zero-crossing method, which detects the difference in the zero-crossing points when two signals pass through zero. Also, although it is not usually suitable for online processing and therefore rarely used in online processing, it is possible to use a phase comparator that applies an FFT (Fast Fourier Transform) to each of the two signals and detects the phase difference.

[0057] While the zero-crossing method has limitations in terms of accuracy, and the FFT method has challenges in real-time processing, these methods are not necessarily the optimal solution. However, they can be applied as needed.

[0058] Alternatively, a so-called dual phase-locked circuit can be applied to the two signals—the manipulated variable signal and the controlled variable signal—to generate signals synchronized with the manipulated variable and the controlled variable, respectively, and then the phase difference detection method described above is applied to these two signals. While this method makes the mechanism for detecting the phase difference slightly more complex, it enables more stable phase difference estimation. The third method involves using a Kalman filter or observer.

[0059] Figure 6 illustrates an example of a configuration for calculating an estimated phase difference between an input signal and an output signal. In extreme value control systems, a sine wave of a certain frequency is often used as the dither signal, and this method is applicable when a sine wave is used as the dither signal. When a sine wave is used as the dither signal, the waveform of the control input can be approximately represented as follows, as shown in Figure 6.

[0060] u(t) = u ０ (t) + Asinωt ... (7) Here, u ０ (t) is a signal that gradually changes toward an extremum through extremum search, and Asinωt is a sinusoidal signal excited by the dither signal. Here, we consider the control input u(t) as the reference, so we use sinωt without loss of generality. At this time, the output signal (evaluation function value) observed through the controlled object may have harmonics superimposed if the controlled object is nonlinear, so it can be approximately written as follows.

[0061] Here, y ０(t) is a signal of the evaluation value that gradually changes toward an extreme value by extreme value search. The second term on the right side of equation (8) is a signal generated by the input of the dither signal. Here, it is assumed that it has harmonics up to an appropriate Nth order. When the characteristics of the controlled object are close to linear, no harmonics occur. In this case, the second term of equation (8) is a １ cos ωt + b １ sin ωt. At this time, the phase shift (delay) is expressed by the appearance of both sin ωt and cos ωt on the right side of equation (8).

[0062] Note that the values of a ｎ and b ｎ cannot be directly measured. Also, u ０ (t) and y ０ (t) are signals that change according to the extreme value search. However, compared with the dither period ω, they are sufficiently gentle. When viewed on the time scale of the dither period, they can be regarded as constant values without any problem. Here, u := u ０ (t), a ０ := y ０ (t) are defined and regarded as constants.

[0063] At this time, if the estimated values of a １ and b １ can be calculated from the output signal, the phase difference (phase delay) of the output signal with respect to the input signal is Φ = tan －１ (b １ / a １ ) and can be obtained.

[0064] To calculate the estimated values of a １ and b １ , an observer or a Kalman filter may be configured for the output signal of equation (8). At this time, not only a １ and b １ but also the estimated values of a ｋ and b ｋ (k = 2, 3,... N) need to be calculated simultaneously. a １ , b １ , a ｋ , b ｋAn observer / Kalman filter can be constructed to simultaneously calculate (k = 2, 3, ..., N), but if you want to avoid increasing the dimensionality, you can construct the observer / Kalman filter using, for example, the methods (1) and (2) below.

[0065] (1) If it is thought that no harmonics are included, consider N=1 and a １ and b １ Only estimate the following. (2) If harmonics are thought to be present, a low-pass filter is constructed in which the frequency ω of the dither signal falls within the passband but the second harmonic 2ω and above are blocked, and a is applied to the signal passed through this low-pass filter. １ and b １ This constructs an observer / Kalman filter that considers only the fundamental harmonic components.

[0066] The following shows the configuration of an observer / Kalman filter that includes only the fundamental harmonic using method (1) or (2). First, the state variable x = [a ０ , a １ , b １ Define ]. Assume that these do not change on the time scale of ω. Then the equation of state is as follows: dx / dt = 0 + w ... (9) Here, w is the system noise of the Kalman filter, and if the Kalman filter is not used as an observer, w = 0 may be set. Next, the output equations can be expressed as shown in equations (10) and (11) below.

[0067] y(t) = C(t)x(t) + v ... (10) C(t) = [1, cosωt, sinωt] ... (11) Here, v is the observation noise of the Kalman filter, and if the Kalman filter is not used as an observer, v = 0 may be used. An observer or Kalman filter can be constructed for these equations, but if a Kalman filter is constructed, it can be constructed as shown in equations (12), (13), and (14) below.

[0068] x ｅ (t)=K(t)(y(t)−C(t)x ｅ )…(12) K(t)=P(t)C Ｔ (t)R －１...(13) d / dt(P(t))=Q-K(t)RK Ｔ (t) ... (14) Here, Q and R are the covariance matrices of the system noise and the observed noise, respectively, and are adjustable parameters. Also, x ｅ (t) is an estimate of x(t), P(t) is an intermediate variable (the covariance matrix of the state), and K(t) is the observer gain, also known as the Kalman gain. If a Kalman filter is not used, K(t) will be adjusted directly.

[0069] With the above configuration, x = [a ０ , a １ , b １ Since an estimated value of ] is obtained, this estimated value is used to determine the phase delay Φ = tan －１ (b ｅ１ / a ｅ１ ) is calculated, and this is used for the delay time L Φ When converting to L Φ This can be converted to Φ / ω. Note that in this observer / Kalman filter configuration, C(t) = [1, cosωt, sinωt] is required, but this is considered based on the process control signal.

[0070] Therefore, one may input a sine wave that has been pre-adjusted to synchronize with the process control variable, or one may consider the process control variable itself normalized by amplitude as sinωt, and then use a signal that is considered as cosωt by introducing a π / 2 phase delay circuit (note that cosωt = sin(ωt + π / 2)). Alternatively, one may configure a phase-locked circuit that takes the process control variable as input, normalize its output so that its amplitude is 1, and use a signal that is considered as cosωt by introducing a π / 2 phase delay circuit to this signal.

[0071] The estimated value of the phase difference Φ is calculated using the three methods described above, and the L corresponding to this estimated phase difference is calculated. Φ By applying a delay circuit equal to Φ / ω to a process control variable, it is possible to generate a phase-adjusted (phase-adjusted) process control variable whose phase is delayed by Φ and has no phase difference with the process evaluation value.

[0072] Figure 7 is a schematic diagram showing an example of the process evaluation amount and process operation amount before phase difference adjustment. Figure 8 is a schematic diagram showing an example of the process evaluation amount and process operation amount after phase difference adjustment. The configuration of the phase difference compensation operation amount calculation unit 130 described above enables online phase compensation, and even if a change occurs in the phase difference between the process operation amount and the evaluation amount for any reason, it becomes possible to compensate for that phase difference in real time. As a result, for example, even if there is a phase difference between the process evaluation amount y(t) and the process operation amount u(t) as shown in Figure 7, the phase difference compensation operation amount u'(t) is adjusted so that the phase of the process evaluation value y(t) and the phase difference compensation operation amount u'(t) are aligned, as shown in Figure 8.

[0073] The evaluation function gradient estimation unit 140 uses the time-series data of process evaluation values ​​output from the process evaluation value calculation unit 120 and the time-series data of phase-compensated process manipulated variables output from the phase difference compensation manipulated variable calculation unit 130 to estimate the gradient of the evaluation function.

[0074] This gradient estimation process is a process that determines the gradient of the process evaluation value with respect to the manipulated variable. The evaluation function gradient estimation unit 140 may, for example, calculate the gradient of the evaluation function using a demodulation dither signal, or it may directly calculate an estimated value of the gradient without using a demodulation dither signal. The latter method directly uses the phase difference compensation manipulated variable u'(t), while the former method is an indirect method that adjusts the phase of the demodulation dither signal without directly using the phase difference compensation manipulated variable u'(t).

[0075] Here, we will explain assuming that the evaluation function gradient estimation unit 140 calculates the estimated gradient using the latter method. As shown in Figure 2, when the process manipulated variable is driven by a dither signal (which does not necessarily have to be a sine wave or a periodic signal, but for the sake of brevity of explanation we will consider a sinusoidal dither), if there is no dynamics in the controlled object, as shown in the left diagram of Figure 2, the process evaluation value moves in opposite phase to the process manipulated variable on the left side of the extreme value (local minimum), and moves in the same phase on the right side. Therefore, by using the time-series data of the two signals, the process manipulated variable and the process evaluation value, the slope at the operating point A can be found, which becomes the gradient of the evaluation function.

[0076] However, in reality, the controlled system exhibits dynamics, and the value of the evaluation function responds with a delay to the process manipulated variable, resulting in a state similar to that shown in Figure 2 (right). When this phase shift exceeds 90°, the left and right sides of the extreme values ​​are reversed, meaning that the correct gradient cannot be obtained by directly using the process manipulated variable and the process evaluation value to determine the slope.

[0077] In this embodiment, taking into account the above-mentioned phase lag, the phase difference compensation operation amount calculation unit 130 generates a phase-compensated process operation amount with the phase difference eliminated. Using this, even if the controlled object has dynamics, the relationship shown in the left diagram of Figure 2 can be maintained between the phase-compensated process operation amount and the process evaluation value, so the evaluation function gradient estimation unit 140 only needs to directly determine the slope at the operating point A. In this case, the overall shape of the evaluation function shown in Figure 2 (the downward-convex graph in Figure 2) is unknown when actually performing extreme value control, but the evaluation function gradient estimation unit 140 can acquire values ​​near the operating point A in real time during control execution. It is clear that if the evaluation function gradient estimation unit 140 can acquire data near the operating point A, that is, time-series data of the most recent operation amount and evaluation amount during control execution, it is possible to calculate a straight line representing the slope as shown in Figure 2 from the data.

[0078] From this perspective, extreme value control that directly estimates the slope of the line in Figure 2 as the gradient of the evaluation function has been used, for example, in Non-Patent Document 2, Non-Patent Document 3, and Patent Document 2. The basic idea behind these methods is extremely simple: predetermined past time series data of the manipulated variable and evaluation variable around the time of control implementation are accumulated, and the coefficients of the line in the following equation are obtained using regression algorithms such as the least squares method or successive least squares method.

[0079] J = aU + b ... (15) Here, J is the evaluation quantity (process evaluation value) in Figure 2, and U is the manipulated quantity in Figure 2. a and b are coefficients of linear regression, and these coefficients are obtained using algorithms such as the least squares method. Once the coefficients of the line are obtained in this way, as is clear from Figure 2, the slope of the line can be considered to be equal to the gradient of the evaluation function, and the coefficient a corresponding to the slope can be used as gradient information of the evaluation function.

[0080] In this embodiment, the coefficients are calculated using equation (16) below instead of equation (15).

[0081] J = aU ｆ +b...(16) Here, U ｆ This is not a manipulated variable, but a manipulated variable that has been phase-compensated and generated by the phase-compensated manipulated variable calculation unit 130. Furthermore, applying the "sequential least squares method with a forgetting factor" to parameter a in equation (16), i.e., the estimated value of the gradient, is a preferred method due to the accuracy of the gradient estimation and the ease of implementation in the controller.

[0082] The following describes how to apply the "sequential least squares method with a forgetting factor." Using equation (16), gradient estimation can be performed using phase-compensated manipulated variables and process evaluation values ​​to eliminate the effect of phase difference and obtain a more accurate gradient estimation. However, considering that the target process is a process with dynamically changing dynamics, the estimation accuracy may vary greatly depending on the implementation method.

[0083] For example, when applying the usual least squares method (LS) to equation (16), the estimation accuracy changes depending on the length of data used in the LS. In other words, it is generally thought that the more data used, the better the estimation accuracy. However, increasing the amount of data in the "time series data" means dragging in longer periods of past data, and considering that the operating point A in Figure 2 also changes over time during the extreme value search process, using a large amount of data can lead to calculating the average gradient from the past to the present, rather than the gradient at the current operating point, which can actually worsen the gradient estimation accuracy.

[0084] To prevent degradation of gradient estimation accuracy, it is necessary to apply LS using data of an appropriate length. Furthermore, since this "appropriate length" of data is related to the convergence speed—how quickly the operating point converges to the extremum in extremum control—determining the appropriate data length is not easy, and adjustment of the data length may be necessary as needed. This becomes a problem when implementing extremum control in controllers such as PLCs, and implementing a mechanism for variable data length in a PLC increases implementation costs. In this respect, using successive least squares (RLS) rather than the usual least squares method is preferable from an implementation standpoint.

[0085] In successive least squares, the estimated gradient is fed back to the estimation mechanism, indirectly considering the influence of past inputs. Therefore, in actual implementation, only "data of the current phase-compensated manipulated variable and process evaluation value" is required, making RLS overwhelmingly superior to LS from the perspective of PLC implementation. However, since a normal RLS corresponds to an LS with infinitely long historical data, it is difficult to track changes in the operating point due to extreme value search. Therefore, by introducing "RLS with a forgetting factor," it becomes possible to adjust the period that should be considered when calculating the gradient estimate.

[0086] This can be adjusted with a single parameter called the forgetting coefficient. When the operating point A changes rapidly (converges quickly to the extremum), adjusting the forgetting coefficient to a value close to 0 allows for quicker forgetting of past information, thus enabling adaptation to changes. Conversely, when the operating point A changes slowly and convergence to the extremum is slow, adjusting the value of the forgetting coefficient to a value close to 1 allows for the calculation of a more stable gradient estimate. In other words, by applying the successive least squares method with a forgetting coefficient to gradient estimation, it becomes possible to easily adjust instantaneous gradient estimation and average gradient estimation over a certain period with a single parameter called the forgetting coefficient, thereby realizing extremum control that is easy to adjust and can be implemented in a PLC.

[0087] By simply introducing a single parameter called the forgetting coefficient into the evaluation function gradient estimation unit 140, the above adjustment can be implemented without worrying about memory allocation when implementing it on a PLC or the like.

[0088] The gradient sign information estimation unit 150 estimates the sign of the gradient of the evaluation function. In this embodiment, the gradient sign information estimation unit 150 does not directly use the "gradient of the evaluation function," but rather uses the direction (sign) of the gradient, ignoring the magnitude of the gradient. Using the direction (sign) of the gradient makes it easy to adjust various parameters of the extreme value control to improve the control performance (convergence speed) of the extreme value control.

[0089] In the example shown in Figure 2, the shape of the evaluation function representing the relationship between the manipulated variable and the evaluated variable is unknown. In reality, it is not necessarily a neat downward-convex function as shown in Figure 2, and the gradient may be extremely steep or extremely gentle depending on the operating point A. Therefore, in extreme value control, it is not possible to know the shape of the evaluation function in advance when the control is executed. For this reason, extreme value control estimates the gradient of the evaluation function in real time. If the gradient is gentle, the control can be applied strongly, but if the gradient is steep, applying strong control risks compromising stability. As a result, when constructing extreme value control based on gradient estimation information, it is necessary to estimate and predict the steepest gradient among the expected gradients and adjust the parameters of the extreme value control accordingly. Consequently, the setting of control parameters must be conservative in order to avoid compromising stability, making it difficult to improve control performance (convergence speed).

[0090] However, as can be seen from the explanatory diagram of extremum control shown in Figure 2, it is not always necessary to estimate and know the gradient itself in order to search for extrema. In principle, if we can obtain only the information on the direction of the gradient, that is, the sign (whether to increase or decrease the manipulated variable), we should be able to search for extrema. For example, Non-Patent Literature 1 clarifies this fact and therefore states that it is preferable to use the sign information rather than the gradient information. Therefore, if we use the following equation as the normalization signal for the gradient estimate, this ultimately boils down to estimating the sign of the gradient.

[0091] RS = 1 / |G(t)| ... (17) Here, RS represents the normalized signal, and |G(t)| is the absolute value of the gradient estimate of the evaluation function. Using this as the normalized signal, the signal G is obtained by normalizing the gradient G(t). ｎ (t) is the sign function of the gradient. However, if we use this function as is, G(t) and G ｎ Since there is a discontinuity at the origin of the plane representing the relationship (t), it is preferable to smooth equation (17) slightly. Various methods can be used to smooth equation (17), but for example, by introducing a regularization constant δ as shown in the following equation, a normalized signal like the following can be obtained.

[0092] RS = 1 / (δ + |G(t)|) ... (18) The normalized signal G obtained by applying the normalized signals from equations (17) and (18) to the gradient G(t) ｎ Using (t) allows for the extraction of the sign of the gradient or the approximate gradient sign, which is essential in this invention. In particular, considering the ease of implementation on a PLC, not only a soft sign function like equation (18) but also a saturation function (a function truncated to ±1) or a hyperbolic tangent function for a simpler gradient estimate G(t) may be applied.

[0093] The extremum search unit 160 performs an extremum search. The extremum search unit 160 controls the extremum search using, for example, an integrator. That is, the search can be performed by multiplying the gradient estimate or normalized gradient estimate by a constant called the integral gain K through the integrator. This integral gain K is an important parameter that determines the convergence speed of the extremum search, but by introducing the phase difference compensation and sign function shown in this embodiment, the convergence speed of the extremum search itself can also be adjusted. That is, if the phase difference is perfectly guaranteed, the gradient is accurately estimated, and the sign of the gradient is extracted using an ideal sign function, the convergence speed of the extremum search will strictly match the integral gain K.

[0094] This method is almost identical to an optimization algorithm called the steepest descent method, a type of gradient method. The integral gain K corresponds to a parameter called the learning rate in gradient method algorithms, and by using the sign of the gradient instead of the gradient itself, it becomes possible to precisely adjust the convergence speed. Furthermore, by using a method that is a slightly modified version of the steepest descent gradient method represented by an integrator (an integral type of adjustment rule), it is possible to fine-tune the extremum search performance even more precisely, and this can be achieved using various gradient method variants known in the field of adaptive control.

[0095] Specifically, by using a robust adaptive adjustment rule that feeds back the gradient using a method called the σ correction method, the shape of convergence to the extremum can be adjusted, making it possible to fine-tune the behavior, especially near the extremum. Furthermore, by using a robust adaptive adjustment rule called the dead zone method, the adjustment can be stopped when the vicinity of the extremum is reached, thus avoiding the manipulated variable continuing to operate near the extremum. Additionally, by adopting a robust adaptive adjustment rule called the projection method, it becomes possible to search for the optimal manipulated variable while incorporating constraints on the manipulated variable and its change. This allows for precise extremum search, considering transient characteristics, steady-state characteristics, and manipulated variable constraints, with only the addition of simple processing that can be implemented in a PLC.

[0096] The dither signal generation unit 170 generates and outputs a dither signal as a forced input from an external source. In the optimal control device 100 of this embodiment, the dither signal generated by the dither signal generation unit 170 is used to forcibly change the data used when regression estimation of a (and b) using RLS or the like in equation (16).

[0097] Furthermore, the extreme value search unit 160 can perform extreme value search even without a dither signal input, in which case the dither signal generation unit 170 may be omitted. When there are fluctuations in the data used for regression estimation, applying a dither signal is not essential, and the dither signal generation unit 170 can be omitted. For example, in the water purification process targeted in this embodiment, the controlled object may show fluctuations in the manipulated variable and evaluation value due to disturbances such as fluctuations in the water intake source. Alternatively, in sewage treatment processes, the amount and quality of incoming sewage cannot be controlled, so the manipulated variable and evaluation value are often constantly fluctuating, driven by fluctuations in the inflow volume and inflow water quality. In such cases, applying a dither signal would be an unnecessary fluctuation input, and it may be better not to apply it.

[0098] On the other hand, if the manipulated variable is not fluctuating due to disturbances, the manipulated variable may remain at a certain value or nearly constant. In such cases, it is desirable to forcibly input an external signal for extremum search to improve the estimation accuracy of the regression parameters a (and b). Therefore, the dither signal generation unit 170 generates a dither signal using a periodically changing signal such as a sine wave or square wave, or a signal containing a wide range of frequency components such as a random number or an M-sequence signal, as a signal to be forcibly driven to improve the parameter estimation accuracy.

[0099] However, since the optimal control device 100 of this embodiment has a phase difference estimation mechanism, it is more preferable to use a dither signal with a constant periodicity, such as a sine wave or a square wave, rather than a dither signal that moves randomly.

[0100] The dither signal switching unit 180 switches the application of the dither signal supplied from the dither signal generation unit 170 to the manipulated variable on or off. If the signal generated by the dither signal generation unit 170 is always used as the dither signal, it may introduce unnecessary disturbances if the manipulated variable is fluctuating due to external disturbances. Therefore, in this embodiment, a predetermined minimum amplitude A is set for the manipulated variable in advance, and if the fluctuation of the manipulated variable is greater than or equal to this minimum amplitude A, the application of the dither signal is stopped, and if the maximum value of the fluctuation of the manipulated variable falls below A, the dither signal is applied. This is provided as a dither signal on / off switch function (dither signal switching unit 180).

[0101] The dither signal switching unit 180 may immediately stop (turn off) the application of the dither signal when the maximum value of the fluctuation of the manipulated variable becomes A or greater, or it may stop (turn off) the application of the dither signal when the number of times the maximum value of the fluctuation of the manipulated variable becomes A or greater during a predetermined period becomes equal to or greater than a threshold. The dither signal switching unit 180 may immediately apply (turn on) the dither signal when the maximum value of the fluctuation of the manipulated variable becomes less than A, or it may apply (turn on) the dither signal when the number of times the maximum value of the fluctuation of the manipulated variable becomes less than A during a predetermined period becomes equal to or greater than a threshold.

[0102] To make the above-mentioned dither signal switching unit 180 more robust, for example, the maximum amplitude A when applying the dither signal with a hysteresis function applied to the amplitude A that determines the switching ｏｎ And the minimum amplitude A when the application of the dither signal is stopped. ｏｆｆ You may set it to and.

[0103] Furthermore, in the dither signal switching unit 180, the amplitude A used to determine whether to apply the dither signal may be made variable according to the magnitude of the gradient estimated by the evaluation function gradient estimation unit 140. Also, even when the manipulated variable is changing due to a disturbance, measures may be taken to turn on the application of the dither signal in the case of a disturbance signal that completely disrupts the periodicity.

[0104] On the other hand, if the disturbance signal is driven by a disturbance that has some kind of daily fluctuation pattern, such as the amount of sewage inflow, the quality of the inflow water, or the quality of the water taken for purification, the dither signal switching unit 180 may be set to forcibly turn off the input of the dither signal.

[0105] The manipulated variable output unit 190 outputs a manipulated variable command for the controlled object 200. This manipulated variable command signal is either the output of the extremum search unit 160 itself, or the output of the extremum search unit 160 plus a dither signal. This drives the manipulated variable and performs the search for extrema.

[0106] Here, an example of extreme value control using the comparative example's optimal control device will be explained using Figures 9 to 11. Figure 9 is a graph showing an example of the change over time (time series data) of the target value (manipulated variable) SV of the moving speed adjusted by value control. Figure 10 shows an example of time series data of sedimentation tank turbidity. Figure 11 shows an example of time series data of total cost used as an evaluation function. In Figures 9 to 11, the results of extreme value control using the comparative example's optimal control device are shown by dotted lines, and the results when the SV value is not adjusted to a constant value are shown by solid lines.

[0107] Comparing the results of extreme value control using the comparative example's optimal control device with the results when the SV value is kept constant, it can be seen that in the first half, the SV is adjusted by extreme value control to reduce the total cost, which is the evaluation function, even if it slightly increases the sedimentation tank turbidity, and in the second half, the SV value is adjusted by extreme value control to maintain the sedimentation tank turbidity below the water quality standard of 0.8. However, looking at the results of this control system, there is room to operate the system so that in the initial stages of control, priority is given to further reducing costs by lowering the SV value more rapidly, and in the second half, the SV value is quickly raised to more strictly maintain the constraint of sedimentation tank turbidity. In particular, regarding sedimentation tank turbidity, it can be confirmed that there are parts where it exceeds the constraint of 0.8 due to the delay in the operation of the control.

[0108] Figure 12 is a schematic diagram showing the functions of the optimal control device of the first embodiment. Compared to the above comparative example, the inventors of the present invention have found a new control method that improves the control performance of extreme value control, particularly the convergence speed, and converges to the extreme value as quickly as possible. The optimal control device 100 of this embodiment improves the performance of subsequent gradient estimation by adding a phase compensation function (phase compensation circuit), and as a result increases the gain (learning rate) of the extreme value control mechanism, thereby improving the convergence performance to the extreme value. This method has the performance of converging to the extreme value very quickly compared to past methods and is promising for the practical application and development of extreme value control in industry.

[0109] When actually putting this extreme value control system into practical use, it is important to integrate it into a PLC or similar device to realize it as an industrial controller. This can be easily inferred from the fact that more than 90% of the control systems widely used in industry are still PID control systems, which were devised in the 1930s. In other words, PID control is a very old, classic control method that has existed since the 1930s, and although many cutting-edge control systems (advanced control systems) have been proposed since then, and many of them have been thoroughly verified to have excellent performance, the main reason why PID control is used in the majority of control systems is thought to be that PID control can be easily integrated into a PLC as a function block, and compared to other advanced control systems, it has fewer adjustable parameters and allows for intuitive adjustment. Considering this actual situation in industry, it is desirable that extreme value control systems can also be implemented in PLCs as easily as possible, and that the number of adjustable parameters to be adjusted is as small as possible and that adjustment is easy.

[0110] The optimal control device 100 of this embodiment has a simpler configuration, is easier to adjust, allows for flexible response to different environments, and enables the implementation of extreme value control in a PLC, making it easier to use in industry.

[0111] According to the optimal control device 100 of this embodiment, by having a function to compensate online for the phase difference between the manipulated variable and the process evaluation value based on the dynamics of the controlled object, it is possible to increase the convergence speed of extreme value control.

[0112] Specifically, the optimal control device 100 of this embodiment simplifies the control configuration as much as possible by directly estimating the phase difference online, without having to pre-determine the time constant and dead time of the target process using step response, or by having to newly implement a function to estimate these online. In other words, by providing a circuit that compensates for the phase difference online, when implementing an extreme value control system on a PLC, it is possible to realize an extreme value control system that improves convergence and stability by almost automatically compensating for the phase difference without having to set pre-estimated external parameters (such as time constants and dead time) and provide a complex estimation mechanism to identify these parameters online.

[0113] Furthermore, the optimal control device 100 of this embodiment facilitates online implementation, enabling a robust and fast convergence extreme value control system that automatically follows changes in plant dynamics and phase differences depending on the state. In other words, the optimal control device 100 has an online phase difference compensation function, enabling an extreme value control system that adaptively follows changes in plant dynamics and always performs phase compensation correctly.

[0114] Furthermore, in the optimal control device 100 of this embodiment, for example, by applying a successive least squares method with a forgetting coefficient during gradient estimation, it becomes possible to adjust both the gradient estimation and the convergence speed by adjusting only one parameter (the forgetting coefficient), while simultaneously making it easier to implement the extreme value control system on the PLC.

[0115] Furthermore, in the optimal control device 100 of this embodiment, for example, by using various adjustment rules that have been conventionally used in the field of adaptive control in the extreme value search unit of the extreme value control system, it becomes possible to improve the convergence shape of the extreme value search and its robustness (robust search).

[0116] Furthermore, in the optimal control device 100 of this embodiment, for example, by providing a function to determine whether or not it is necessary to apply a forced dither signal from an external source, it is possible to configure an extreme value control system with a fast convergence speed without unnecessarily disturbing the control system.

[0117] In other words, according to this embodiment, it is possible to provide an optimal control device, an optimal control method, and a computer program that realize the search for an optimal value that improves control performance while maintaining stability.

[0118] Next, the optimal control device, optimal control method, and computer program of the second embodiment will be described. In the following description, components similar to those of the first embodiment described above will be denoted by the same reference numerals and their descriptions will be omitted.

[0119] Figure 13 is a block diagram schematically showing one example configuration of the optimal control device of the second embodiment. The target plant 200 is any plant having an manipulated variable input U and a process output Z, but in this embodiment, flocculant injection will be used as an example. Here, the manipulated variable input U is the target value of the transfer speed in the flocculant injection process. The actual manipulated variable is the PAC injection rate, but as mentioned earlier, this control system has a two-stage cascade configuration, so the manipulated variable adjusted by extreme value control is the target value of the transfer speed. The output Y is the PAC injection rate, the sludge concentration in the sludge tank, the sludge flow rate, the washing frequency of the grit tank, the sludge concentration at the outlet of the sedimentation tank, and the next sludge concentration in the water treatment tank.

[0120] Figure 14 is a block diagram schematically showing a partial configuration example of the optimal control device of the second embodiment shown in Figure 3. The optimal control device 100 includes a process measurement value acquisition unit 110, a phase difference calculation unit 300, a process evaluation value calculation unit 120, a phase difference compensation manipulated variable calculation unit 130, an evaluation function gradient estimation unit 140, a gradient code information estimation unit 150, an extreme value search unit (optimal manipulated variable adaptation adjustment unit) 160, a dither signal generation unit 170, a dither signal switching unit 180, and a manipulated variable output unit 190.

[0121] The optimal control device 100 of this embodiment differs from the first embodiment described above in that it includes a phase difference calculation unit 300. The phase difference calculation unit 300 includes a first phase difference compensation process value calculation unit 31, a second phase difference compensation process value calculation unit 32, ... a third M phase difference compensation process value calculation unit 3M.

[0122] The process measurement value acquisition unit 110 acquires measurement information necessary for calculating evaluation quantities and constraints. For example, when setting an operational cost as an evaluation function, which is the sum of chemical costs, sludge disposal costs, and filtration pond cleaning costs, the process measurement value acquisition unit 110 acquires measurement information such as the PAC injection rate, sludge concentration in the sludge tank, sludge flow rate, and grit tank cleaning frequency. Furthermore, if constraints are placed on the sludge concentration at the sedimentation tank outlet and the water treatment tank outlet, the process measurement value acquisition unit 110 also measures the sludge concentration values ​​at the sedimentation tank outlet and the water treatment tank outlet at predetermined intervals and saves them as time-series data in a predetermined format.

[0123] The phase difference calculation unit 300 compensates for the phase difference of multiple (M) process values ​​(partial evaluation values) of the evaluation function obtained using the measurement information acquired by the process measurement value acquisition unit 110. In this embodiment, for example, M = 5, and the five process values ​​included in the evaluation function are the PAC injection amount, sludge concentration, filter resistance change rate, sedimentation tank outlet turbidity, and water treatment tank outlet turbidity.

[0124] Here, the process evaluation value calculation unit 120 uses the information acquired by the process measurement value acquisition unit 110 to calculate the evaluation value of a pre-set evaluation function in real time. The operating cost is defined as the sum of chemical costs (PAC costs), sludge disposal costs, and filtration pond cleaning costs. Chemical costs can be converted into chemical costs by multiplying the time-series data of the PAC injection amount by coefficients such as the chemical unit price and dilution ratio, and this can be obtained as time-series data.

[0125] Since the drug cost has the same dynamics as the PAC injection volume, it has the same phase difference as the phase difference between the target value of the movement speed, which corresponds to the manipulated variable of this extreme value control system, and the behavior of the PAC injection volume. Note that the drug unit price and dilution ratio do not exhibit dynamics. In this embodiment, the phase difference of the drug cost is denoted as Φ1.

[0126] Sludge disposal costs can be calculated by multiplying the amount of sludge generated per unit time by the sludge disposal unit price. The amount of sludge generated per unit time can be calculated by the product of the sludge concentration in the sludge tank and the sludge flow rate per unit time. In other words, time-series data of sludge disposal costs can be obtained using time-series data of generated sludge and the sludge disposal unit price. Therefore, sludge disposal costs have the same dynamics as generated sludge, which is sludge concentration and sludge flow rate, but since the change in concentration is generally slower than the change in flow rate, it is considered that sludge disposal costs are governed by the same dynamics as sludge concentration. Accordingly, the phase difference between the manipulated quantity (target value of movement speed) and sludge disposal costs is approximately the same as the phase difference between the manipulated quantity and sludge concentration, and this is denoted as Φ2.

[0127] Furthermore, since filter pond cleaning is usually performed when the filter resistance exceeds a predetermined threshold, the timing of cleaning can be determined from time-series data of filter resistance and cleaning history data, and the cleaning cost at the time of cleaning can be obtained from the history data. Cleaning can be considered as the cost from the time of the previous cleaning until the next cleaning, so by evenly distributing these cleaning days over that period, the cleaning cost can be converted into time-series data. However, while this conversion can be easily performed using past performance data, in real time, the timing of the next cleaning is uncertain, so it is necessary to monitor the rate of change of filter resistance, estimate the timing of the next cleaning, and then convert that into time-series data of cleaning costs in real time. In this case, since the data monitored is the rate of change of filter resistance, the cleaning cost has almost the same dynamics as the rate of change of filter resistance, so the phase difference between the manipulated amount and the rate of change of filter resistance is set to Φ3.

[0128] As described above, in this embodiment, the time-series data of chemical costs (PAC costs), sludge disposal costs, and filtration pond cleaning costs included in the operating costs each have a different phase difference from the movement speed, which is the operational variable for extreme value search.

[0129] On the other hand, water quality constraints are obtained by acquiring turbidity data at the sedimentation tank outlet and the water purification tank outlet in Figure 1 as time-series data, and incorporating constraints on turbidity, such as Tlim = 0.8 degrees or less. Since the constraints cannot be directly handled in extreme value control, they can be converted into an evaluation function, for example, using the method described in Non-Patent Document 5. Here, the evaluation function can be treated using the concept of a penalty function, which is well known in the field of optimization. That is, the following conversion can be performed, similar to the first embodiment described above.

[0130] Wcost = max(0, a × (exp(T - Tlim) - 1)) ... (19) Here, T is the turbidity measurement value, Tlim is the turbidity upper limit value, and a > 0 is the design parameter. Wcost is the converted evaluation function, and the conversion of equation (19) is performed for each turbidity at the sedimentation tank outlet and the water treatment tank outlet. However, in this case as well, there is a phase difference between the manipulated variable, the moving speed, and the sedimentation tank outlet turbidity, or between the manipulated variable, the moving speed, and the water treatment tank outlet turbidity, so these phase differences are set to Φ4 and Φ5. In the first embodiment, the total cost used for extreme value search was defined as shown in equation (2) below.

[0131] Total cost J = chemical cost + sludge disposal cost + filtration pond cleaning cost + water quality cost ... (2) However, as stated here, there are different phase differences Φ1 to Φ5 between each cost and the target value of the moving speed, which is the operating quantity. Therefore, if the total cost defined in equation (2) is used directly, it becomes difficult to accurately guarantee the phase difference between the various costs that make up that total cost and the target value of the moving speed, which is the operating quantity.

[0132] Therefore, in the optimal control device 100 of this embodiment, the phase difference calculation unit 300 adjusts the phase of each item of the total cost to the largest phase difference among Φ1 to Φ5, using the one with the largest phase difference as a reference, and supplies multiple process values ​​(partial evaluation values) after phase difference adjustment to the process evaluation value calculation unit 120. The one with the largest phase difference among Φ1 to Φ5 needs to be defined in advance by analyzing process measurement information, but considering the physical phenomena in the process shown in Figure 1, for example, it can be assumed that the order in which the phase differences are largest is as follows: Φ5 > Φ4 > Φ2 > Φ1 > Φ3 = 0 ... (20) Here, the rate of change of filter resistance is an indicator of clogging, so it is thought to change the most slowly, but it is unlikely to respond directly to the periodic fluctuation of the manipulated variable with a dither signal, so it is assumed that there is no phase difference with respect to the dither signal. Considering the remaining four from the process flow shown in Figure 1, it is highly likely that the relationship will be as shown in equation (20).

[0133] In practice, a magnitude relationship equivalent to equation (20) may be defined from actual data. In this embodiment, we assume that the relationship in equation (20) is recognized. According to equation (20), the phase difference Φ5 between the turbidity at the outlet of the water purification tank and the target value of the moving speed, which is the manipulated quantity, is the largest, so phase compensation is performed so that the phase difference of the remaining four costs is also Φ5. That is, for chemical costs, the phase difference is compensated by Φ5 - Φ1, for sludge disposal costs by Φ5 - Φ2, for washing costs by Φ5 - Φ3, and for the penalty cost of turbidity at the outlet of the sedimentation tank by Φ5 - Φ4. The mechanism for this phase difference compensation may be exactly the same as the method for compensating the phase difference between the manipulated quantity and the evaluation value described in the previous embodiment. The total cost as an evaluation function is defined as follows.

[0134] Total cost = Phase-compensated chemical cost + Phase-compensated sludge disposal cost + Phase-compensated filter tank cleaning cost + Phase-compensated sedimentation tank turbidity water quality cost + Water treatment tank turbidity water quality cost... (21) Each of the first phase difference compensation process value calculation unit 31 to the M phase difference compensation process value calculation unit 3M calculates each item of the total cost in formula (21). In the above example, M = 5 (including phase difference compensation 0), and the PAC injection amount, sludge concentration, filter resistance change rate, sedimentation tank outlet turbidity, and water treatment tank outlet turbidity correspond to process values. If a standard process value (without phase difference compensation) is set in advance, the phase difference compensation process value calculation unit for the corresponding process value (for example, water treatment tank turbidity water quality cost) may be omitted.

[0135] The process evaluation value calculation unit 120 obtains phase-compensated process values ​​(including the reference process value that is not phase-compensated) from each of the first phase difference compensated process value calculation unit 31 to the Mth phase difference compensated process value calculation unit 3M, and calculates the total cost according to equation (21). Once the total cost is defined in the process evaluation value calculation unit 120 as in equation (21), the phase difference between the manipulated variable and the evaluation value (total cost) will all be unified to Φ5.

[0136] The phase difference compensation manipulated variable calculation unit 130 introduces a delay to the manipulated variable so as to compensate for the phase difference Φ5 between the total cost and the manipulated variable. Here, the phase difference compensation between the process measured value and the manipulated variable, which are components of the evaluation function, is treated conceptually as separate entities. However, in practice, it goes without saying that this can be achieved by simultaneously configuring a phase compensation circuit so that the timing of all time-series data is synchronized, by treating the manipulated variable and other process measured values ​​that constitute the total cost in the same way for the time-series data of the turbidity at the outlet of the water purification plant. The optimal control device 100 of this embodiment is the same as that of the first embodiment described above in terms of configuration other than those mentioned above.

[0137] The optimal control device 100 of this embodiment can achieve the same effects as the first embodiment described above. Furthermore, with the optimal control device 100 of this embodiment, when the evaluation function for extreme value search is defined using process measurement values ​​consisting of multiple different dynamics, the phase difference between the manipulated variable and the evaluated variable can be compensated more precisely and rigorously, resulting in an extreme value control system with improved convergence and stability. In other words, when the evaluation function is defined using multiple process measurement values ​​with different time delays from the manipulated variable, the phase difference between each process measurement value can also be guaranteed individually, enabling more accurate phase delay compensation and significantly improving the convergence speed and stability of extreme value control.

[0138] In other words, according to this embodiment, it is possible to provide an optimal control device, an optimal control method, and a computer program that realize the search for an optimal value that improves control performance while maintaining stability.

[0139] The program according to this embodiment may be transferred while stored on an electronic device, or it may be transferred while not stored on an electronic device. In the latter case, the program may be transferred via a network, or it may be transferred while stored on a storage medium. The storage medium is a non-temporary tangible medium. The storage medium is a computer-readable medium. The storage medium may be any medium that is capable of storing a program and is readable by a computer, such as a CD-ROM or memory card, and its form is not limited.

[0140] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

Claims

1. An optimal control device that searches for the optimal value of an evaluation function by manipulating a manipulated variable in real time based on an manipulated variable of a controlled process and an evaluation value of an evaluation function that indicates an index for optimizing the controlled process based on a controlled variable that changes according to the manipulated variable, comprising: a process measurement value acquisition unit that acquires a measurement value measured in the controlled process; a process evaluation value calculation unit that calculates an evaluation value of the controlled process using the measurement value; a phase difference compensation manipulated variable calculation unit that calculates an estimated amount of phase delay from the manipulated variable to the evaluation value, taking the manipulated variable and the evaluation value as inputs, and adjusts the phase of the manipulated variable so that the phase difference between the manipulated variable and the evaluation value becomes zero; an evaluation function gradient estimation unit that calculates gradient information of the evaluation function, which is the rate of change of the evaluation value with respect to the manipulated variable, using the manipulated variable after phase adjustment and information on the evaluation value; a gradient sign information estimation unit that extracts sign information of the gradient of the evaluation function using the gradient information; and an extreme value search unit that determines the direction and amount that the manipulated variable should move from the sign information.

2. The optimal control device according to claim 1, wherein the evaluation value is the sum of two or more partial evaluation values ​​using a plurality of the measured values, and the device comprises a phase difference calculation unit that calculates a compensated partial evaluation value obtained by compensating so that the phase difference between the two or more partial evaluation values ​​is equal.

3. The optimal control device according to claim 2, wherein the phase difference calculation unit calculates a compensated partial evaluation value based on the phase difference between the two or more partial evaluation values ​​and the manipulated variable, such that the phase difference between the two or more partial evaluation values ​​becomes the reference.

4. The optimal control device according to claim 1, wherein the evaluation function gradient estimation unit calculates the gradient information of the evaluation function by applying a successive least squares method with a forgetting factor to the data of the manipulated variable after phase adjustment and the data of the evaluation value.

5. The optimal control device according to claim 1, wherein the extreme value search unit determines the direction and amount of movement of the manipulated variable using a robust adaptive adjustment rule of the σ correction method, the dead zone method, or the projection method.

6. The optimal control device according to claim 1, comprising: a dither signal switching unit that switches the output of a periodic signal acquired from an external source; and an manipulated variable output unit that applies the output from the dither signal switching unit to the manipulated variable adjusted by the extreme value search unit and outputs it to the controlled process.

7. A method for searching for the optimal value of an evaluation function by manipulating an manipulated variable in real time based on an manipulated variable of a controlled process and an evaluation value of an evaluation function that indicates an index for optimizing the controlled process based on a controlled variable that changes according to the manipulated variable, comprising: acquiring a measured value measured in the controlled process; calculating an evaluation value of the controlled process using the measured value; calculating an estimated amount of phase delay from the manipulated variable to the evaluation value using the manipulated variable and the evaluation value; adjusting the phase of the manipulated variable so that the phase difference between the manipulated variable and the evaluation value becomes zero; calculating gradient information of the evaluation function, which is the rate of change of the evaluation value with respect to the manipulated variable, using the manipulated variable after phase adjustment and information of the evaluation value; extracting sign information of the gradient of the evaluation function using the gradient information; and determining the direction and amount by which the manipulated variable should move from the sign information.

8. A computer program that causes a computer to execute the optimal control method described in claim 7.

Images