Control device

Abstract

[Problem] To provide a control device that adjusts the value of a control parameter while ensuring the stability of a controlled object. [Solution] A control device according to the present embodiment comprises: an acquisition unit that acquires, at predetermined time intervals, a measured value of a control amount for a controlled object that operates in accordance with a manipulated variable provided to the controlled object; a control unit that calculates the manipulated variable for the controlled object on the basis of a deviation between the measured value acquired by the acquisition unit and a target value of the control amount and a first value of a control parameter, and that provides the manipulated variable to the controlled object; a detection unit that detects a change in a control characteristic of the controlled object, on the basis of the deviation between the measured value and the target value; a determination unit that determines a second value of the control parameter by performing machine learning on the basis of the measured value and the target value in response to detection of a change in the control characteristic; and an evaluation unit that evaluates stability of a control system including the control unit and the controlled object, on the basis of the second value of the control parameter and the control characteristic of the controlled object. The control unit calculates the manipulated variable on the basis of the deviation and the second value of the control parameter in response to the evaluation result from the evaluation unit.

Description

control device

[0001] This embodiment relates to a control device.

[0002] In control systems related to social infrastructure, the plants they control are typically operated continuously for long periods. Over time, the control characteristics of a plant change due to factors such as aging and changes in the operating environment. In this case, it is necessary to readjust the control parameters of the control device to accommodate these changes in the plant's control characteristics.

[0003] Several methods have been proposed for adjusting the control parameters of control devices. However, conventional control parameter adjustment methods are performed through process models, requiring significant time and effort for manual work to acquire response performance after the system has been shut down. On the other hand, automatic adjustment of control parameters using machine learning is difficult to guarantee stability. Specifically, changing the control parameters of a control device without prior confirmation of the stability of the control system when applying adjusted control parameters poses a safety problem. Therefore, a method is needed to ensure the stability of automatically adjusted control parameters without shutting down the system.

[0004] Japanese Patent Publication No. 2015-18388

[0005] The objective of this embodiment is to provide a control device that can adjust control parameters while ensuring the stability of the control system without stopping the operation of the controlled object.

[0006] The control device of this embodiment includes: an acquisition unit that acquires a measured value of a control variable of a controlled object that operates in accordance with an operation variable provided to the controlled object at predetermined time intervals; a control unit that calculates the operation variable for the controlled object and provides the operation variable to the controlled object based on the deviation between the measured value acquired by the acquisition unit and the target value of the control variable and a first value of a control parameter; a detection unit that detects a change in the control characteristics of the controlled object based on the deviation between the measured value and the target value; a determination unit that determines a second value of the control parameter by performing machine learning based on the measured value and the target value in response to the detection of a change in the control characteristics; and a determination unit that determines the stability of a control system including the control unit and the controlled object based on the second value of the control parameter and the control characteristics of the controlled object. The control unit calculates the operation variable based on the deviation and the second value of the control parameter in accordance with the determination result of the determination unit.

[0007] This figure shows the configuration of a sewage treatment system as a control system according to this embodiment. This figure shows the configuration of a control system including a control unit and a controlled object. This figure shows the functional configuration of the control parameter determination unit. This is a flowchart for determining the stability of control parameters. This is a block diagram showing the transfer function of the control system. This figure shows an example of a screen image displayed on the display unit. This is a flowchart explaining the operation of the control device according to this embodiment. This figure shows the hardware configuration of the control device according to this embodiment.

[0008] This embodiment will be described below with reference to the drawings. In the drawings, the same or corresponding elements are denoted by the same reference numerals, and detailed descriptions will be omitted as appropriate. Furthermore, the following description will explain an example in which the technology according to this embodiment is applied to a sewage treatment system, but the scope of applicability of the technology according to this embodiment is not limited to sewage treatment systems.

[0009] Figure 1 shows the configuration of a sewage treatment system as a control system according to this embodiment. The sewage treatment system comprises a sewage treatment device (hereinafter referred to as the sewage treatment device 10), which is an example of a plant that is the control target 10, and a control device 20 that controls the operation of the sewage treatment device 10. The control target 10 can be any device that is subject to feedback control by the control device 20, and is not limited to plants such as sewage treatment devices, but can be various other plants or other devices.

[0010] The sewage treatment device 10 includes an aeration tank 11, a blower 12 that supplies air into the aeration tank 11, an airflow control valve 13 that adjusts the amount of air supplied from the blower 12 to the aeration tank 11, and a DO sensor 14 that detects the dissolved oxygen (DO) concentration in the aeration tank 11. When the opening of the airflow control valve 13 increases and the amount of air supplied to the aeration tank 11 increases, the DO concentration in the aeration tank 11 rises. Conversely, when the opening of the airflow control valve 13 decreases and the amount of air supplied to the aeration tank 11 decreases, the DO concentration in the aeration tank 11 falls. The DO concentration in the aeration tank 11 corresponds to the control amount of the controlled object (sewage treatment device) 10.

[0011] The control device 20 uses PID control to adjust the opening of the airflow control valve 13 so that the DO concentration in the aeration tank 11 is maintained at a predetermined target value Ref. Specifically, the control device 20 generates an manipulated variable u(t) that instructs the opening of the airflow control valve 13 using PID control so that the deviation e(t) between the measured value y(t) of the DO concentration in the aeration tank 11 detected by the DO sensor 14 and the target value Ref input from an external device becomes small, for example, zero. The control device 20 provides the generated manipulated variable u(t) to the sewage treatment device 10. The target value Ref is input from an external device by the operator (user) of the sewage treatment system. From this point forward, the target value Ref will be described as a constant value, but the target value Ref may change. Furthermore, the control method of the control device 20 is not limited to PID control, but may also be, for example, derivative-first PID control, proportional-derivative-first PID control, two-degree-of-freedom PID control, or model-driven PID control.

[0012] When the measured value y(t) of the DO concentration falls below the target value Ref, the control device 20 increases the manipulated variable u(t). This increases the opening degree of the airflow control valve 13, increasing the amount of air supplied to the aeration tank 11. Conversely, when the measured value y(t) of the DO concentration exceeds the target value Ref, the control device 20 decreases the manipulated variable u(t). This decreases the opening degree of the airflow control valve 13, decreasing the amount of air supplied to the aeration tank 11.

[0013] When the control device 20 detects a change in the control characteristics of the sewage treatment device 10 due to aging deterioration or changes in the operating environment, it adjusts the control parameters P, specifically the proportional gain Kp, integral gain Ki, and differential gain Kd of the PID control. However, the control parameters P are not limited to these. For example, the control parameters P may include fine-tuning gains. The control device 20 comprises a control unit 21, a data acquisition unit 22 (acquisition unit), a data storage unit 23 (storage unit), a system deterioration detection unit 24 (detection unit), a control parameter determination unit 25 (determination unit), a system identification unit 26 (estimation unit), a stability determination unit 27 (determination unit), and a monitor unit 28 (display unit).

[0014] Figure 2 shows the configuration of a feedback control system including a control unit 21 and a sewage treatment device 10. The control unit 21 is set to control parameters P, specifically a proportional gain Kp, an integral gain Ki, and a differential gain Kd. This feedback control system includes the control parameters P and the control characteristics of the controlled object 10. The control unit 21 calculates an manipulated variable u(t) based on the gains Kp, Ki, and Kd from the deviation e(t) between the measured value y(t) of DO concentration and the target value Ref, and provides or outputs the calculated manipulated variable u(t) to the sewage treatment device 10. When the manipulated variable u(t) is provided or input, the sewage treatment device 10 performs an operation according to the manipulated variable u(t) and outputs the measured value y(t). In this way, the control unit 21 controls the sewage treatment device 10 based on the deviation e(t) and the control parameters P (gains Kp, Ki, Kd).

[0015] The data acquisition unit 22 sequentially acquires the measured value y(t) and manipulated quantity u(t) of the DO concentration in the aeration tank 11 from the DO sensor 14 of the sewage treatment device 10 at predetermined time intervals. The data storage unit 23 sequentially stores the measured value y(t) and manipulated quantity u(t) of the DO concentration acquired by the data acquisition unit 22. When the amount of data for the measured value y(t) and manipulated quantity u(t) of the DO concentration reaches a predetermined upper limit, the oldest data may be deleted in order. In this case, the data storage unit 23 stores a certain amount of measured value y(t) of the DO concentration in order from the most recent. In addition, information other than the measured value y(t) (control quantity), such as the airflow rate of the blower of any component in the controlled object 10 and the rate of change of the DO concentration, may be acquired as information indicating the state of the controlled object 10.

[0016] The system deterioration detection unit 24 detects changes in the control characteristics of the sewage treatment device 10. Specifically, when the control characteristics of the sewage treatment device 10 change due to aging deterioration or changes in the operating environment, the control parameter P (first value of the control parameter) currently set in the control unit 21 deviates from an appropriate value, and as a result, the accuracy of the control that matches the measured value y(t) of DO concentration with the target value Ref may decrease. At predetermined intervals, the system deterioration detection unit 24 checks whether the determination formula shown in equation (1) below, based on the absolute value of the deviation e(t) between the measured value y(t) of DO concentration and the target value Ref, continues to hold true for a predetermined period of time. The predetermined period and predetermined time can be arbitrarily determined according to the controlled object. For example, the predetermined period may be one day and the predetermined time may be one hour. The determination formula shown in equation (1) may also be performed each time data is stored in the data storage unit 23.

[0017]

[0018] However, in equation (1), δ is a predetermined threshold, which is set according to the desired control accuracy. For example, δ is set to 0.1. If the above determination formula continues to hold true for a predetermined period of time, the system deterioration detection unit 24 determines that the control characteristics of the sewage treatment device 10 have changed and transmits a characteristic change detection signal to the control parameter determination unit 25 and the system identification unit 26. Here, we assume that the change in control characteristics is a deterioration of the control characteristics, and transmit the system deterioration detection signal.

[0019] The method for detecting changes in control characteristics is not limited to the method described above. For example, changes in control characteristics may be detected based on the average value of the absolute value of the deviation e(t) over a predetermined period. Furthermore, the operator may be able to specify the judgment conditions from an external device. Additionally, the AI ​​may dynamically generate the judgment conditions.

[0020] When a system degradation detection signal is input, the system identification unit 26 uses the data in the data storage unit 23 to estimate the control characteristics (characteristic parameters of the transfer function of the sewage treatment device 10) after the change. The estimated control characteristics are used to determine the stability of the control parameter P determined by the control parameter determination unit 25. The system identification unit 26 does not operate if no system degradation detection signal is input. The data in the data storage unit 23 used for estimation may be limited to data from the oldest time point onward used to determine that the above determination formula is true.

[0021] In this embodiment, the control characteristics of the sewage treatment device 10 are modeled as the transfer function G(s) of a first-order lag system shown in equation (2). The transfer function G(s) corresponds to the first transfer function, which is the transfer function of the controlled object (sewage treatment device) 10.

[0022]

[0023] However, in the above equation, K is the process gain and T is the time constant. Note that the method of modeling the control characteristics of the sewage treatment device 10 is not limited to this. For example, the control characteristics of the sewage treatment device 10 may be modeled as a higher-order transfer function or a transfer function with a dead time L.

[0024] A change in the control characteristics of the sewage treatment device 10 means that at least one of the characteristic parameters included in the transfer function G(s), namely the process gain K and the time constant T, changes.

[0025] The system identification unit 26 estimates the values of the characteristic parameters of the transfer function G(s) after the change. Here, the details of the estimation of the characteristic parameters of the transfer function G(s) using the least squares method will be described. However, the method for estimating the parameters of the system model is not limited to this.

[0026] As shown in FIG. 1, u(t) and y(t) are the sample values of the input and output signals of the system at the discrete time t, respectively. The following estimates the characteristic parameters, specifically the process gain K and the time constant T, using the data u(t) and y(t). A control system modeled by the transfer function G(s) of a first-order lag system is expressed by Equation (3).

[0027]

[0028]

[0029]

[0030]

[0031]

[0032]

[0033] However,

[0034] At this time, in order to calculate γ that minimizes J, solve ∂J / ∂γ = 0. Here, the left side is ∂J / ∂γ = Φ Ｔ Φγ - Φ Ｔ can be expanded like Y. Therefore, if we solve ∂J / ∂γ = 0, the following Equation (8) is obtained.

[0035]

[0036] Therefore, the characteristic parameters T and K of the transfer function G(s) are updated as shown in the following Equation (9). As described above, the system identification unit 26 estimates the characteristic parameters of the transfer function G(s) after the control characteristics have changed, that is, the process gain K and the time constant T. The system identification unit 26 transmits the estimated characteristic parameters K and T to the stability determination unit 27.

[0037]

[0038] Figure 3 is a block diagram showing the detailed functional configuration of the control parameter determination unit 25. Automatic adjustment of control parameters will be explained based on Figure 3.

[0039] The control parameter determination unit 25 includes a reward calculation unit 250, an inference unit 251, an online learning unit 252, and a memory unit 253. The control parameter determination unit 25 updates the parameter adjustment policy (stochastic policy function) by reinforcement learning according to the operating conditions, and infers (estimates) the optimized control parameters P based on the updated stochastic policy function, mainly using the DO concentration measurement value y(t) and the control target value Ref. The reinforcement learning model is a model that takes the DO concentration measurement value y(t) and the control target value Ref as input to the stochastic policy function and outputs the control parameters P.

[0040] The operation of the inference unit 251 is determined by the system degradation detection signal from the system degradation detection unit 24. If no system degradation detection signal is input, the inference unit 251 does not operate, and the control parameters are not updated. On the other hand, if a system degradation detection signal is input, the inference unit 251 determines and outputs an action to maximize value based on the probabilistic policy function π of the reinforcement learning model obtained from the storage unit 253, with respect to the operating status of the control system (such as the Do concentration measurement value y(t) and target value Ref stored in the data storage unit 23, and the deviation between these measurement values ​​y(t) and target value Ref). Here, the action output by reinforcement learning is the control parameter. For example, the inference unit 251 uses the latest time data stored in the data storage unit 23 (for example, the Do concentration measurement value y(t) and target value Ref) to determine state s ｔ This is used as input to the stochastic policy function π, and the output determines the action (control parameter) at t+1 that maximizes the probability. Details of the stochastic policy function π will be described later.

[0041] The reward calculation unit 250 outputs an immediate reward value r(t) (also simply referred to as the reward value) based on the measured value y(t) and the target value Ref. The immediate reward value r is designed based on, for example, the absolute value of the deviation between the measured value y(t) and the target value Ref as shown in Equation (10). Also, ISE (Integral of Squared Error), ITAE (Integral of Time multiplied by Absolute value of Error), IAE (Integral of Absolute value of Error), and ITSE (Integral of Time multiplied by Squared Error), which are evaluation functions for evaluating control performance, can also be designed. Further, based on Equation (10), supplementary items of the reward can be added according to the state of the controlled object.

[0042]

[0043] The online learning unit 252 observes the control parameter P, the measured value y(t), and the target value Ref, and updates the probabilistic policy function π of the reinforcement learning model. Here, the processing of the online learning unit 252 will be described. The data in the data storage unit 23 used for online learning may be the data after the oldest time of the data used for the determination that the above determination formula (1) holds.

[0044] For example, the policy search method is a learning method for expressing the action probability of the reinforcement learning agent by a neural network. To search for a policy, three elements of state s ∈ S, action a ∈ A, and reward r ∈ R are required. At discrete time t, the reinforcement learning agent observes state s ｔ and executes action a ｔ and as a result of the state transition, receives reward r ｔ . Generally, the reward and the state of the transition destination are random variables, and their distributions depend on state s ｔ and action a ｔ . The probabilistic policy function π that characterizes the probabilistic policy in action selection represents the probability of selecting action a ｔ in state s ｔ . That is, action a according to the probability of the policy corresponding to state s ｔ ​ｔ This is executed. This will result in state s ｔ From state s ｔ＋１ The system transitions according to a stochastic policy function π. The state is specified, for example, by the measured value y(t), the target value Ref, and the control parameter P. However, the definition of the state is not limited to this, and information other than the measured value y(t) and the target value Ref may be added. For example, a manipulated variable may be added.

[0045] In the online learning unit 252, the inference unit 251 is in state s ｔ The inferred control parameter P (the control parameter P currently used in the control unit) is known action a ｔ The state of the actual operating environment in which that action was performed is s ｔ＋１ The data is observed from the data storage unit 23, and the reward calculated by the reward calculation unit 250 is r ｔ It is received as such. As mentioned above, the reward r ｔ The destination state s ｔ＋１ Next step ｔ＋１ The reward calculation unit 250 calculates the reward based on the absolute value of the deviation between the control target value Ref and the control measurement value y(t).

[0046] Online learning section 252 is a stochastic policy function π(a ｔ | s ｔ The online learning unit 252 acquires knowledge (information) related to the above from the memory unit 253 and learns (updates) the stochastic policy function π that maximizes the performance of the online learning unit 252. That is, the online learning unit 252 updates the stochastic policy function π so as to maximize the evaluation value based on the total discounted reward shown in equation (11).

[0047] Here, the discount rate 0 < γ ≤ 1 (in this embodiment, for example, set to 0.9 to 0.99) indicates the importance of future rewards. Also, E(•) represents the expected value calculation.

[0048] Evaluation value V based on state transitions π We define TDerr, which represents the change in (s). That is, TDerr = V after change π (s) - V before change π (s) In the learning of online learning section 252, TDerr is used as the behavioral evaluation and the stochastic policy function π(aｔ | s ｔ ) will be updated to improve it. In other words, when TDerr is positive, the online learning unit 252 is considered to have transitioned to a good state, so state s ｔ Action a in ｔ The stochastic policy function π is updated to increase the probability of selecting . Conversely, when TDDerr is negative, the online learning unit 252 checks the state s ｔ Action a in ｔ The stochastic policy function π is updated to reduce the probability of selection.

[0049] Here, the stochastic policy function π(a ｔ | s ｔ The stochastic policy function π(a) is expressed using the policy parameter vector θ. ｔ | s ｔ ) is expressed including the policy parameter vector θ.

[0050] The online learning unit 252 changes the action selection probability by adjusting the policy parameter vector θ. For example, the online learning unit 252 updates the policy parameter vector θ as shown in equation (12), where η > 0 is the step size (learning rate).

[0051]

[0052] The memory unit 253 is implemented by, for example, a semiconductor memory element such as RAM or flash memory, or a hard disk. The memory unit 253 may also be provided with a replay buffer. The replay buffer is implemented by, for example, a semiconductor memory element such as RAM (Random Access Memory) or flash memory, or a hard disk. The replay buffer is one of the methods used to save and update the learning model in deep reinforcement learning. The memory unit 253 may store data in sets of state s, action a (control parameter P), and reward r as action transitions in its replay buffer (note that state s also includes the measured value y and the target value Ref). The data stored in the replay buffer may be repeatedly randomly sampled and used during learning. The reinforcement learning model (stochastic policy function π) obtained by reinforcement learning is also stored in the memory unit 253.

[0053] The system determines whether the control performance can be achieved with the new control parameter P. For example, it calculates the reward r when the new control parameter P is used. For example, it calculates the reward r when a state transition is performed under the new control parameter P for the measured value (state) indicated by the latest time data in the data storage unit 23. In this case, a stochastic policy function π may be used. The reward r output in the process of calculating the new control parameter P using the stochastic policy function π may also be used. If the reward r is greater than the desired control performance r', it is determined that the control performance is achievable; if it is smaller, it is determined that it is not achievable.

[0054] If the control performance cannot be achieved with the new control parameter P, the control parameter determination unit 25 sets the default control parameter P0 (the third value of the control parameter) to the control unit 21. The method for determining the default control parameter P0 is not particularly limited, but various well-known methods can be used, such as the Ziegler and Nicholas tuning method or a method based on the step response waveform. The control unit 21 controls the operation of the sewage treatment device 10 using the default control parameter P0.

[0055] On the other hand, if the control performance can be achieved with the new control parameter P (for example, if the reward r(t) is equal to or greater than the desired control performance r'), the control parameter determination unit 25 transmits the new control parameter P and the characteristic parameters K and T received from the system identification unit 26 to the stability determination unit 27 in order to determine the stability of the new control parameter P, that is, the stability of the control system when the new control parameter P is applied.

[0056] Figure 4 is a flowchart illustrating the detailed processing flow of the system degradation detection unit 24, the control parameter determination unit 25, the system identification unit 26, and the stability determination unit 27. When a system degradation detection signal is generated in the system degradation detection unit 24 (S201), the system identification unit 26 estimates the characteristic parameters of the transfer function G(s) using the least squares method with respect to the data y(t) and u(t) in the data storage unit 23 (S202). Meanwhile, the control parameter determination unit 25 infers new control parameters P using the data y(t), the target value Ref, and the current control parameters P in the data storage unit 23 (S203). However, the method for estimating characteristic parameters and deriving control parameters are not limited to these. In step S204, the stability determination unit 27 determines the stability of the new control parameters P inferred by the control parameter determination unit 25 based on the input of the system degradation detection signal and the characteristic parameters estimated by the system identification unit 26. Step S204 is described in more detail below.

[0057] The stability determination unit 27 determines the stability of the new control parameter P received from the control parameter determination unit 25. More specifically, the stability determination unit 27 determines the stability of the closed-loop transfer function T(s) using the transfer function C(s) of the control unit 21 and the transfer function G(s) of the sewage treatment device 10, as shown in equation (13) below. The transfer function C(s) of the control unit 21 corresponds to the second transfer function, which is the transfer function of the control unit 21 (a transfer function including the control parameter).

[0058]

[0059] The transfer function C(s) of the control unit 21 is expressed as shown in equation (14) below, using the new control parameters P = (Kp, Ki, Kd).

[0060]

[0061] The transfer function G(s) of the sewage treatment device 10 is expressed as shown in equation (15) below, using characteristic parameters K and T received from the system identification unit 26.

[0062]

[0063] Therefore, the transfer function T(s) of the control system is given by equation (16) below. Figure 5 is a block diagram of the control system.

[0064]

[0065] The stability determination unit 27 determines the stability of the transfer function T(s). The method for determining stability is not particularly limited, but for example, the Routh-Hurwitz stability determination method can be used. For this purpose, the characteristic equation D(s) of the transfer function T(s) is obtained as shown in equation (17) below.

[0066]

[0067] From the characteristic equation D(s) above, the sequence [T+Kd, Kp・K+1, Ki・K] is extracted from the first column of the Routh array. According to the Routh-Hurwitz stability criterion, the condition for the control system to be stable is given by equation (18) below.

[0068]

[0069] If all of the above conditions are met, the stability determination unit 27 obtains "stable" as the stability determination result. Also, {Kpo, Kio, Kdo} are defined, for example, Kpo = -T, Kio = 0, Kdo = -1 / K. The stability margin defined in equation (19) is calculated. The stability determination unit 27 returns the determination result and the stability margin to the control parameter determination unit 25.

[0070]

[0071] Conversely, if at least one of the conditions shown in equation (18) is not met, the stability determination unit 27 obtains "unstable" as the stability determination result. The stability determination unit 27 returns the determination result of "unstable" and information on which of the conditions shown in equation (18) was not met to the control parameter determination unit 25.

[0072] If the determination result received from the stability determination unit 27 indicates stability, the control parameter determination unit 25 sets a new control parameter P (the second value of the control parameter) in the control unit 21. The control unit 21 controls the operation of the sewage treatment device 10 using the newly set control parameter P. The control parameter determination unit 25 also displays the value of the new control parameter P and the stability margin on the monitor unit 28. Figure 6 is an example of a screen image displayed on the monitor unit 28.

[0073] On the other hand, if the determination result received from the stability determination unit 27 indicates instability, the control parameter determination unit 25 re-determines the control parameter P. If stability cannot be achieved even after repeating the determination of the control parameter P a predetermined number of times, the control parameter determination unit 25 sets the default control parameter P0 (the third value of the control parameter) to the control unit 21. The control unit 21 uses the default control parameter P0 to control the operation of the sewage treatment device 10.

[0074] Figure 7 is a flowchart of the control method according to this embodiment. More specifically, Figure 7 is a flowchart of the operation of the control device 20 according to this embodiment. The operation of the flowchart in Figure 7 may also be realized by having a computer execute a computer program.

[0075] In step S101, the system degradation detection unit 24 detects a change in the control characteristics of the sewage treatment device 10 and transmits a characteristic change detection signal (system degradation detection signal) to the system identification unit 26 and the control parameter determination unit 25. In step S102, the system identification unit 26 estimates the characteristic parameters K and T of the sewage treatment device 10 after the change in control characteristics.

[0076] In step S103, the control parameter determination unit 25 determines (estimates) new control parameters P, namely the proportional gain Kp, the integral gain Ki, and the differential gain Kd.

[0077] When the control parameter determination unit 25 estimates a new control parameter P, it also calculates a reward r(t), compares r(t) with the desired control performance r', and determines whether the control performance can be achieved. If the control performance cannot be achieved (S=104=NO), the control parameter determination unit 25 sets the default control parameter P0 to the control unit 21 (S110). On the other hand, if the control performance can be achieved (S104=YES), the control parameter determination unit 25 transmits the new control parameter P and the characteristic parameters K and T estimated by the system identification unit 26 to the stability determination unit 27.

[0078] In step S105, the stability determination unit 27 determines the stability of the transfer function T(s) of the control system, including the new control parameters P = (Kp, Ki, Kd) and the characteristic parameters K and T after the change in control characteristics. If the determination result is "stable" (S106 = YES), the control parameter determination unit 25 sets the new control parameter P to the control unit 21 (S108). The monitor unit 28 displays the value of the new control parameter P and the stability margin (S109).

[0079] On the other hand, if the determination result is "unstable" (S106 = NO) and the number of trials is less than the predetermined number (S107 = NO), the control parameter determination unit 25 re-determines the control parameter P (S103 to S104). If stability cannot be achieved even after repeating the determination of the control parameter P a predetermined number of times (S107 = YES), the control parameter determination unit 25 sets the default control parameter P0 to the control unit 21 (S110).

[0080] As described above, the control device 20 according to this embodiment includes a control parameter determination unit 25 that determines the control parameter P in accordance with changes in the control characteristics of the controlled object, and a stability determination unit 27 that determines the stability of the control system including the control characteristics of the controlled object and the control parameter P. Due to these features, the control device 20 according to this embodiment can guarantee the stability of the control parameter P that has been automatically adjusted during operation.

[0081] (Hardware Configuration) Figure 8 shows the hardware configuration of the control device 20 according to this embodiment. The control device 20 is composed of a computer device 600. The computer device 600 includes a CPU 601, an input interface 602, a display device 603, a communication device 604, a main memory 605, and an external memory device 606, which are interconnected by a bus 607.

[0082] The CPU (Central Processing Unit) 601 executes a computer program on the main memory 605. The computer program is a program that implements the above-described functional configurations of the control device 20. The computer program may not be a single program, but rather a combination of multiple programs or scripts. The CPU 601 executes the computer program, thereby realizing each functional configuration.

[0083] The input interface 602 is a circuit for inputting operation signals from input devices such as keyboards, mice, and touch panels to the control device 20.

[0084] The display device 603 displays data output from the control device 20. The display device 603 is, for example, an LCD (liquid crystal display), an organic electroluminescent display, a CRT (cathode ray tube), or a PDP (plasma display), but is not limited to these. Data output from the computer device 600 can be displayed on the display device 603. The display device 603 corresponds to the monitor 28 (display unit) of the control device 20 according to this embodiment.

[0085] The communication device 604 is a circuit for the control device 20 to communicate with an external device wirelessly or via a wired connection. Data can be input from an external device via the communication device 604. The data input from the external device can be stored in the main memory 605 or the external memory 606. The communication device 604 corresponds to the data acquisition unit 22 of the control device 20 according to this embodiment.

[0086] The main memory 605 stores computer programs, data necessary for the execution of computer programs, and data generated by the execution of computer programs. Computer programs are loaded and executed on the main memory 605. The main memory 605 is, for example, RAM, DRAM, or SRAM, but is not limited to these. The data storage unit 23 of the control device 20 according to this embodiment may be built on the main memory 605.

[0087] The external storage device 606 stores computer programs, data necessary for the execution of computer programs, and data generated by the execution of computer programs. These computer programs and data are read into the main memory 605 when the computer programs are executed. The external storage device 606 is, for example, a hard disk, optical disk, flash memory, and magnetic tape, but is not limited to these. The data storage unit 23 of the control device 20 according to this embodiment may be built on the external storage device 606.

[0088] The computer program may be pre-installed on the computer device 600, or it may be stored on a storage medium such as a CD-ROM. Furthermore, the computer program may be uploaded to the internet.

[0089] Furthermore, the control device 20 according to this embodiment may be configured as a single computer device 600, or as a system consisting of multiple interconnected computer devices 600.

[0090] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the embodiments. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations can be made without departing from the spirit of the embodiments. These embodiments and their variations are included in the scope and spirit of the embodiments, as well as in the claims and their equivalents.

[0091] Furthermore, this embodiment may also take the following configuration: [Item 1] A control device comprising: an acquisition unit that acquires a measured value of a control variable of a controlled object that operates in accordance with an operation variable provided to the controlled object at predetermined time intervals; a control unit that calculates the operation variable for the controlled object and provides the operation variable to the controlled object based on the deviation between the measured value acquired by the acquisition unit and the target value of the control variable and a first value of a control parameter; a detection unit that detects a change in the control characteristics of the controlled object based on the deviation between the measured value and the target value; a determination unit that determines a second value of the control parameter by performing machine learning based on the measured value and the target value in response to the detection of a change in the control characteristics; and a determination unit that determines the stability of a control system including the control unit and the controlled object based on the second value of the control parameter and the control characteristics of the controlled object, wherein the control unit calculates the operation variable based on the deviation and the second value of the control parameter in accordance with the determination result of the determination unit. [Item 2] The control device according to Item 1, wherein the discrimination unit determines the stability of the control system based on a first transfer function which is the transfer function of the controlled object and a second transfer function which is the transfer function of the control unit including the control parameters. [Item 3] The control device according to Item 2, wherein the acquisition unit acquires the manipulated variable provided to the controlled object at predetermined time intervals, and further comprises an estimation unit which estimates the characteristic parameters of the controlled object included in the first transfer function based on the measured value and the manipulated variable, and the discrimination unit determines the stability of the control system based on the characteristic parameters. [Item 4] The control device according to Item 3, wherein the first transfer function is the transfer function of a first-order lag system, and the characteristic parameters include process gain and time constant. [Item 5] The control device according to any one of Items 1 to 4, wherein the detection unit detects deterioration of the control characteristics of the controlled object. [Item 6] The control device according to any one of Items 1 to 5, further comprising a display unit which displays the second value of the control parameters. [Item 7] The control device according to Item 6, wherein the control unit calculates the stability margin of the control system based on the second value of the control parameter, and the display unit further displays the stability margin.[Item 8] The control device according to any one of Items 1 to 7, wherein the determination unit determines the second value of the control parameter in order to reduce the deviation by performing reinforcement learning as machine learning. [Item 9] The control device according to Item 8, wherein the determination unit determines whether the desired control performance based on the deviation can be achieved based on the second value of the control parameter, and the control unit controls the controlled object based on the default value of the control parameter if the desired control performance cannot be achieved. [Item 10] The control device according to any one of Items 1 to 9, wherein the control unit controls the controlled object based on the deviation and the default value of the control parameter if the determination result of the determination unit indicates instability, and controls the controlled object based on the deviation and the second value of the control parameter if the determination result of the determination unit indicates stability. [Item 11] A control method comprising: acquiring measured values ​​of a control variable of a controlled object that operates in accordance with an input variable provided to the controlled object at predetermined time intervals; calculating the input variable for the controlled object by a control unit based on the deviation between the acquired measured values ​​and the target value of the control variable and a first value of a control parameter, and providing the input variable to the controlled object; detecting a change in the control characteristics of the controlled object based on the deviation between the measured values ​​and the target value; determining a second value of the control parameter by performing machine learning based on the measured values ​​and the target value in response to the detection of a change in the control characteristics; determining the stability of a control system including the control unit and the controlled object based on the second value of the control parameter and the control characteristics of the controlled object; and calculating the input variable based on the deviation and the second value of the control parameter in accordance with the result of the stability determination.[Item 12] A computer program to be executed by a computer, which acquires a measured value of a control variable of a controlled object that operates in accordance with an input variable provided to the controlled object at predetermined time intervals; calculates the input variable for the controlled object by the control unit based on the deviation between the acquired measured value and the target value of the control variable and a first value of the control parameter, and provides the input variable to the controlled object; detects a change in the control characteristics of the controlled object based on the deviation between the measured value and the target value; determines a second value of the control parameter by performing machine learning based on the measured value and the target value in response to the detection of a change in the control characteristics; determines the stability of the control system including the control unit and the controlled object based on the second value of the control parameter and the control characteristics of the controlled object; and calculates the input variable based on the deviation and the second value of the control parameter in accordance with the result of the stability determination.

[0092] 10 Wastewater treatment device 11 Aeration tank 12 Blower 13 Air volume control valve 14 DO sensor 20 Control device 21 Control unit 22 Data acquisition unit (acquisition unit) 23 Data storage unit (storage unit) 24 System degradation detection unit (detection unit) 25 Control parameter determination unit (determination unit) 26 System identification unit (estimation unit) 27 Stability determination unit (determination unit) 28 Monitor unit (display unit) 600 Computer device 601 CPU 602 Input interface 603 Display device 604 Communication device 605 Main memory 606 External memory 607 Bus e Deviation Kd Differential gain Ki Integral gain Kp Proportional gain K Process gain P Control parameter Ref Target value of DO concentration T Time constant u Manipulated variable y Measured value of DO concentration

Claims

1. A control device comprising: an acquisition unit that acquires a measured value of a control variable of a controlled object that operates in accordance with an operator provided to the controlled object at predetermined time intervals; a control unit that calculates the operator variable for the controlled object and provides the operator variable to the controlled object based on the deviation between the measured value acquired by the acquisition unit and the target value of the control variable, and a first value of a control parameter; a detection unit that detects a change in the control characteristics of the controlled object based on the deviation between the measured value and the target value; a determination unit that determines a second value of the control parameter by performing machine learning based on the measured value and the target value in response to the detection of a change in the control characteristics; and a determination unit that determines the stability of a control system including the control unit and the controlled object based on the second value of the control parameter and the control characteristics of the controlled object, wherein the control unit calculates the operator variable based on the deviation and the second value of the control parameter in accordance with the determination result of the determination unit.

2. The control device according to claim 1, wherein the discrimination unit determines the stability of the control system based on a first transfer function which is the transfer function of the controlled object and a second transfer function which is the transfer function of the control unit including the control parameters.

3. The control device according to claim 2, wherein the acquisition unit acquires the manipulated variable provided to the controlled object at predetermined time intervals, and further comprises an estimation unit that estimates characteristic parameters of the controlled object included in the first transfer function based on the measured value and the manipulated variable, and the determination unit determines the stability of the control system based on the characteristic parameters.

4. The control device according to claim 3, wherein the first transfer function is the transfer function of a first-order lag system, and the characteristic parameters include process gain and time constant.

5. The control device according to claim 1, wherein the detection unit detects deterioration of the control characteristics of the controlled object.

6. The control device according to claim 1, further comprising a display unit for displaying the second value of the control parameter.

7. The control unit calculates the stability margin of the control system based on the second value of the control parameter, and the display unit further displays the stability margin, as described in claim 6.

8. The control device according to claim 1, wherein the determination unit determines the second value of the control parameter in such a way as to reduce the deviation by performing reinforcement learning as machine learning.

9. The control device according to claim 8, wherein the determination unit determines whether the desired control performance based on the deviation can be achieved based on the second value of the control parameter, and the control unit controls the controlled object based on the default value of the control parameter if the desired control performance cannot be achieved.

10. The control device according to claim 1, wherein the control unit controls the target to be controlled based on the deviation and the default value of the control parameter when the determination result of the determination unit indicates instability, and controls the target to be controlled based on the deviation and the second value of the control parameter when the determination result of the determination unit indicates stability.

Images