Calibration device, calibration method, and calibration program

Abstract

This makes it easier to maintain the accuracy of the automatic control of the device. [Solution] In the calibration device 10, the calculation unit 15b calculates a calibration value for the control value based on the deviation between the measured value and the target value of the evaluation target in the device being processed, which is controlled by the control value. The addition unit 15c adds the calculated calibration value to the recommended value of the control value output by the model 14a, which has been learned by imitation learning.

Description

Technical Field

[0001] The present invention relates to a calibration device, a calibration method, and a calibration program.

Background Art

[0002] Conventionally, PID (Proportional Integral Derivative) control is often used for automatic control of devices in factories. PID control is a method of calculating an output value based on three elements: the deviation between a measured value and a set value related to a control target, its integral, and its derivative, and performing control (see Patent Document 1). In PID control, if the delay time (idle time) from when control is performed until the output value changes is long, PID control cannot be applied, and it may be necessary for a person to perform manual operation.

[0003] In such a case, it is possible to reproduce a person's operation and realize automatic control of the device by imitation learning that learns the past operation history (the history of operation data representing what kind of situation, and the history of control values representing what kind of changes were made to the set value and the output value).

[0004] When reproducing a person's operation, it is possible to imitate the person's operation regardless of environmental changes by re-learning (JIT, Just In Time) using the past history performed in a situation similar to the situation to be reproduced (see Non-Patent Document 1).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

[0007] However, maintaining the accuracy of automatic control of the equipment can be difficult. For example, when the equipment's condition changes due to seasonal changes, weather changes, changes in production conditions, etc., a distribution shift occurs in the distribution of the operating data (explanatory variables) and the control values ​​(target variables) of the imitation learning model, changing from the time of learning.

[0008] Consequently, since there is no data collected in past history that is similar to the situation to be reproduced, there is a risk that the accuracy of learning will decrease even with retraining. Furthermore, while automatic control that reproduces human operations using imitation learning is in operation, no human operations are performed, making it difficult to obtain additional training data to use for imitation learning. Therefore, it is difficult to maintain the accuracy of imitation learning. In such cases, it is necessary to interrupt automatic control and perform manual control to add training data, which may lead to a decrease in the continuation rate of autonomous driving. In addition, if a distribution shift occurs, the waveform of the control value can be reproduced, but a constant deviation occurs between the measured value and the recommended value of the control value, and as a result, a constant deviation may occur between the value being evaluated and the target value.

[0009] Furthermore, in automated control based on imitation learning, it is difficult to eliminate steady-state deviations and bring the evaluated value closer to the target value because there is no mechanism for doing so.

[0010] The present invention has been made in view of the above, and aims to eliminate steady-state deviations caused by distribution shifts, etc., and to easily maintain the accuracy of automatic control of the device. [Means for solving the problem]

[0011] To solve the above-mentioned problems and achieve the objective, the calibration device according to the present invention is characterized by having a calculation unit that calculates a calibration value for a control value based on the deviation between the measured value and the target value of the evaluation target in a device that is controlled by the control value, and an addition unit that adds the calculated calibration value to the recommended value of the control value output by a model learned by imitation learning. [Effects of the Invention]

[0012] According to the present invention, it becomes easy to maintain the accuracy of automatic control of the device. [Brief explanation of the drawing]

[0013] [Figure 1] Figure 1 is a diagram illustrating the overview of the calibration device. [Figure 2] Figure 2 is a diagram illustrating the overview of the calibration device. [Figure 3] Figure 3 is a diagram illustrating the overview of the calibration device. [Figure 4] Figure 4 is a diagram illustrating the overview of the calibration device. [Figure 5] Figure 5 is a diagram illustrating the overview of the calibration device. [Figure 6] Figure 6 is a diagram illustrating the overview of the calibration device. [Figure 7] Figure 7 is a diagram illustrating the overview of the calibration device. [Figure 8] Figure 8 is a schematic diagram illustrating the general configuration of a calibration device. [Figure 9] Figure 9 is a diagram illustrating the processing of the calculation unit. [Figure 10] Figure 10 is a diagram illustrating the processing of the addition section. [Figure 11]FIG. 11 is a flowchart illustrating a calibration processing procedure. [Figure 12] FIG. 12 is a diagram illustrating a computer that executes a calibration program. Mode for Carrying Out the Invention

[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited by this embodiment. Also, in the description of the drawings, the same parts are denoted by the same reference numerals.

[0015] [Outline of Calibration Device] FIGS. 1 to 7 are diagrams for explaining the outline of the calibration device. In a PID control device, an output value is determined by three elements: the deviation between the measured value and the set value of the control target, its integral, and its derivative. For example, as illustrated in FIG. 1, PID control is performed for each control target such as temperature and pressure. The operator may manually change the set value and the output value of each PID control (hereinafter, the set value and the output value are referred to as control values) so that the evaluation target value (for example, impurity concentration) related to the operation target of the plant reaches a desired state (for example, stabilizes the impurity concentration around 1%).

[0016] Also, automatic control that reproduces human operations is realized by imitation learning that learns the past operation history (the history of operation data representing what kind of situation and the change history of control values representing what kind of operations were performed). That is, the control target device is PID-controlled using the control value calculated by the imitation-learned AI.

[0017] Specifically, PID control of a controlled device using imitation learning has two modes, guidance mode and autopilot mode, as illustrated in Figure 1. Guidance mode is a manual driving mode in which the operator manually controls the device based on recommended control values ​​calculated by the AI, as illustrated in Figure 2(a). In contrast, autopilot mode is an automated driving (automatic control) mode in which the device is directly controlled using recommended control values ​​calculated by the AI, as illustrated in Figure 2(b).

[0018] As illustrated in Figure 3(a), when the condition of the equipment changes due to seasonal variations, aging, changes in production volume, repairs, etc., a distribution shift occurs in the distribution of the explanatory variables (operational data) and the objective variable (control values), changing from the time of training. As a result, the accuracy of the AI ​​that has been trained by imitation decreases.

[0019] Therefore, as illustrated in Figure 3(b), the accuracy of the AI, which mimics human operations in response to environmental changes, is maintained by JIT retraining using past history of operations performed under similar conditions to the current situation. In this way, maintenance costs can be reduced by automatically responding to changes in the situation. Here, past history includes the history of operating data that describes the situation and the operation history that describes how the control values ​​were changed.

[0020] However, when a distribution shift occurs, the accuracy of the AI ​​decreases even with retraining because there is no data collected in past history that is similar to the situation you want to reproduce.

[0021] Furthermore, with automatic control based on imitation learning of human operations (autopilot mode as exemplified in Figure 2(b)), it is not possible to obtain additional learning data. To obtain learning data, for example, it is necessary to stop automatic control, switch to guidance mode as exemplified in Figure 2(a), and temporarily perform manual operations to collect history.

[0022] Here, the value used to evaluate whether the controlled device is operating as intended is defined as the evaluation value. For example, if the controlled device has an operating goal of stabilizing the impurity concentration at around 1%, then the impurity concentration is the evaluation value, and 1% is the target value for the evaluation value. If the current evaluation value is 2%, and the operating goal cannot be achieved with the set control value, then the control value of the controlled device is manually changed to bring this value closer to the target value of 1%.

[0023] However, automatic control based on imitation learning lacks a mechanism to bring the evaluated value closer to the target value. In other words, it is not clear how much the control value should be increased or decreased in order to bring the evaluated value closer to the target value. Therefore, as illustrated in Figure 4, corrections are made by manual operation based on the experience of veteran operators. For example, in the example shown in Figure 4, for the raw material input setting value, which is the control value, the AI ​​recommended value of "+3.0" causes the evaluated value to deviate by "+30" from the target value, so it is corrected to "+3.2" by manual operation by a veteran operator.

[0024] Therefore, the calibration device of this embodiment automatically performs calibration by adding or subtracting a certain value (calibration value) to the recommended control value calculated by the AI ​​so that the value to be evaluated approaches the target value. Here, Figure 5 illustrates the time change of the value to be evaluated over time. Specifically, the value to be evaluated (measured value) resulting from control using the recommended value of the AI ​​learned through imitation learning is shown as a dashed line, and the target value of the value to be evaluated is shown as a solid line.

[0025] Furthermore, Figure 6 illustrates the time evolution of the control value. Specifically, the measured control value is illustrated by a solid line, and the recommended control value calculated by the AI ​​is illustrated by a dashed line. As illustrated in Figure 6(a), if, as a result of imitation learning, the measured control value and the recommended control value calculated by the AI ​​are not similar at all, including the waveform, and cannot be predicted, calibration becomes difficult. On the other hand, as illustrated in Figure 6(b), if there is a steady-state deviation between the measured control value and the recommended control value calculated by the AI, it is possible to perform calibration on the recommended control value calculated by the AI ​​in a way that cancels out the deviation.

[0026] In that case, the calibration device performs calibration against the recommended value of the control value calculated by the AI ​​in accordance with this steady-state deviation, as illustrated in Figure 6(c). For example, as illustrated in Figure 7, the calibration device 10 calculates a calibration value according to the deviation between the measured value of the value to be evaluated and the target value, and automatically performs calibration to cancel out the deviation by adding or subtracting this calibration value to the recommended value of the control value calculated by the AI. This makes it easy to bring the value to be evaluated closer to the target value in PID control, regardless of the decrease in the accuracy of imitation learning due to changes in the situation.

[0027] [Configuration of the calibration device] Figure 8 is a schematic diagram illustrating the general configuration of a calibration device. As illustrated in Figure 8, the calibration device 10 of this embodiment is implemented using a general-purpose computer such as a personal computer, and comprises an input unit 11, an output unit 12, a communication control unit 13, a storage unit 14, and a control unit 15.

[0028] The input unit 11 is implemented using an input device such as a keyboard or mouse, and in response to input operations by the operator, it inputs various instruction information, such as processing start, to the control unit 15. The output unit 12 is implemented using a display device such as a liquid crystal display, a printing device such as a printer, etc. The input unit 11 and the output unit 12 may be composed of separate devices connected via a network.

[0029] The communication control unit 13 is implemented using a NIC (Network Interface Card) or the like, and controls communication between the control unit 15 and external devices via the network. For example, the communication control unit 13 controls communication between the control unit 15 and a PID control device that is the target of the calibration process described later, or a management device that manages the operation history of the controlled device.

[0030] The memory unit 14 is implemented using semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or storage devices such as hard disks or optical discs. The memory unit 14 pre-stores processing programs for operating the calibration device 10, as well as data used during the execution of the processing programs, or temporarily stores them each time processing is performed. The memory unit 14 may also be configured to communicate with the control unit 15 via the communication control unit 13.

[0031] In this embodiment, the memory unit 14 stores the learned model (AI) 14a and the operation history 14b, which are used in the calibration process described later.

[0032] The operation history 14b includes past evaluation values ​​(see Figure 4(b)) and past control values ​​of the controlled device.

[0033] The control unit 15 is implemented using a CPU (Central Processing Unit) or the like, and executes a processing program stored in memory. As a result, the control unit 15 functions as an acquisition unit 15a, a calculation unit 15b, and an addition unit 15c, as illustrated in Figure 8. Note that each or part of these functional units may be implemented on different hardware. For example, the acquisition unit 15a may be implemented on a separate device from the other functional units. Furthermore, the control unit 15 may also include other functional units.

[0034] The acquisition unit 15a acquires the operation history 14b. For example, the acquisition unit 15a acquires past evaluation target values, control values, etc., of the controlled device from a management server or the like that manages the operation history of the controlled device, via the communication control unit 13. The acquisition unit 15a stores the previously acquired information as the operation history 14b in the storage unit 14. Alternatively, the acquisition unit 15a may transfer the acquired information to the calculation unit 15b, which will be described below, in real time without storing it in the storage unit 14.

[0035] The calculation unit 15b calculates a calibration value for the recommended control value calculated by the AI, based on the deviation between the measured value and the target value of the evaluation target in the controlled processing target device. Specifically, the calculation unit 15b calculates the calibration value using the average value of the deviation over a predetermined period in the past. The calculation unit 15b also calculates the calibration value by multiplying the average value of the deviation by a predetermined coefficient.

[0036] Here, Figure 9 is a diagram illustrating the processing of the calculation unit. Figure 9 shows an example of the time change between the measured value and the target value of the evaluation target, extracted from the operation history 14b. In Figure 9, the evaluation target (measured value) resulting from control using the recommended value output by the mimic-learned model 14a is shown as a dashed line, and the target value of the evaluation target is shown as a solid line. The calculation unit 15b calculates the average of the measured value and the average of the target value of the evaluation target over a predetermined deviation evaluation period M, i.e., the period going back from the present to the most recent M, calculates the difference between the two, and takes this as the deviation between the measured value and the target value of the evaluation target.

[0037] Then, the calculation unit 15b calculates a calibration value by multiplying the average value of the calculated deviations by a predetermined calibration coefficient, as shown in the following equation (1).

[0038]

number

[0039] Here, the deviation evaluation period M is predetermined by referring to past operation history 14b and taking into account fluctuations in the value to be evaluated. The calibration coefficient is also predetermined by referring to past operation history 14b and based on the relationship between the deviation of the value to be evaluated and the manipulated variable of the control value.

[0040] Returning to the explanation of Figure 8, the adder 15c adds the calculated calibration value to the recommended control value output by the model 14a, which has been learned through imitation learning. At that time, if the increase or decrease range of the calculated calibration value is greater than a predetermined threshold, the adder 15c corrects the calibration value so that the increase or decrease range becomes the threshold, and then adds the calibration value to the recommended control value. In other words, the adder 15c determines the maximum value (maximum calibration increase or decrease range) of the increase or decrease range (absolute value of the change range) of the calibration value per calibration as a predetermined threshold in advance, and updates the calibration value within the range of the maximum calibration increase or decrease range. That is, if the increase or decrease range of the calculated calibration value is less than or equal to the maximum calibration increase or decrease range, the adder 15c performs calibration with that calibration value. On the other hand, if the increase or decrease range of the calculated calibration value exceeds the maximum calibration increase or decrease range, the adder 15c corrects the calibration value so that the increase or decrease range becomes the maximum calibration increase or decrease range, and then performs calibration.

[0041] Whether the absolute value of the calibration value is added to or subtracted from the control value is determined by referring to the operation history 14b and the sign of the calibration coefficient, depending on the operating characteristics of the controlled device.

[0042] Here, Figure 10 is a diagram illustrating the processing of the addition unit. As illustrated in Figure 10, the addition unit 15c refers to past operation history 14b to determine the maximum calibration increase / decrease range. That is, the addition unit 15c refers to past manual manipulation amounts of the control value (increase / decrease range of the calibration value), calculates the average of the manipulation amounts per operation, and sets this as the maximum calibration increase / decrease range. This makes it possible to change the calibration value within a range of manipulation amounts that is safe for operation.

[0043] Furthermore, the addition unit 15c may use the maximum value of the manually manipulated amount of past control values ​​as the maximum calibration increase / decrease range. This also makes it possible to perform automatic calibration within a range where operational safety has been confirmed.

[0044] Furthermore, the addition unit 15c adds the calibration value calculated at a predetermined period to the recommended value of the control value. In other words, the calibration device 10 takes into account the dead time (delay time) from the input of the control value until the control result appears, and sets a period longer than the dead time in advance. Then, the calculation unit 15b and the addition unit 15c repeatedly perform calibration at this period. This enables effective automatic calibration.

[0045] Furthermore, the calibration device 10 is equipped with a toggle switch for switching automatic calibration on and off, allowing switching between automatic and manual calibration. This means that, for example, if a sudden deviation occurs in the evaluation target value due to a change in the operating conditions of the controlled device, manual calibration can be performed to eliminate the deviation caused by the change in operating conditions, and then the system can switch to automatic calibration.

[0046] [Calibration Procedure] Next, with reference to Figure 11, an example of the calibration process by the calibration device 10 according to this embodiment will be described. Figure 11 is a flowchart illustrating the calibration process procedure. The flowchart in Figure 11 starts, for example, when an input instructing the start of the calibration process is received.

[0047] First, the acquisition unit 15a acquires the operation history 14b (step S1). For example, the acquisition unit 15a acquires past evaluation target values ​​and control values ​​from a management server or the like that manages the history of evaluation target values ​​and operation history of the controlled device, via the communication control unit 13.

[0048] Next, the calculation unit 15b calculates a calibration value for the control value based on the deviation between the measured value and the target value of the value to be evaluated in the controlled device (step S2). Specifically, the calculation unit 15b derives, for example, the average value of the deviation over a predetermined deviation evaluation period M from the time change of the value to be evaluated in the past, and uses this to calculate the calibration value. Alternatively, the calculation unit 15b calculates the calibration value by multiplying the average value of the deviation by a predetermined calibration coefficient.

[0049] Then, the adder 15c performs automatic calibration by adding the calculated calibration value to the recommended control value output by the model 14a, which has been learned through imitation learning (step S3).

[0050] In this process, the addition unit 15c derives, for example, the maximum calibration increase / decrease range from the manipulated amount of past control values. If the calculated increase / decrease range of the calibration value is greater than the derived maximum calibration increase / decrease range, the addition unit 15c corrects the calibration value so that the increase / decrease range becomes the maximum calibration increase / decrease range, and then performs calibration by adding the calibration value to the recommended value of the control value. Furthermore, the calculation unit 15b and the addition unit 15c repeat the calibration at a set period considering the dead time of the controlled device. This completes the series of calibration processes.

[0051] [effect] As described above, in the calibration device 10 of the above embodiment, the calculation unit 15b calculates a calibration value for the control value based on the deviation between the measured value and the target value of the evaluation target in the device being processed and controlled by the control value. The addition unit 15c adds the calculated calibration value to the recommended value of the control value output by the model 14a learned by imitation learning.

[0052] Specifically, the calculation unit 15b calculates a calibration value using the average value of deviations over a predetermined period in the past. The calculation unit 15b also calculates a calibration value by multiplying the average value of deviations by a predetermined coefficient. This enables automatic calibration. Therefore, in PID control, it becomes easy to bring the value to be evaluated closer to the target value regardless of the decrease in accuracy of imitation learning due to changes in circumstances. In this way, the calibration device 10 eliminates steady-state deviations caused by distribution shifts, etc., and makes it easy and inexpensive to maintain the accuracy of automatic control of the controlled device.

[0053] Furthermore, if the calculated increase or decrease range of the calibration value is greater than a predetermined threshold, the addition unit 15c corrects the calibration value so that the increase or decrease range becomes the threshold, and adds the calibration value to the recommended value of the control value. For example, by setting a threshold by referring to past operation history, it becomes possible to automatically perform calibration within a range of increase or decrease of the control variable that is safe for operation.

[0054] Furthermore, the addition unit 15c adds the calibration value calculated at a predetermined period to the recommended value of the control value. For example, by having the calculation unit 15b calculate the calibration value at a predetermined period set considering dead time, and the addition unit 15c perform calibration with the calculated calibration value, effective automatic calibration becomes possible.

[0055] [System configuration, etc.] Each component of the illustrated device is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Furthermore, each processing function performed by each device can be implemented, all or any part of it, by a CPU or GPU and programs that are analyzed and executed by that CPU or GPU, or by hardware using wired logic.

[0056] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[0057] [program] It is also possible to create a program that describes the processes performed by the calibration device described in the above embodiment in a computer-executable language. For example, it is possible to create a program that describes the processes performed by the calibration device 10 according to the embodiment in a computer-executable language. In this case, the same effects as in the above embodiment can be obtained by having a computer execute the program. Furthermore, the same processes as in the above embodiment may be realized by recording such a program on a computer-readable recording medium and having a computer read and execute the program recorded on this recording medium.

[0058] Figure 12 shows an example of a computer running a calibration program. Computer 1000 includes, for example, memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0059] Memory 1010 includes ROM (Read Only Memory) 1011 and RAM 1012. ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031. The disk drive interface 1040 is connected to the disk drive 1041. A removable storage medium, such as a magnetic disk or optical disk, is inserted into the disk drive 1041. A serial port interface 1050 is connected to, for example, a mouse 1051 and a keyboard 1052. A video adapter 1060 is connected to, for example, a display 1061.

[0060] Here, the hard disk drive 1031 stores, for example, the OS 1091, the application program 1092, the program module 1093, and the program data 1094. The information described in the above embodiment is stored, for example, in the hard disk drive 1031 or the memory 1010.

[0061] Furthermore, the calibration program is stored in the hard disk drive 1031 as a program module 1093 containing instructions to be executed by the computer 1000, for example. Specifically, the program module 1093 containing instructions for each process performed by the calibration device 10 described in the above embodiment is stored in the hard disk drive 1031.

[0062] Furthermore, the data used for information processing by the calibration program is stored as program data 1094, for example, in the hard disk drive 1031. The CPU 1020 then reads the program module 1093 and program data 1094 stored in the hard disk drive 1031 into the RAM 1012 as needed and executes the procedures described above.

[0063] Furthermore, the program module 1093 and program data 1094 related to the calibration program are not limited to being stored on the hard disk drive 1031; for example, they may be stored on a removable storage medium and read by the CPU 1020 via a disk drive 1041 or the like. Alternatively, the program module 1093 and program data 1094 related to the calibration program may be stored on another computer connected via a network such as a LAN (Local Area Network) or WAN (Wide Area Network) and read by the CPU 1020 via a network interface 1070.

[0064] Although embodiments applying the invention made by the present inventors have been described above, the present invention is not limited by the descriptions and drawings that constitute part of the disclosure of the present invention in this embodiment. That is, all other embodiments, examples, and operational techniques made by those skilled in the art based on this embodiment are included in the scope of the present invention. [Explanation of symbols]

[0065] 10 Calibration device 11 Input section 12 Output section 13 Communication Control Unit 14 Storage section 14a Model 14b Operation History 15 Control Unit 15a Acquisition part 15b Calculation part 15c Addition section

Claims

1. A calculation unit calculates a calibration value for the control value based on the deviation between the measured value and the target value of the evaluation target in a device that is controlled by the control value, An adder adds the calculated calibration value to the recommended value of the control value output by the model learned through imitation learning, A calibration device characterized by having the following features.

2. The calibration device according to claim 1, characterized in that the calculation unit calculates the calibration value using the average value of the deviations over a predetermined period in the past.

3. The calibration device according to claim 2, characterized in that the calculation unit calculates the calibration value by multiplying the average value of the deviations by a predetermined coefficient.

4. The calibration apparatus according to claim 1, characterized in that, if the absolute value of the calculated calibration value is greater than a predetermined threshold, the addition unit corrects the calibration value so that the range of increase or decrease of the calibration value becomes the threshold, and then adds the calibration value to the recommended value of the control value.

5. The calibration device according to claim 1, characterized in that the addition unit adds the calibration value calculated at a predetermined period to the recommended value of the control value.

6. A calibration method performed by a calibration device, A calculation step of calculating a calibration value for the control value based on the deviation between the measured value and the target value of the evaluation target in the device being processed, which is controlled by the control value, An addition step is to add the calculated calibration value to the recommended value of the control value output by the model learned through imitation learning, A calibration method characterized by including the following:

7. A calculation step of calculating a calibration value for the control value based on the deviation between the measured value and the target value of the evaluation target in the device being processed, which is controlled by the control value, An addition step is to add the calculated calibration value to the recommended value of the control value output by the model learned through imitation learning, A calibration program to get a computer to run something.

Images