Control device and control method

Abstract

To provide a controller which outputs a control input capable of early detecting failure occurrence, such as a malfunction and a trouble of a control section and ensuring control safety even if a malfunction occurs in the control section.SOLUTION: A controller includes: one learning section for outputting a state model on the basis of a control technique by mechanical learning; a plurality of control sections for computing control inputs on the basis of the state model; a model updating section for updating the state model stored in an internal storage of the control section into a state model from the learning section; a control input selection section for selecting one from the plurality of control sections and outputting the control input thereof to a control target; and a failure detection section for detecting a failure mounted on the control section.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a control device and a control method using a control technique based on machine learning.

Background Art

[0002] For the purpose of improving the control accuracy of a control device and reducing the set value of the control device, a control device using a control technique based on machine learning (hereinafter referred to as a learning-based control device) is applied. For example, Patent Document 1 etc. can be cited. A learning-based control device generally has a control unit and a learning unit. The control unit outputs an operation amount calculated based on a state model to an actuator, and the learning unit generates a state model based on a sensor signal, and also repeats a process of updating the state model in the control unit with the state model generated by the learning unit every predetermined time.

[0003] On the other hand, the state model used in a learning-based control device often has a large data size for defining the model, and in the update process of the state model, the potential for malfunctions such as overwrite failure and failures of memory (hardware) is increasing.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The present invention is considered in view of the above circumstances, and an object is to provide a control device and a control method that can detect early the occurrence of problems such as malfunctions and failures in a control unit, and output an operation amount that can ensure control safety even when a problem occurs in the control unit.

Means for Solving the Problems

[0006] From the above, in the present invention The control device there is , machine one learning unit that outputs a state model based on a control technique by machine learning, the a control unit that calculates an operation amount based on the state model three and, the a model update unit that updates the state model stored in the internal storage of the control unit with the state model from the learning unit, three and an operation amount selection unit that selects one control unit from the control units and outputs the operation amount thereof to a control target, the a control unit of is provided with an abnormality detection unit that detects a malfunction, The operation amount selection unit compares the operation amount by the control unit that is output to the control target with the operation amount by the control unit that is not output to the control target. When there is no operation amount that is the same as the operation amount by the control unit that is output to the control target and the operation amounts by the control units that are not output to the control target are the same value, one of the operation amounts by the control units that are not output to the control target is selected, and the selected operation amount is output to the control target characterized in that .

[0007] Also, the control method according to the present invention creates a state model based on a control technique by machine learning, calculates an operation amount based on the state model by three control systems, updates the state model used by the three control systems with the learned state model, selects one of the three control systems, outputs the operation amount thereof to a control target, and detects a malfunction of the control system, compares the operation amount output to the control target with other operation amounts not output to the control target, and if there is no operation amount that is the same as the operation amount output to the control target and among the other operation amounts there are two two same threshold values, one is selected from among the operation amounts that are threshold values, and the operation amount is output to the control target. two same characterized by that.

Advantages of the Invention

[0008] According to the present invention, malfunctions of the learning-based control device can be detected from input / output signals.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

Examples

[0011] FIG. 1 shows a configuration example of a control device according to Embodiment 1 of the present invention. In FIG. 1, the control device 1 receives the state quantity 3 acquired by a sensor or the like installed in the control target 2 as an input, calculates an operation amount, and outputs it to the control target 2.

[0012] The control device 1 mainly includes a learning unit 11 that generates a state model using the state quantity 3 as an input, a model update unit 12 that updates the state model of the control unit 13 based on the output of the learning unit 11, a control unit 13 (13A, 13B, 13C) that calculates the operation quantity using the state model, an operation quantity selection unit 14 that selects one of the three control units 13A, 13B, 13C and outputs the operation quantity to the control target 2, and an abnormality detection unit 15 that compares the operation quantities of the three control units 13A, 13B, 13C and detects an abnormality in the control unit.

[0013] Among these, the learning unit 11 generates a state model used in the control technology by machine learning based on the state quantity 3 of the control target 2. Further, the state model is output to any one of the control units 13A, 13B, 13C through the model update unit 12. As the control technology by machine learning used in the learning unit 11 and the form of the state model, a known method can be applied. For example, those disclosed in Patent Document 1 can be used.

[0014] The control unit 13 (13A, 13B, 13C) has a state model in the internal storage and calculates the operation quantity based on the state model. Further, the operation quantity is output to the control target 2 by the output of one control unit selected through the operation quantity selection unit 14. The state model in the internal storage is appropriately updated to the latest state model output from the learning unit 11 by the model update unit 12. As a method for calculating the operation quantity based on the state model, a known method can be applied. For example, those disclosed in Patent Document 1 can be used.

[0015] The model update unit 12 selects one of the three control units 13 (13A, 13B, 13C) and overwrites the output (new state model) of the learning unit 11 in the internal storage where the state model of the selected control unit is stored. The feature of the model update unit 12 is to select one of the control units that have not output the operation quantity to the control target 2 and update the control units in order.

[0016] An example of the processing of such a model update unit 12 will be described using the flowchart of FIG. 2.

[0017] First, in processing step S201, it is determined whether (YES) or not (NO) the learning unit 11 has generated and output a new state model. If a new state model has been output (YES), the process proceeds to processing step S202. On the other hand, if the learning unit 11 has not output a new state model (NO), processing step S201 is repeated.

[0018] In processing step S202, after processing step S201, if there is a control unit that has not been updated to the new state model, one of the control units is selected from among them and the process proceeds to processing step S203. On the other hand, if all the control units have been updated to the new state model, the process returns to processing step S201.

[0019] In processing step S203, if the control unit selected in processing step S202 (hereinafter referred to as control unit i) has not output an operation amount to the control target 2 (YES), the process proceeds to processing step S204. On the other hand, if control unit i has output an operation amount to the control target 2 (NO), the process returns to processing step S202.

[0020] In processing step S204, the state model inside control unit i is overwritten with the output of the learning unit 11. After the overwriting is completed, the process returns to processing step S202. Also, during the overwriting, it is notified to the abnormality detection unit 15 in FIG. 1 described later that the state model of the control unit is being updated.

[0021] According to the process described above, when the learning unit 11 outputs a new state model, one control unit after another is selected from among the control units that have not output an operation amount to the control target 2, and the state model inside the control unit can be updated to the output of the learning unit 11 in order.

[0022] Returning to FIG. 1, the operation amount selection unit 14 receives the operation amounts output from the three control units 13A, 13B, and 13C, selects one of the operation amounts, and outputs the selected operation amount to the control target 2. The feature of the operation amount selection unit 14 is that if three operation amounts are compared and there are two or more identical operation amounts, one of them is selected. An example of the processing of such an operation amount selection unit 14 will be described using the flowchart of FIG. 3.

[0023] In the process of FIG. 3, first, in processing step S401, the operation amount selected at the current time (hereinafter, operation amount i) is compared with the other operation amounts. If there is no operation amount identical to operation amount i among the other operation amounts (NO), the process proceeds to processing step S402. On the other hand, if there is one or more operation amounts identical to operation amount i among the other operation amounts (YES), processing step S401 is repeated.

[0024] In processing step S402, the other operation amounts are compared. If there are two or more identical operation amounts among them (YES), the process proceeds to processing step S403. On the other hand, if all the operation amounts are different (NO), the process proceeds to processing step S404.

[0025] In processing step S403, one of the operation amounts with two or more identical values is selected, and the selected operation amount is output to the control target 2. After the operation amount is switched, the process returns to processing step S401.

[0026] In processing step S404, it is notified to the abnormality detection unit 15, which will be described later, that all the operation amounts are different.

[0027] According to the above-described processing, when there is no operation amount identical to operation amount i and there are two or more identical values among the operation amounts excluding operation amount i, the operation amount i can be switched to the operation amount selected from among them and output to the control target 2.

[0028] When the abnormality detection unit 15 receives notifications from the model update unit 12 and the operation amount selection unit 14, and all the operation amounts output from the control units are different and the state models of all the control units 13 are not being updated, it determines that an abnormality has occurred in one of the control units 13A, 13B, and 13C and notifies the administrator.

[0029] An example of the processing of such an abnormality detection unit 15 will be described using the flowchart of FIG. 4.

[0030] First, in processing step S501, when the abnormality detection unit 15 receives a notification from the operation amount selection unit 14 and determines that all the operation amounts output from the control units are different (YES), it proceeds to processing step S502. On the other hand, when it has not received a notification from the operation amount selection unit 14 (NO), it repeats processing step S501.

[0031] In processing step S502, when the abnormality detection unit 15 receives a notification from the model update unit 12 and determines that the state models of all the control units are not being updated (YES), it proceeds to processing step S503. On the other hand, when it determines that any one of the state models of the control units is being updated (NO), it returns to processing step S501.

[0032] In processing step S503, it notifies the administrator that an abnormality has occurred in one of the control units 13A, 13B, and 13C. The method of notifying the administrator may be a known method such as screen display or message transmission.

[0033] According to the processing described above, when all the operation amounts output from the control units are different and the state models of all the control units are not being updated, it is possible to notify the administrator that an abnormality has occurred in one of the control units 13A, 13B, and 13C.

[0034] The processing of the control device 1 including the learning unit 11, the model update unit 12, the control units 13A, 13B, 13C, the operation amount selection unit 14, and the abnormality detection unit 15 described above will be explained over time.

[0035] First, using FIG. 5, a process will be described in which the control units 13A, 13B, and 13C are normal, a new state model is output from the learning unit 11, and the state models of the control units 13A, 13B, and 13C are updated.

[0036] FIG. 5 shows the processing content of the control device when no abnormality occurs in the control unit. In FIG. 5, the horizontal axis shows the states at that time in the learning unit 11, the control units 13 (A, B, C), the operation amount selection unit 14, and the abnormality detection unit 15, and the vertical axis shows the time from 0 to 10. In this description, S(0) and S(1) are the models S from the learning unit, and the symbols 0 and 1 in the parentheses are the model type numbers, meaning that the model with model type number 1 is the latest model. Also, the symbol * attached to the model indicates that the output of the control unit with this symbol attached is selected.

[0037] In the following description, first, in the initial state, the state models of the control units 13A, 13B, and 13C are the same as S(0), and in the operation amount selection unit 14, the control unit 13A is selected, and it is assumed that the operation amount of the control unit 13A is output to the control target 2. On the other hand, at time 0, the learning unit 11 outputs a new state model S(1).

[0038] In response to this change, at time 1, the model update unit 12 selected the control unit 13B, and the state model of the control unit 13B started to be updated from S(0) to S(1). To specifically explain the processing of the model update unit 12, first, it was determined in processing step S201 that a new state model was output from the learning unit 11. It proceeded to processing step S202, selected the control unit 13A, and proceeded to processing step S203. In processing step S203, it was determined that the control unit 13A was outputting an operation amount to the controlled object 2, and it returned to processing step S202. Again in processing step S202, the control unit 13B was selected and it proceeded to processing step S203. In processing step S203, it was determined that the control unit 13B was not outputting an operation amount to the controlled object 2, and it proceeded to processing step S204. In processing step S204, it was determined that the state model S(0) of the control unit 13B was different from the output S(1) of the learning unit 11, and it proceeded to processing step S205 to start the state model update operation and notify the anomaly detection unit 15 that the state model was being updated.

[0039] At time 2, the update operation of the state model of the control unit 13B was completed. Thereafter, it proceeded from processing step S205 to processing step S202, selected the control unit 13C, and in the same manner as in the case of the control unit 13B, it proceeded to processing steps S203 and S204, and at time 4, the state model update operation for the control unit 13C was started.

[0040] At time 4, on the other hand, in the operation amount selection unit 14, in processing step S401, the operation amount of the control unit 13A was compared with other operation amounts (the operation amounts of the control units 13B and 13C), and it was determined that there was no same value. It proceeded to processing step S402, compared the operation amounts of the control units 13B and 13C, and determined that they were different (there were not two or more same operation amounts), and it proceeded to processing step S404 to notify the anomaly detection unit 15. In the anomaly detection unit 15, upon receiving the notification from the operation amount selection unit 14, in processing step S501, it was determined that all the operation amounts were different, and it proceeded to processing step S502. In processing step S502, it was determined from the notification received from the model update unit 12 that one of the control units was being updated, and it returned to processing step S501, and there was no notification to the administrator.

[0041] At time 5, the update operation of the state model of the control unit 13C was completed. In the operation amount selection unit 14, in processing step S402, the operation amounts of the control units 13B and 13C were compared, and it was determined that they were the same value, and the process proceeded to processing step S403. At time 6, in processing step S403, the operation amount output to the control target 2 was switched from the control unit 13A to the control unit 13B.

[0042] At time 7, on the one hand, in the model update unit 12, the control unit 13A was selected in processing step S202, and similar to the case of the control unit 13C, the process proceeded to processing step S203 and processing step S204, and the state model update operation for the control unit 13A was started.

[0043] At time 8, after the state model update operation of the control unit 13A was completed, the model update unit 12 returned to processing step S202. In processing step S202, after processing step S201, it was determined that the state models of all the control units had been updated, and the process returned to processing step S201.

[0044] As described above, according to the present invention, after the learning unit 11 outputs a new state model, the state models of the control units 13A, 13B, and 13C can be updated in order, and the operation amount output to the control target 2 can be smoothly switched.

[0045] FIG. 6 shows the processing content of the control device when a control unit that does not output an operation amount to the control target fails. Next, with reference to FIG. 6, when one of the control units that does not output an operation amount to the control target 2 fails, the process of detecting this will be described. The notation in FIG. 6 is basically the same as that in FIG. 5, but the time on the vertical axis shows the time from i to i + 7.

[0046] In the initial state, assume that the state models of the control units 13A and 13B are the same as S(n), and the control unit 13C is broken. Also, assume that the operation amount of the control unit 13A is output to the control target 2. Then, at time i, the learning unit 11 outputs a new state model S(n + 1).

[0047] At time i + 1, the control unit 13B was selected by the model update unit 12, and the state model of the control unit 13B started to be updated from S(n) to S(n + 1) and ended at time i + 2.

[0048] At time i + 3, in the operation amount selection unit 14, in processing step S404, it notified the abnormality detection unit 15 that all operation amounts were different. The abnormality detection unit 15 received the notification from the operation amount selection unit 14 and advanced from processing step S501 to processing step S502. In processing step S502, it was determined that the model was not being updated, and it advanced to processing step S503 to notify the administrator that an abnormality had occurred in the control unit.

[0049] During times i + 4 and i + 5, while the state model of the control unit 13C was being updated, the abnormality notification to the administrator temporarily stopped. However, after time i + 6, after the update of the state model of the control unit 13C was completed, the abnormality notification to the administrator continued.

[0050] Thus, according to the present invention, when one of the control units fails, it is possible to detect and notify that an abnormality has occurred. Also, by continuously outputting the operation amount of the control unit that is not faulty to the control target 2, control safety can be ensured.

[0051] In the above, an example of detecting a failure of a control unit that does not output an operation amount to the control device 2 among the control units, such as when a control unit fails during the update of the state model, was described. The present invention is not limited to the above, and even when a control unit that outputs an operation amount to the control device 2 fails, control safety can be ensured, and abnormalities can be detected and notified.

[0052] FIG. 7 shows the processing content of the control device when the control unit that outputs the operation amount to the control target fails. In this example, in the initial state (time j), the state models of the control units 13A, 13B, and 13C are the same as S(n), and the operation amount of the control unit 13C is output to the control target 2. At time j+1, the control unit 13C fails. At this time, the operation amount selection unit 14 performs a process of switching the operation amount output to the control target 2 to the control unit 13A through processing step S401 and processing step S402 and then to processing step S403. The state at time j+2 is the same as the state at time i shown in FIG. 6, and an abnormality is detected and notified when a new state model is output from the learning unit 11 next.

[0053] As described above, according to the present invention, when the control unit that outputs the operation amount to the control device 2 fails, the control safety can be ensured by switching the operation amount to another control unit. Further, when the state model is newly updated next, it is possible to detect and notify that an abnormality has occurred in the control unit.

[0054] FIG. 8 is a diagram showing the processing content of the control device when the data of the control unit is tampered with. Next, with reference to FIG. 8, a process of detecting that the state model of one control unit is tampered with due to a cyber attack or the like will be described. Here, however, the time on the vertical axis is represented by each time from k to k+9.

[0055] In the initial state, the state models of the control units 13A, 13B, and 13C are the same as S(m), and the operation amount of the control unit 13A is output to the control target 2, but the state model of the control unit 13C is tampered with to S(X). When the state model of the control unit that outputs the operation amount to the control target 2 is tampered with, as described in FIG. 7, the operation amount is switched by the operation amount selection unit 14. At time k, the learning unit 11 outputs a new state model S(m+1).

[0056] Thereafter, at time k+1, the control unit 13B starts to be updated (S(m) ⇒ S(m+1)) and finishes the update at time k+2. At time k+3, since the operation amounts of all the control units are different and the model is not being updated, the abnormality detection unit 15 notifies the administrator of the abnormality. At k+4, the tampered control unit 13C is also updated, and all subsequent processes become normal.

[0057] Thus, according to the present invention, when the state model of one control unit is tampered with, an abnormality can be notified to the administrator.

[0058] Also, according to the present invention, it is possible to detect the case where the update of the state model of the control unit fails. When the update of the control unit 13C fails in the same initial state as above, it reaches the state at time k+3, and the abnormality detection unit 15 notifies the administrator of the abnormality.

[0059] As described above, as malfunctions of the control unit, there are (a) the case where the hardware of the control unit fails, (b) the case where the state model data of the control unit is tampered with, and (c) the case where the update operation of the state model of the control unit fails. The processing of the present invention and the abnormality notification in each case have been described. The abnormality notification pattern differs depending on the case, and the cause can be estimated from this difference in pattern. Case (a) is a temporary and then continuous abnormality notification, case (b) is a temporary abnormality notification, and case (c) is a continuous abnormality notification.

[0060] Thus, according to the present invention, in order to repeat the operation amount calculation work and the state model update work, for a control unit with a high potential for malfunction, three or more control units having the same function are provided, and by comparing these signals, various malfunctions can be detected. Also, based on the signal comparison result, by switching the signal, even when a malfunction occurs, an appropriate operation amount can be output and the control safety can be ensured.

Example

[0061] FIG. 9 shows a configuration example of the control device according to Embodiment 2 of the present invention. The difference from Embodiment 1 of the present invention lies in having four control units and the processing of the abnormality detection unit 15. The features and processing of other elements, the learning unit 11, the model update unit 12, the control units 13A, 13B, 13C, and the operation amount selection unit 14 are the same as those in Embodiment 1.

[0062] Similar to Embodiment 1, Embodiment 2 can detect malfunctions of the control unit and ensure control safety even when a malfunction occurs. A further advantage is that in Embodiment 1, when a malfunction occurs in one control unit, it is difficult to output the operation amount from the control unit updated to the new state model to the control target 2, whereas in Embodiment 2, even when a malfunction occurs in one control unit, the operation amount can be output from the control unit updated to the new state model to the control target 2. Also, it can handle cases where there is a subsequent new update of the state model compared to the three normal control units.

[0063] The difference in the processing of the abnormality detection unit 15 will be described with reference to FIG. 10. The difference from Embodiment 1 is that a notification from the model update unit 12 is not required and there is no processing step S502. When receiving a notification from the operation amount selection unit 14 and determining that all operation amounts are different (processing step S501), a notification is sent to the administrator (processing step S503). The processing of Embodiment 2 will be described over time. FIG. 11 shows the processing of updating the state model when all control units are normal, FIG. 12 shows the processing of updating the state model when one control unit fails, and FIG. 13 shows the processing of updating the state model when the data of one control unit is tampered with.

[0064] When all the control units shown in FIG. 11 are normal, a new state model is output at time 0, and the control units 13B, 13C, and 13D are updated by the model update unit 12. At time 6, the operation amount output to the control target 2 by the operation amount selection unit 14 is switched to the control unit 13B. After that, the control unit 13A is also updated, and the update work is completed.

[0065] When one control unit (control unit 13D) shown in FIG. 12 fails, it proceeds to time 2 in the same manner as when all control units are normal. However, at time 3, since the states of all control units are different from those before update, after update, during update, and failure, and all operation amounts are different, an abnormality is notified by the abnormality detection unit 15. After that, even when the control unit 13D has failed, at time 6, the operation amount output to the control target 2 by the operation amount selection unit 14 is switched to the control unit 13B.

[0066] When the data of one control unit (control unit 13D) shown in FIG. 13 is tampered with, at time 3, in the same manner as when one control unit fails, since the states of all control units are different from those before update, after update, during update, and tampering, and all operation amounts are different, an abnormality is notified by the abnormality detection unit 15.

[0067] Thus, according to the second embodiment, the same effects as those of the first embodiment can be achieved. Even when one control unit fails, an operation amount can be output from the control unit updated to the new state model to the control target 2, and the subsequent update of the state model can be continued.

Description of Reference Numerals

[0068] 1: Control device 2: Control target 3: State quantity 11: Learning unit 12: Model update unit 13A, 13B, 13C: Control units 14: Operation amount selection unit 15: Abnormality detection unit

Claims

1. One learning unit that outputs a state model based on a control technique by machine learning, three control units that calculate an operation amount based on the state model, and a model update unit that updates the state model stored in the internal storage of the control unit to the state model from the learning unit, and three From the above control units, one control unit is selected, and an operation amount selection unit that outputs the operation amount to the control target, and an abnormality detection unit that detects a defect of the control unit are provided. The operation amount selection unit compares the operation amount by the control unit that is output to the control target with the operation amount by the control unit that is not output to the control target, and there is no operation amount that is the same as the operation amount by the control unit that is output to the control target, and When the operation amounts by the control units that are not output to the control target are the same value, one is selected from the operation amounts by the control units that are not output to the control target, and the selected operation amount is output to the control target. A control device characterized by the above.

2. The control device according to claim 1, The model update unit selects one control unit from the control units not selected by the operation amount selection unit, and sequentially updates the state model inside the control unit to the state model from the learning unit one by one. A control device characterized by the above.

3. The control device according to claim 1, The abnormality detection unit compares the operation amounts from the three control units, and determines that an abnormality has occurred when there are not two or more control units that output the same operation amount and the state models of all the control units are not being updated. A control device characterized by the above.

4. Create a state model based on a control technique by machine learning, calculate the operation amount by three control systems based on the state model, update the state model used in the three control systems to the learned state model, and select one of the three control systems , Output the operation amount to the control target, and detect the defect of the control system. Compare the operation amount output to the control target with other operation amounts that are not output to the control target, and if there is no operation amount that is the same as the operation amount output to the control target and two of the other operation amounts are the same value, select one from the two operation amounts with the same value, and A control method characterized by outputting the operation amount to the control target.

5. The control method according to claim 4, When updating to a state model obtained by learning state models used in three control systems, one of the control systems not selected for controlling the controlled object is selected, and the state model inside the control system is updated to the learned state model one by one in order. A control method characterized by the above.

6. The control method according to claim 4, wherein the operation amounts of the control systems are compared, and when there are not two or more control systems that output the same operation amount, it is determined that an abnormality has occurred. A control method characterized by the above.

Images