Control Device, Control Method, and Recording Medium Recording a Control Program

Abstract

Provided are a control device, a control method, and a recording medium having a control program recorded thereon. The control device comprises: a control unit that controls the control object by using a control model obtained by machine learning in a manner that outputs an operation quantity of the control object according to the state of the device provided with the control object; a simulation unit that simulates the state of the device when the control object is given the operation quantity output by the control model by using the simulation model; and a stop unit that stops the control of the control object by the control model based on the simulation result.

Description

Technical Field

[0001] The present invention relates to a control device, a control method, and a recording medium recording a control program. Background Art

[0002] Patent document 1 states that "machine learning is performed on the correction amount of the robot's teaching position in response to the interference applied to the motors driving the joints of the robot, and based on the results of the machine learning, the robot is controlled while correcting the teaching position in a manner that suppresses the interference as it moves toward the teaching position."

[0003] Patent Document 1: Japanese Patent Application Laid-Open No. 2018-202564 Summary of the Invention

[0004] (Item 1)

[0005] In a first aspect of the present invention, a control device is provided. The control device may include a control unit that controls a controlled object using a control model obtained through machine learning to output an operation variable of the controlled object based on the state of the device provided with the controlled object. The control device may include a simulation unit that uses the simulation model to simulate the state of the device when the operation variable output by the control model is applied to the controlled object. The control device may include a stopping unit that stops control of the controlled object by the control model based on the simulation results.

[0006] (Item 2)

[0007] The stopping unit may stop the control of the control object by the control model when the simulation result indicates that an abnormality has occurred in the device.

[0008] (Item 3)

[0009] The control device may further include an output unit that outputs the simulation result. The stopping unit may stop the control of the controlled object by the control model when receiving an instruction to stop the control in response to the output of the simulation result.

[0010] (Item 4)

[0011] The control device may further include a frequency adjustment unit that adjusts a frequency for simulating the state of the device.

[0012] (Item 5)

[0013] The frequency adjustment unit may reduce the frequency of the simulation as the elapsed time from the start of control of the controlled object by the control model increases.

[0014] (Item 6)

[0015] The control device may further include a switching unit that switches whether to input or shut off the output of the control model to the controlled object. The stopping unit may shut off the switching unit when stopping the control of the controlled object by the control model.

[0016] (Item 7)

[0017] The switching unit may be formed of a physical switch.

[0018] (Item 8)

[0019] The control device may further include an instruction unit that instructs the controlled object to switch to control by another control unit when the control model stops controlling the controlled object.

[0020] (Item 9)

[0021] The simulation model may be a simple model that can simulate the state of the device in a shorter time than the actual operation of the device.

[0022] (Item 10)

[0023] The control device may further include a state data acquisition unit that acquires state data representing the state of the device. The control device may further include an operation quantity data acquisition unit that acquires operation quantity data representing the operation quantity. The control device may further include a control model generation unit that generates a control model through machine learning using the state data and the operation quantity data.

[0024] (Item 11)

[0025] The control model generation unit may generate the control model by performing reinforcement learning in accordance with input of state data so as to output an operation amount having a higher reward value defined by a predetermined reward function as a more recommended operation amount.

[0026] (Item 12)

[0027] In a second aspect of the present invention, a control method is provided. The control method may include the step of controlling a controlled object using a control model obtained through machine learning in a manner that outputs an operation variable of the controlled object based on the state of the device in which the controlled object is installed. The control method may include the step of simulating, using a simulation model, the state of the device when the operation variable output by the control model is applied to the controlled object. The control method may include the step of stopping control of the controlled object by the control model based on the simulation result.

[0028] (Item 13)

[0029] In a third aspect of the present invention, a recording medium is provided that records a control program. The control program can be executed by a computer. The control program can cause the computer to function as a control unit that controls the control object using a control model obtained by machine learning in a manner that outputs an operation amount of the control object according to the state of the device provided with the control object. The control program can cause the computer to function as a simulation unit that simulates the state of the device when the operation amount output by the control model is given to the control object using the simulation model. The control program can cause the computer to function as a stop unit that stops the control of the control object by the control model based on the simulation result.

[0030] The above summary of the invention does not list all the features of the present invention. In addition, sub-components of the above-mentioned feature groups can also constitute the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 An example of a block diagram of the device 10 provided with the control target 20 is shown together with the control apparatus 100 according to the present embodiment.

[0032] Figure 2 An example of a flow in which the control device 100 according to the present embodiment stops the AI control is shown.

[0033] Figure 3 An example of a block diagram of a device 10 provided with a control target 20 is shown together with a control apparatus 100 according to a first modification of the present embodiment.

[0034] Figure 4 An example of a flow in which the control device 100 according to the first modified example of the present embodiment stops the AI control is shown.

[0035] Figure 5 An example of a block diagram of a device 10 provided with a control target 20 is shown together with a control apparatus 100 according to a second modification of the present embodiment.

[0036] Figure 6 An example of a flow in which the control device 100 according to the second modified example of the present embodiment stops the AI control is shown.

[0037] Figure 7 An example of a block diagram of a device 10 provided with a control target 20 is shown together with a control apparatus 100 according to a third modified example of the present embodiment.

[0038] Figure 8 An example of a block diagram of a device 10 provided with a control target 20 is shown together with a control apparatus 100 according to a fourth modification of the present embodiment.

[0039] Figure 9 An example of a computer 9900 that can embody all or part of the various aspects of the present invention is shown. DETAILED DESCRIPTION

[0040] The present invention will be described below by way of embodiments of the invention, but the following embodiments do not limit the invention as defined in the claims. Furthermore, all combinations of features described in the embodiments are not necessarily essential for the solution to the problem.

[0041] Figure 1 An example of a block diagram of a device 10 equipped with a controlled object 20 is shown together with a control device 100 according to this embodiment. The control device 100 according to this embodiment simulates the state of the device 10 when the output of the learning model is applied to the controlled object 20, while controlling the controlled object 20 based on a learning model generated through machine learning (artificial intelligence, also known as AI control). Furthermore, the control device 100 according to this embodiment stops AI control based on the simulation results.

[0042] Equipment 10 is a facility, device, or the like equipped with a controlled object 20. For example, equipment 10 may be a factory or a complex device composed of multiple instruments. Examples of factories include, in addition to chemical and biological industrial plants, plants that manage and control the wellheads of gas and oil fields and their surrounding areas; plants that manage and control hydropower, thermal power, and nuclear power generation; plants that manage and control environmental resource generation such as solar and wind power; and plants that manage and control water supply and drainage systems, dams, and the like.

[0043] A control target 20 is provided for the device 10. In this figure, a case where only one control target 20 is provided for the device 10 is shown as an example, but the present invention is not limited to this. A plurality of control targets 20 may be provided for the device 10.

[0044] Furthermore, the device 10 may be provided with one or more sensors (not shown) that measure various states (physical quantities) inside and outside the device 10. Such sensors measure, for example, operating data, consumption data, and external environment data.

[0045] Here, the operating data represents the operating state resulting from controlling the controlled object 20. For example, the operating data may represent a measured value PV (Process Variable) measured for the controlled object 20, and as an example, may represent the output (controlled variable) of the controlled object 20, or may represent various values that change depending on the output of the controlled object 20.

[0046] The consumption data indicates the consumption of at least one of energy and raw materials by the equipment 10. For example, the consumption data may indicate the consumption of electricity or fuel (for example, LPG: Liquefied Petroleum Gas) as energy consumption.

[0047] External environment data represents physical quantities that can act as interference with the control of control object 20. For example, external environment data may represent the temperature, humidity, sunshine, wind direction, wind volume, precipitation, and other physical quantities of the atmosphere outside device 10 that vary with the control of other devices installed in device 10.

[0048] Controlled object 20 is an instrument or device that is controlled. For example, controlled object 20 may be an actuator such as a valve, pump, heater, fan, motor, or switch that controls at least one physical quantity of the process in equipment 10, such as pressure, temperature, pH, velocity, or flow rate. It receives a manipulated variable (MV) as input and outputs a controlled variable.

[0049] The control device 100 according to this embodiment simulates the state of the device 10 when the output of the learning model is applied to the controlled object 20 during control of the controlled object 20 based on the learning model generated by machine learning (AI control). The control device 100 according to this embodiment then stops the AI control based on the simulation results.

[0050] The control device 100 can be a computer such as a PC (personal computer), a tablet computer, a smart phone, a workstation, a server computer, or a general-purpose computer, or a computer system formed by connecting multiple computers. In addition, this computer system is also a computer in a broad sense. In addition, the control device 100 can be installed in a computer using one or more executable virtual computer environments. Alternatively, the control device 100 can be a dedicated computer designed for AI control, or it can be dedicated hardware implemented using dedicated circuits. In addition, when the control device 100 can be connected to the Internet, the control device 100 can be implemented through cloud computing.

[0051] The control device 100 includes a state data acquisition unit 110, an operation variable data acquisition unit 120, a control model generation unit 130, a control model 135, a control unit 140, a simulation unit 150, a simulation model 155, and a stop unit 160. Furthermore, these modules may be functionally separate modules and may not correspond to the actual device structure. That is, while shown as a single module in this figure, they do not need to be composed of a single device. Furthermore, while shown as separate modules in this figure, they do not need to be composed of separate devices.

[0052] The status data acquisition unit 110 acquires status data representing the status of the device 10 in which the control target 20 is installed. For example, the status data acquisition unit 110 acquires operating data, consumption data, and external environmental data measured by sensors installed in the device 10 via a network as status data. However, this is not a limitation. The status data acquisition unit 110 may acquire this status data from an operator or from various memory devices. The status data acquisition unit 110 supplies the acquired status data to the control model generation unit 130 and the control model 135.

[0053] The manipulated variable data acquisition unit 120 acquires manipulated variable data representing the manipulated variable of the controlled object 20. For example, the manipulated variable data acquisition unit 120 acquires, from the control unit 140, data representing the manipulated variable MV(AI) output by the control model 135 when AI control is performed on the controlled object 20 as the manipulated variable data. However, this is not a limitation. The manipulated variable data acquisition unit 120 may acquire such manipulated variable data from the operator or from various memory devices. The manipulated variable data acquisition unit 120 supplies the acquired manipulated variable data to the control model generation unit 130.

[0054] Furthermore, this figure illustrates, as an example, a case where the manipulated variable data acquisition unit 120 acquires data representing the manipulated variable MV(AI) output by the control model 135 as manipulated variable data. However, this is not limiting. When the control device 100 uses data from the control of the controlled object 20 by another controller (not shown) as learning data to perform machine learning on the model, the manipulated variable data acquisition unit 120 may acquire data representing the manipulated variable applied to the controlled object 20 from the other controller as manipulated variable data. As an example, when the controlled object 20 is capable of switching between feedback control based on the manipulated variable MV (FB) applied by the other controller and AI control based on the manipulated variable MV(AI) applied by the control model 135, the manipulated variable data acquisition unit 120 may acquire data representing the manipulated variable MV(FB) applied to the controlled object 20 from the other controller as manipulated variable data. Furthermore, this FB control may be, for example, control utilizing at least one of proportional control (P control), integral control (I control), or differential control (D control), and may also be PID control. In addition, such another controller may be integrally configured as a part of the control device 100 according to the present embodiment, or may be configured as a separate structure independent of the control device 100 .

[0055] The control model generation unit 130 uses state data and manipulated variable data through machine learning to generate a control model 135 that outputs manipulated variables corresponding to the state of the device 10. For example, the control model generation unit 130 performs reinforcement learning using the state data supplied by the state data acquisition unit 110 and the data representing the manipulated variable MV (AI) supplied by the manipulated variable data acquisition unit 120 as learning data, thereby generating a control model 135 that outputs the manipulated variable MV (AI) corresponding to the state of the device 10. Specifically, the control model generation unit 130 generates the control model 135 by performing reinforcement learning based on the input state data, such that the manipulated variable with a higher reward value specified by a predetermined reward function is output as a more recommended manipulated variable. This will be described in detail later.

[0056] The control model 135 is a learning model generated by the control model generation unit 130 through reinforcement learning, and outputs the operation amount MV (AI) corresponding to the state of the device 10. For example, the control model 135 inputs the state data supplied from the state data acquisition unit 110, and outputs the recommended operation amount MV (AI) that should be given to the control object 20 according to the state of the device 10. In addition, in this figure, as an example, the control model 135 is shown as being built into the control device 100, but it is not limited to this. The control model 135 can be stored in a device different from the control device 100 (for example, a cloud server). Similarly, the control model generation unit 130 can also be equipped with a device different from the control device 100.

[0057] The control unit 140 supplies the manipulated variable MV (AI) output by the control model 135 to the controlled object 20. Thus, the control unit 140 controls the controlled object 20 using the control model 135, which has undergone machine learning to output the manipulated variable of the controlled object 20 in accordance with the state of the device 10 in which the controlled object 20 is installed. Furthermore, the control unit 140 supplies the manipulated variable MV (AI) output by the control model 135 to the simulation unit 150 and the manipulated variable data acquisition unit 120.

[0058] The simulation unit 150 uses a simulation model 155 to simulate the state of the device 10 when the manipulated variable MV (AI) output by the control model 135 is applied to the controlled object 20. The term "simulation" herein encompasses not only the simulation unit 150 itself simulating the state of the device 10, but also the simulation unit 150 causing another device (e.g., a simulator (not shown)) to simulate the state of the device 10, and obtaining the simulated state of the device 10 from another device. For example, the simulation unit 150 inputs the manipulated variable MV (AI) output by the control model 135 into the simulation model 155 and obtains multiple output values from the simulation model 155 as simulation results. The simulation unit 150 supplies the obtained simulation results to the stop unit 160.

[0059] The simulation model 155 is a model (e.g., a plant model) constructed in a manner that simulates the action of the device 10. For example, the simulation model 155 inputs the manipulated variable MV (AI) and simulates the action of the device 10 when the manipulated variable MV (AI) is assigned to the control object 20. Furthermore, the simulation model 155 outputs a plurality of output values representing the state of the simulated device 10. As an example, the simulation model 155 may be a simple model that can simulate the state of the device 10 in a shorter time than the actual action of the device 10, such as a simple physical model with a lighter processing load, or a linear model with a lower dimension. In addition, in this figure, as an example, a case where the simulation model 155 is built into the control device 100 is shown, but it is not limited to this. Similar to the control model 135, the simulation model 155 can be stored in a device different from the control device 100 (e.g., a cloud server). In addition, the above-mentioned simulator can be equipped in a device different from the control device 100.

[0060] The stop unit 160 stops the control of the controlled object 20 by the control model 135 based on the simulation results. For example, the stop unit 160 determines whether the simulation results supplied by the simulation unit 150 satisfy predetermined conditions (e.g., abnormality diagnosis conditions). Furthermore, if the simulation results indicate that an abnormality has occurred in the device 10, the stop unit 160 notifies the control unit 140 of this fact. In response, the control unit 140 stops supplying the manipulated variable MV (AI) output by the control model 135 to the controlled object 20. Thus, if the simulation results indicate that an abnormality has occurred in the device 10 (e.g., if an abnormality is predicted to occur in the device 10 within a few days), the stop unit 160 stops the control of the controlled object 20 by the control model 135.

[0061] Figure 2 An example of a flow in which the control device 100 according to the present embodiment stops the AI control is shown.

[0062] In step 210, the control device 100 acquires status data. For example, the status data acquisition unit 110 acquires status data indicating the status of the device 10 in which the control target 20 is installed. As an example, the status data acquisition unit 110 acquires operating data, consumption data, and external environment data measured by sensors installed in the device 10 via a network as status data. The status data acquisition unit 110 supplies the acquired status data to the control model generation unit 130 and the control model 135.

[0063] In step 220, the control device 100 acquires the operation quantity data. For example, the operation quantity data acquisition unit 120 acquires the operation quantity data representing the operation quantity of the control object 20. As an example, the operation quantity data acquisition unit 120 acquires data representing the operation quantity MV (AI) output by the control model 135 when the control object 20 is subjected to AI control from the control unit 140 as the operation quantity data. The operation quantity data acquisition unit 120 supplies the acquired operation quantity data to the control model generation unit 130. In addition, in this figure, as an example, a case where the control device 100 acquires the operation quantity data after acquiring the status data is shown, but it is not limited to this. The control device 100 may acquire the status data after acquiring the operation quantity data, or may acquire the status data and the operation quantity data at the same time.

[0064] In step 230, control device 100 generates control model 135. For example, control model generation unit 130 utilizes state data and manipulated variable data through machine learning to generate control model 135 that outputs manipulated variables corresponding to the state of device 10. As an example, control model generation unit 130 performs reinforcement learning using the state data acquired in step 210 and the data representing manipulated variable MV (AI) acquired in step 220 as learning data, thereby generating control model 135 that outputs manipulated variable MV (AI) corresponding to the state of device 10.

[0065] Typically, if an agent observes the state of the environment and chooses an action, the environment changes based on that action. In reinforcement learning, by giving a certain reward along with changes in the environment, the agent learns to choose a better action (willingness decision). In contrast to learning with a teacher, which can obtain a completely correct answer, in reinforcement learning, rewards are given as discontinuous values based on changes in part of the environment. Therefore, the agent learns by choosing the action that will maximize the total reward in the future. In this way, in reinforcement learning, the agent learns appropriate actions by learning actions and considering the interactions that actions have on the environment, that is, learning actions to maximize future rewards.

[0066] In this embodiment, the reward for reinforcement learning can be an indicator used to evaluate the operation of the device 10, or a value determined by a predetermined reward function. Here, a function refers to a mapping with a rule that establishes a one-to-one correspondence between elements of another set and elements of a particular set. For example, it can be a mathematical formula or a table.

[0067] The reward function outputs a value (reward value) that evaluates the state of the device 10 represented by the state data based on the input of the state data. As described above, for example, the state data includes the measured value PV measured for the control object 20. Therefore, the reward function can be defined as a function in which the closer the measured value PV is to the target value SV (Setting Variable), the higher the reward value. Here, a function with the absolute value of the difference between the measured value PV and the target value SV as a variable can be defined as an evaluation function. That is, as an example, when the control object 20 is a valve, the evaluation function can be a function with the absolute value of the difference between the valve opening actually measured by the sensor, that is, the measured value PV, and the valve opening set as the target, that is, the target value SV, as a variable. Furthermore, the reward function can be a function with the value of the evaluation function obtained by this evaluation function as a variable.

[0068] Furthermore, as described above, in addition to the measured value PV, the status data also includes, for example, various values that change depending on the output of the controlled object 20, consumption data, external environmental data, and the like. Therefore, the reward function can be a function that increases or decreases the reward value based on these various values, consumption data, and external environmental data. As an example, if constraints are imposed on these various values or consumption data, the reward function can be a function that minimizes the reward value if, with reference to the external environmental data, these various values or consumption data do not meet the constraints. Furthermore, if targets are imposed on these various values or consumption data, the reward function can be a function that increases the reward value as these various values or consumption data approach the targets, and decreases the reward value as these values or consumption data move further away from the targets, with reference to the external environmental data.

[0069] The control model generation unit 130 obtains reward values for various learning data based on this reward function. Furthermore, the control model generation unit 130 performs reinforcement learning using each set of learning data and reward values. In this case, the control model generation unit 130 can perform learning based on well-known methods such as steepest descent, neural networks, DQN (Deep Q-Network), Gaussian processes, and deep learning. Furthermore, the control model generation unit 130 performs learning such that higher reward values are preferentially output as more recommended operation variables. Specifically, the control model generation unit 130 performs reinforcement learning based on the input state data such that higher reward values, as defined by a predetermined reward function, are output as more recommended operation variables, thereby generating the control model 135. This updates the model and generates the control model 135.

[0070] In step 240, the control device 100 performs AI control on the controlled object 20. For example, the control unit 140 supplies the manipulated variable MV (AI) output by the control model 135 to the controlled object 20, and controls the controlled object 20 using the control model 135, which has undergone machine learning to output the manipulated variable of the controlled object 20 based on the state of the device 10 in which the controlled object 20 is installed. In parallel, the control unit 140 supplies the manipulated variable MV (AI) output by the control model 135 to the simulation unit 150.

[0071] In step 250, the control device 100 performs a simulation. For example, the simulation unit 150 uses the simulation model 155 to simulate the state of the device 10 when the manipulated variable MV (AI) output by the control model 135 is applied to the controlled object 20. As an example, the simulation unit 150 inputs the manipulated variable MV (AI) output by the control model 135 generated in step 230 into the simulation model 155 and obtains a plurality of output values output by the simulation model 155 as simulation results. In this way, in parallel with the AI control, the control device 100 simulates the state of the device 10 when the manipulated variable MV (AI) output by the control model 135 is applied to the controlled object 20. The simulation unit 150 supplies the obtained simulation results to the stop unit 160.

[0072] In step 255, control device 100 determines whether an abnormality has occurred in device 10. For example, stop unit 160 may pre-store abnormality diagnosis conditions for diagnosing abnormalities in device 10. Furthermore, if none of the multiple output values output by simulation model 155 satisfy the abnormality diagnosis conditions, stop unit 160 may infer that no abnormality has occurred in device 10. Alternatively, if at least one of the multiple output values output by simulation model 155 satisfies the abnormality diagnosis conditions, stop unit 160 may infer that an abnormality has occurred in device 10.

[0073] In step 255, if the simulation result does not indicate that an abnormality has occurred in the device 10 (in the case of No), the control device 100 returns the process to step 250 to continue the flow. In this case, the AI control of step 240 is continued.

[0074] In step 255 , when the simulation result indicates that an abnormality has occurred in the device 10 (if Yes), the control device 100 advances the process to step 260 .

[0075] In step 260, the control device 100 stops AI control. For example, the stopping unit 160 stops the control of the controlled object 20 by the control model 135 based on the simulation results. As an example, if the simulation results indicate that an abnormality has occurred in the device 10, the stopping unit 160 notifies the control unit 140 of this fact. In response, the control unit 140 stops supplying the manipulated variable MV (AI) output by the control model 135 to the controlled object 20. Thus, if the simulation results indicate that an abnormality has occurred in the device 10, the stopping unit 160 stops the control of the controlled object 20 by the control model 135.

[0076] Typically, in machine learning, input data is used to determine the parameters of the learning model. These parameters are subject to probability and cannot be theoretically guaranteed. Therefore, the learning model may output abnormal inference data. Therefore, in parallel with AI control of the controlled object 20, the control device 100 according to this embodiment uses a simulation model 155 to simulate the state of the device 10 when the manipulated variable MV(AI) output by the control model 135 is applied to the controlled object 20. Furthermore, the control device 100 stops the AI control based on the simulation results. Thus, the control device 100 according to this embodiment can stop the AI control if it is inferred that the device 10 is operating abnormally while under AI control. It is also conceivable to determine whether to stop the AI control based on whether the manipulated variable MV(AI) output by the control model 135 meets a predetermined criterion. However, such a criterion is artificially assigned based on experience, and even if the manipulated variable MV(AI) meets this criterion, the device 10 may not be abnormal. Similarly, even if the manipulated variable MV(AI) does not meet this criterion, the device 10 may not be abnormal. In contrast, the control device 100 according to this embodiment determines whether to stop the AI control based on the result obtained by simulating the state of the device 10 when the operating variable MV (AI) is given to the control object 20, rather than the operating variable MV (AI) itself. Therefore, the stopping of the AI control can be determined based on a basis that is closer to the actual operation.

[0077] Furthermore, in the control device 100 according to this embodiment, the simulation model 155 used for simulation can be a simpler model than the actual device 10. Thus, according to the control device 100 according to this embodiment, the state of the device 10 can be simulated in a shorter time than the actual operation of the device 10, and thus AI control can be stopped before an abnormality occurs in the actual device 10.

[0078] Figure 3 An example of a block diagram of a device 10 provided with a control target 20 is shown together with a control device 100 according to a first modification of the present embodiment. Figure 3 In the Figure 1 Components with the same functions and structures are designated by the same reference numerals, and descriptions thereof will be omitted except for the following differences. The control device 100 according to the above embodiment illustrates an example in which the AI control is automatically stopped based on simulation results. However, the control device 100 according to the first modification outputs the simulation results, and AI control is stopped based on instructions from an operator, etc., who is analyzing the simulation results. The control device 100 according to the first modification further includes an output unit 310 and an input unit 320.

[0079] In the control device 100 according to this variation, the simulation unit 150, in addition to the stop unit 160, also supplies the simulation results to the output unit 310. Furthermore, the output unit 310 outputs the simulation results. For example, the output unit 310 may output the simulation results by displaying them on a monitor, printing them, or transmitting the data to another device.

[0080] The input unit 320 receives user input from an operator who examines the simulation results, etc., in response to the output of the simulation results. The input unit 320 supplies the stop unit 160 with the instruction from the operator input by the user.

[0081] Stop unit 160 determines to stop AI control when the instruction supplied from input unit 320 indicates that AI control should be stopped. In other words, when stopping unit 160 receives an instruction to stop control in response to outputting simulation results, it stops control model 135 from controlling object 20.

[0082] Figure 4 An example of a flow in which the control device 100 according to the first modified example of the present embodiment stops the AI control is shown. Figure 4 In, with Figure 2 The same processing is denoted by the same reference numerals, and description thereof will be omitted except for the following differences: In this flow, step 255 is replaced with steps 410 and 420 .

[0083] In step 410 , the control device 100 outputs the simulation result. For example, the output unit 310 acquires the simulation result of the simulation unit 150 in step 250 and outputs the simulation result by displaying it on a monitor.

[0084] In step 420, the control device 100 determines whether an instruction to stop the AI control has been given. For example, the stop unit 160 determines whether an instruction to stop the AI control has been received from an operator examining the simulation results, etc., via the input unit 320. If no instruction to stop the AI control has been received in step 420 (if "No"), the control device 100 returns the process to step 250 and continues the flow. If an instruction to stop the AI control has been received in step 420 (if "Yes"), the control device 100 proceeds to step 260. Furthermore, the stop unit 160 notifies the control unit 140 of the instruction to stop the AI control. In response, the control unit 140 stops supplying the manipulated variable MV (AI) output by the control model 135 to the controlled object 20. Thus, when the stop unit 160 receives an instruction to stop the control in accordance with the output of the simulation results, it stops the control model 135 from controlling the controlled object 20.

[0085] Thus, the control device 100 according to the first modification outputs simulation results and stops AI control based on an instruction from an operator or the like who examines the simulation results. Thus, the control device 100 according to the first modification can reflect the operator's intention when stopping AI control.

[0086] In addition, the above description shows an example in which the control device 100 executes steps 410 and 420 instead of step 255, but the present invention is not limited to this. The control device 100 involved in the first modification can execute steps 410 and 420 in addition to step 255. In this case, the control device 100 can stop the AI control when either a stop instruction from an operator or an automatic determination by the computer is satisfied. Alternatively, the control device 100 can first stop the AI control when both a stop instruction from the operator and an automatic determination by the computer are satisfied. In this case, the control device 100 can, for example, output a simulation result indicating that an abnormality has occurred in the device 10 and notify the operator of the abnormality. In response, the control device 100 can stop the AI control when a stop instruction is received from the operator. Thus, according to the control device 100 involved in the first modification, the AI control can be stopped using both the automatic determination by the computer and the manual determination by the operator.

[0087] Figure 5 An example of a block diagram of a device 10 provided with a control target 20 is shown together with a control device 100 according to a second modification of the present embodiment. Figure 5 In the Figure 1Components with the same functions and structures are designated by the same reference numerals, and descriptions thereof will be omitted except for the following differences. In the control device 100 according to the above embodiment, as an example, when the control model 135 outputs the manipulated variable MV(AI), the state of the device 10 when the manipulated variable MV(AI) is applied to the controlled object 20 is always simulated. However, in the control device 100 according to the second modification, the frequency of the simulated state of the device 10 is adjusted. The control device 100 according to the second modification further includes a frequency adjustment unit 510.

[0088] The frequency adjustment unit 510 triggers the simulation unit 150 to perform a simulation. For example, when the simulation is timed, the frequency adjustment unit 510 instructs the simulation unit 150 to simulate the state of the device 10. In response, the simulation unit 150 simulates the state of the device 10. In this case, the frequency adjustment unit 510 can be configured to adjust the frequency of simulating the state of the device 10. For example, the frequency adjustment unit 510 can decrease the simulation frequency as the time elapsed since the control model 135 began controlling the controlled object 20 increases.

[0089] Figure 6 FIG. 2 shows an example of a flow in which the control device 100 according to the second modified example of the present embodiment stops the AI control. Figure 6 In, with Figure 2 The same processing is marked with the same reference numerals, and description thereof is omitted except for the following differences. This flow also includes steps 610 and 620.

[0090] In step 610, the control device 100 (e.g., the frequency adjustment unit 510) determines whether the timing is simulated. The initial values of the simulated timing and the simulated interval may be stored in advance. If the determination in step 610 is that the timing is not simulated ("No"), the control device 100 returns the process to step 610 and continues the flow.

[0091] If it is determined in step 610 that it is simulation timing (if Yes), the control device 100 triggers simulation by the simulation unit 150. For example, in the case of simulation timing, the frequency adjustment unit 510 instructs the simulation unit 150 to simulate the state of the device 10. In response, in step 250, the simulation unit 150 simulates the state of the device 10.

[0092] Then, in step 255 , when the simulation result does not indicate that an abnormality has occurred in the device 10 (in the case of No), the control device 100 advances the processing to step 620 .

[0093] In step 620, the control device 100 reduces the simulation frequency. For example, the frequency adjustment unit 510 adds a fixed length to the pre-stored simulation interval and updates the simulation interval. That is, the frequency adjustment unit 510 lengthens the interval until the next simulation timing and reduces the frequency of the simulation. In this way, when the AI control is not stopped and the time elapsed since the start of the AI control becomes longer, the frequency adjustment unit 510 determines that the AI control is stable and the simulation frequency can be reduced. Moreover, the control device 100 returns the processing to step 610 and continues the process. In this way, in the control device 100 involved in the second variant, the frequency adjustment unit 510 can be set to be able to adjust the frequency of simulating the state of the device 10. In more detail, the frequency adjustment unit 510 can reduce the frequency of the simulation as the time elapsed since the control model 135 starts controlling the control object 20 becomes longer.

[0094] The control device 100 according to the second modification can adjust the frequency of simulating the state of the device 10. In particular, the control device 100 according to the second modification reduces the frequency of simulation as the time elapsed since the start of AI control increases. That is, the control device 100 according to the second modification frequently performs simulations immediately after the start of AI control, and reduces the frequency of simulation execution as a longer period of time passes after the start of AI control. Thus, the control device 100 according to the second modification can adjust the frequency of simulations in accordance with the actual results of AI control, thereby reducing the processing load of the simulation on the control device 100.

[0095] Figure 7 An example of a block diagram of a device 10 provided with a control target 20 is shown together with a control device 100 according to a third modified example of the present embodiment. Figure 7 In the Figure 1 Components with the same functions and structures are designated by the same reference numerals, and description thereof will be omitted except for the following differences. In the control device 100 according to the above embodiment, a case where the output of the control unit 140 (i.e., the output of the control model 135) is directly connected to the controlled object 20 is shown as an example. However, in the control device 100 according to the third modified example, it is possible to switch whether the output of the control model 135 is input to the controlled object 20 or is disconnected. The control device 100 according to the third modified example further includes a switching unit 710.

[0096] One end of the switching unit 710 is connected to the output of the control unit 140 (i.e., the output of the control model 135), and the other end is connected to the input of the controlled object 20. Furthermore, the switching unit 710 switches whether the output of the control model 135 is connected to the input of the controlled object 20 or disconnected. This switching unit 710 can be, for example, a switch that opens and closes an electrical circuit, and in particular, can be a physical switch that physically disconnects the circuit. In the control device 100 according to the third modification, the stopping unit 160 disconnects the switching unit 710 when stopping the control of the controlled object 20 by the control model 135.

[0097] In this manner, the control device 100 according to the third modification can switch between inputting and shutting off the output of the control model 135 to the controlled object 20. Thus, according to the control device 100 according to the third modification, when AI control is stopped, the supply of the output of the control model 135 to the controlled object 20 can be physically shut off, thereby preventing the output of the control model 135 from being erroneously input to the controlled object 20.

[0098] Figure 8 An example of a block diagram of a device 10 provided with a control target 20 is shown together with a control device 100 according to a fourth modification of the present embodiment. Figure 8 In the Figure 1 Components with the same functions and structures are designated by the same reference numerals, and descriptions thereof will be omitted except for the following differences. The control device 100 according to the above embodiment illustrates, as an example, a case where only AI control is stopped based on simulation results. However, in the control device 100 according to the fourth modification, when AI control is stopped, the control device 100 instructs the controlled object 20 to switch to control by another control unit. The control device 100 according to the fourth modification further includes an instruction unit 810.

[0099] In the fourth modification, the controlled object 20 can be switched between feedback control based on the manipulated variable MV(FB) given from another controller (not shown) and AI control based on the manipulated variable MV(AI) given from the control model.

[0100] Furthermore, in the control device 100 according to the fourth modification, when the stopping unit 160 stops the AI control, the stopping unit 160 notifies the instructing unit 810 of the fact.

[0101] Accordingly, when the control model 135 stops controlling the controlled object 20, the instruction unit 810 instructs the controlled object 20 to switch to control by another control unit. For example, when the controlled object 20 can perform feedback control in addition to AI control, the instruction unit 810 instructs the controlled object 20 to switch to feedback control.

[0102] The control device 100 according to the fourth modification instructs the controlled object 20 to switch to control by another control unit when AI control is stopped. Thus, the control device 100 according to the fourth modification allows the controlled object 20 to continue to be controlled by another control unit even when AI control is stopped.

[0103] Various embodiments of the present invention may be described with reference to flowcharts and block diagrams, where a module may represent (1) a stage of a process for performing an operation, or (2) a portion of a device having the function of performing an operation. Specific stages and portions may be implemented using dedicated circuits, programmable circuits supplied with computer-readable instructions stored on a computer-readable medium, and / or processors supplied with computer-readable instructions stored on a computer-readable medium. Dedicated circuits may include digital and / or analog hardware circuits, as well as integrated circuits (ICs) and / or discrete circuits. Programmable circuits may include reconfigurable hardware circuits such as logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, trigger circuits, registers, memory elements such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0104] The computer-readable medium may include any tangible device capable of storing instructions for execution using an appropriate device. As a result, the computer-readable medium having the instructions stored therein has a product containing executable instructions for making a means for performing the operations specified in the flowchart or block diagram. Examples of computer-readable media include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable media include Floppy (registered trademark) floppy disks, floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray (RTM) disks, memory sticks, integrated circuit cards, etc.

[0105] Computer readable instructions may include any source code or object code described in assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or any combination of one or more programming languages including object-oriented programming languages such as Smalltalk, JAVA (registered trademark), C++, and procedural programming languages such as the "C" programming language or similar programming languages.

[0106] Computer-readable instructions may be provided to a processor or programmable circuit of a general-purpose computer, a special-purpose computer, or other programmable data processing device via a local area network (LAN), a wide area network (WAN), or the like, and the computer-readable instructions may be executed to create a means for performing the operations specified in the flowchart or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, and microcontrollers.

[0107] Figure 9 This represents an example of a computer 9900 that can embody all or part of the various aspects of the present invention. The program installed in computer 9900 can cause computer 9900 to function as an operation associated with a device according to an embodiment of the present invention or one or more components of such a device, or to execute such operation or such one or more components, and / or can cause computer 9900 to perform a process or a stage of such a process according to an embodiment of the present invention. This program can be executed by CPU 9912 to cause computer 9900 to perform specific operations associated with some or all of the modules in the flowcharts and block diagrams described in this specification.

[0108] The computer 9900 according to this embodiment includes a CPU 9912, a RAM 9914, a graphics controller 9916, and a display device 9918, and these components are interconnected via a main controller 9910. Furthermore, the computer 9900 includes input / output units such as a communication interface 9922, a hard disk drive 9924, a DVD-ROM drive 9926, and an IC card drive, and these components are connected to the main controller 9910 via an input / output controller 9920. Furthermore, the computer includes a ROM 9930 and conventional input / output units such as a keyboard 9942, and these components are connected to the input / output controller 9920 via an input / output chip 9940.

[0109] The CPU 9912 controls each unit by executing operations according to programs stored in the ROM 9930 and the RAM 9914. The graphics controller 9916 obtains image data generated by the CPU 9912 in a frame buffer or the like or in the graphics controller itself and supplies it to the RAM 9914, and displays the image data on the display device 9918.

[0110] The communication interface 9922 communicates with other electronic devices via a network. The hard disk drive 9924 stores programs and data used by the CPU 9912 in the computer 9900. The DVD-ROM drive 9926 reads programs and data from the DVD-ROM 9901 and provides the programs and data to the hard disk drive 9924 via the RAM 9914. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0111] The ROM 9930 stores therein a startup program and the like executed by the computer 9900 upon activation, and / or programs that depend on the hardware of the computer 9900. In addition, the input / output chip 9940 connects various input / output units to the input / output controller 9920 via a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0112] The program is provided on a computer-readable medium such as a DVD-ROM 9901 or an IC card. The program is read from the computer-readable medium and installed in a hard disk drive 9924, a RAM 9914, or a ROM 9930, also examples of computer-readable media, and then executed by the CPU 9912. The information processing described in the program is read by the computer 9900, and the program and the various types of hardware resources described above are coordinated. By using the computer 9900 to implement information manipulation or processing, an apparatus or method can be constructed.

[0113] For example, when communication is performed between the computer 9900 and an external device, the CPU 9912 can execute a communication program loaded in the RAM 9914 and, based on the processing described in the communication program, issue communication processing instructions to the communication interface 9922. Under the control of the CPU 9912, the communication interface 9922 reads transmission data stored in a transmission buffer area provided in the RAM 9914, the hard disk drive 9924, the DVD-ROM 9901, or a recording medium such as an IC card, and transmits the read transmission data to the network or writes reception data received from the network to a reception buffer area provided on the recording medium.

[0114] Furthermore, the CPU 9912 can read all or a required portion of a file or database stored in an external recording medium such as the hard disk drive 9924, DVD-ROM drive 9926 (DVD-ROM 9901), or IC card into the RAM 9914, and perform various types of processing on the data in the RAM 9914. The CPU 9912 then writes the processed data back to the external recording medium.

[0115] Various types of information such as various types of programs, data, tables, and databases can be stored in a recording medium and subjected to information processing. The CPU 9912 can perform various types of processing including various types of operations, information processing, conditional judgments, conditional branches, unconditional branches, information retrieval / replacement, etc., which are recorded at any location in the present disclosure and specified by the instruction sequence of the program, on the data read from the RAM 9914, and write the results back to the RAM 9914. In addition, the CPU 9912 can retrieve information in files, databases, etc. in the recording medium. For example, in the case where a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 9912 can retrieve an entry that specifies an attribute value of the first attribute that is consistent with the condition from the plurality of records, read the attribute value of the second attribute stored in the entry, and obtain the attribute value of the second attribute associated with the first attribute that satisfies the condition predetermined thereby.

[0116] The program or software module described above can be stored in a computer-readable medium on or near the computer 9900. In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable medium, thereby providing the program to the computer 9900 via the network.

[0117] The present invention has been described above using the embodiments. However, the technical scope of the present invention is not limited to the scope described in the above embodiments. Those skilled in the art will appreciate that various modifications or improvements may be made to the above embodiments. As is clear from the claims, embodiments incorporating such modifications or improvements are also within the technical scope of the present invention.

[0118] Regarding the order in which actions, sequences, steps, and stages, etc., of the apparatuses, systems, programs, and methods described in the claims, specifications, and drawings are executed, it should be noted that unless otherwise expressly indicated as "before," "before," or the like, and unless the output of a previous process is used in a subsequent process, the execution order may be any order. Even if the flow of actions in the claims, specifications, and drawings is described using the phrases "first," "next," or the like for convenience, it does not necessarily mean that the actions must be executed in that order.

[0119] Description of the label

[0120] 10 Equipment

[0121] 20 Control Objects

[0122] 100 Control Device

[0123] 110 Status data acquisition unit

[0124] 120 Operation volume data acquisition unit

[0125] 130 Control Model Generation Unit

[0126] 135 Control Model

[0127] 140 Control Department

[0128] 150 Simulation Department

[0129] 155 simulation model

[0130] 160 Stop

[0131] 310 Output

[0132] 320 Input

[0133] 510 Frequency Adjustment Unit

[0134] 710 Switching Unit

[0135] 810 Instruction Department

[0136] 9900 Computer

[0137] 9901 DVD-ROM

[0138] 9910 Master Controller

[0139] 9912 CPU

[0140] 9914 RAM

[0141] 9916 Graphics Controller

[0142] 9918 Display Device

[0143] 9920 Input / Output Controller

[0144] 9922 Communication Interface

[0145] 9924 Hard Drive

[0146] 9926 DVD drive

[0147] 9930 ROM

[0148] 9940 Input / Output Chip

[0149] 9942 Keyboard

Claims

1. A control device, wherein: The control device has: a control unit that controls the control object using a control model obtained by machine learning in a manner that outputs an operation amount of the control object according to a state of a device provided with the control object; a simulation unit that simulates, using a simulation model, a state of the device when the manipulated variable output by the control model is applied to the controlled object; a frequency adjustment unit configured to adjust a frequency for simulating the state of the device; as well as a stopping unit that stops the control of the control object by the control model based on a simulation result, The frequency adjustment unit reduces the frequency of the simulation as the elapsed time from the start of control of the control object by the control model increases.

2. The control device according to claim 1, wherein: The stopping unit stops the control of the control object by the control model when the simulation result indicates that an abnormality has occurred in the device.

3. The control device according to claim 1 or 2, wherein: The control device further includes an output unit for outputting the simulation result. The stopping unit stops the control of the control object by the control model when receiving an instruction to stop the control in response to the output of the simulation result.

4. The control device according to claim 1 or 2, wherein: The control device further includes a switching unit that switches whether the output of the control model is input to the control object or is cut off. The stopping unit turns off the switching unit when stopping the control of the control object by the control model.

5. The control device according to claim 4, wherein: The switching unit is composed of a physical switch.

6. The control device according to claim 1 or 2, wherein: The control device further includes an instruction unit that, when the control model stops controlling the controlled object, instructs the controlled object to switch to control by another control unit.

7. The control device according to claim 1 or 2, wherein: The simulation model is a simple model that can simulate the state of the device in a shorter time than the actual operation of the device.

8. The control device according to claim 1 or 2, wherein: The control device also has: a status data acquisition unit configured to acquire status data indicating a status of the device; an operation amount data acquisition unit that acquires operation amount data representing the operation amount; as well as A control model generation unit generates the control model through machine learning using the state data and the operation amount data.

9. The control device according to claim 8, wherein: The control model generation unit generates the control model by performing reinforcement learning in accordance with the input of the state data so as to output an operation amount having a higher reward value defined by a predetermined reward function as a more recommended operation amount.

10. A control method, wherein: The control method has the following steps: Controlling the control object using a control model obtained by machine learning in a manner that outputs an operation amount of the control object according to a state of a device provided with the control object; simulating, using a simulation model, a state of the device when the manipulated variable output by the control model is applied to the controlled object; adjusting the frequency of simulating the state of the device; and Based on the simulation result, the control model stops controlling the controlled object. Adjusting the frequency of the simulation includes reducing the frequency of the simulation as the elapsed time from the start of control of the control object by the control model increases.

11. A recording medium having a control program recorded thereon, wherein: The control program is executed by a computer, causing the computer to function as the following functional unit: a control unit that controls the control object using a control model obtained by machine learning in a manner that outputs an operation amount of the control object according to a state of a device provided with the control object; a simulation unit that simulates, using a simulation model, a state of the device when the manipulated variable output by the control model is applied to the controlled object; a frequency adjustment unit configured to adjust a frequency for simulating the state of the device; as well as a stopping unit that stops the control of the control object by the control model based on a simulation result, The frequency adjustment unit reduces the frequency of the simulation as the elapsed time from the start of control of the control object by the control model increases.

Images