Control device, control method and computer program product
By introducing a switching unit into the control device, switching feedback control and AI control based on the difference between the measured value and the target value, the problems of unstable operation and low efficiency in the equipment control are solved, and stable and efficient control of the equipment operation are achieved.
Patent Information
- Application Number
- CN202110440905.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-04-24
- Filing Date
- 2021-04-23
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-04-23
AI Technical Summary
The prior art is difficult to effectively switch between feedback control and learning control in equipment control, resulting in unstable equipment operation and inefficient efficiency.
A control device and method are provided, by switching between the feedback control unit and the AI control unit, switching according to the difference between the measured value and the target value and the reward function, and optimizing the equipment operation amount by using the learning model to improve the equipment operation efficiency.
The stability and efficiency of equipment operation volume are improved, and the control strategy can be automatically switched under different conditions, quickly approaching the target value, and reducing human intervention.
Smart Images

Figure CN113552842B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a control device, a control method and a recording medium. Background Art
[0002] Conventionally, various methods of controlling a device have been proposed (for example, refer to Patent Document 1).
[0003] Prior art literature
[0004] Patent Literature
[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-202564 Summary of the invention
[0006] In order to solve the above-mentioned problems, in a first embodiment of the present invention, a control device is provided. The control device may include an acquisition unit that acquires a measured value measured for a control target device. The control device may include a first control unit that outputs an operation amount of the control target device corresponding to the measured value through at least one of feedback control and feedforward control. The control device may include a second control unit that uses a model learned using learning data to output an operation amount of the control target device corresponding to the measured value. The control device may include a switching unit that switches which of the first control unit and the second control unit controls the control target device.
[0007] The switching unit may switch according to the difference between the measured value and the target value.
[0008] The switching unit may switch from the control by the first control unit to the control by the second control unit when the difference between the measured value and the target value becomes larger than the reference value a plurality of times within the reference time window.
[0009] The first control unit may output an operation amount calculated based on the measured value and the target value. The model of the second control unit may be learned using measurement data including the measured value and learning data including the operation amount of the controlled object device, and output an operation amount of the controlled object device recommended for increasing a reward value determined by a preset reward function based on the input of the measurement data.
[0010] The reward function may be a function in which the closer the measured value is to a target value, the higher the reward value is. The switching unit may switch based on a comparison result between a threshold value based on the target value and the measured value.
[0011] The switching unit may provide a threshold used for switching from control by the first control unit to control by the second control unit and a threshold used for switching from control by the second control unit to control by the first control unit with hysteresis characteristics.
[0012] The switching unit may cause the second control unit to perform control before a reference time has passed since the control of the control target device was started. The switching unit may cause the first control unit to perform control after a reference time has passed since the control of the control target device was started.
[0013] The first control unit may perform feedback control using at least one of proportional control, integral control, or differential control.
[0014] The first control unit can operate in an automatic mode and a manual mode. The automatic mode is a mode in which the operation amount of the control target device corresponding to the measured value is calculated and output according to the input of the measured value, and the manual mode is a mode in which the operation amount to be output is output according to the input of the operation amount. The second control unit can input the operation amount of the control target device to the first control unit. The switching unit can switch by switching the mode of the first control unit.
[0015] When switching from the manual mode to the automatic mode, the first control unit may control the operation amount before and after the switching without disturbance.
[0016] In a second aspect of the present invention, a control method is provided. The control method may include an acquisition phase, in which a measured value measured for a control target device is acquired. The control method may include a first control phase, in which an operation amount of a control target device corresponding to the measured value is output through at least one of feedback control and feedforward control. The control method may include a second control phase, in which an operation amount of a control target device corresponding to the measured value is output using a model learned using learning data. The control device may include a switching phase, in which a switch is performed as to which of the first control phase and the second control phase is used to control the control target device.
[0017] In a third aspect of the present invention, a recording medium having a program recorded thereon is provided. The program can cause a computer to function as an acquisition unit that acquires a measured value measured for a control target device. The program can cause a computer to function as a first control unit that outputs an operation amount of a control target device corresponding to the measured value through at least one of feedback control and feedforward control. The program can cause a computer to function as a second control unit that uses a model learned using learning data to output an operation amount of a control target device corresponding to the measured value. The program can cause a computer to function as a switching unit that switches which of the first control unit and the second control unit controls the control target device.
[0018] In addition, the above invention summary does not list all the necessary features of the present invention. In addition, sub-combinations of these feature groups may also constitute inventions. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 A system 1 according to this embodiment is shown.
[0020] Figure 2 The operation of the control device 4 according to the present embodiment in the learning phase is shown.
[0021] Figure 3 The operation of the control device 4 according to the present embodiment in the operation phase is shown.
[0022] Figure 4 An application example of the system 1 is shown.
[0023] Figure 5 An example of a computer 2200 is shown that can implement all or part of the various aspects of the present invention.
[0024] Description of symbols
[0025] 1 system, 2 facility, 4 control device, 20 equipment, 21 sensor, 40 measurement data acquisition unit, 41 operation amount acquisition unit, 42 reward value acquisition unit, 44 learning processing unit, 45 AI control unit, 46 feedback control unit, 47 switching unit, 49 control unit, 200 pipeline, 450 model, 2200 computer, 2201 DVD-ROM, 2210 host controller, 2212 CPU, 2214 RAM, 2216 graphics controller, 2218 display device, 2220 input and output controller, 2222 communication interface, 2224 hard disk drive, 2226 DVD-ROM drive, 2230 ROM, 2240 input and output chip, 2242 keyboard. DETAILED DESCRIPTION
[0026] Hereinafter, the present invention will be described by way of the embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all combinations of features described in the embodiments are essential to the solution means of the invention.
[0027] [1. Structure of System 1]
[0028] Figure 1 A system 1 according to this embodiment is shown. The system 1 includes facilities (equipment) 2 and a control device 4 .
[0029] [1-1. Facility 2]
[0030] The facility 2 is equipped with a plurality of devices (machines) 20. For example, the facility 2 may be a factory, or a composite device obtained by combining a plurality of devices 20. As a factory, in addition to chemical, biological and other industrial factories, there may be mentioned: a factory that manages and controls the wellheads of gas fields, oil fields and the like and their surroundings; a factory that manages and controls power generation by water, thermal power, nuclear power and the like; a factory that manages and controls environmental power generation by sunlight, wind power and the like; a factory that manages and controls water supply and drainage, dams and the like, etc. In the present embodiment, as an example, the facility 2 has one or more devices 20 and one or more sensors 21.
[0031] [1-1-1. Equipment 20]
[0032] Each device 20 is an instrument, machine or device, for example, it can be an actuator such as a valve, pump, heater, fan, motor, switch, etc. that controls at least one physical quantity such as pressure, temperature, pH, speed and flow in the process of facility 2.
[0033] In this embodiment, as an example, the facility 2 includes a plurality of devices 20. The devices 20 may be of different types from each other, or at least a part of two or more devices 20 may be of the same type.
[0034] Each device 20 can be controlled from the outside by wired or wireless means via a network not shown in the figure, or can be controlled manually. At least a part of the multiple devices 20 can be a control object device 20 (T) controlled by the control device 4. In the case where the system 1 has multiple control object devices 20 (T), the multiple control object devices 20 (T) can have a relationship of being controlled in a linked manner (for example, a master-slave relationship, a relationship that is not independently controlled). In addition, each control object device 20 (T) can be a device 20 of the same type or a device 20 of a different type.
[0035] In addition, a controller (not shown) may be provided in at least a part of the plurality of devices 20. The so-called provision of a controller in the device 20 may be a built-in controller in the device 20 or a controller connected to the outside of the device 20. The controller may perform feedback control on the device 20 according to the setting of the target value (set value) to reduce the difference between the target value and the current value. The target value of the controller provided in the control object device 20 (T) may be supplied from the control device 4, and in the present embodiment, as an example, may be the operation amount of the control object device 20 (T). Feedback control may be a control using at least one of proportional control (P control), integral control (I control) and differential control (D control).
[0036] [1-1-2. Sensor 21]
[0037] Each sensor 21 measures a physical quantity inside and outside the facility 2. Each sensor 21 can provide the control device 4 with measurement data obtained by the measurement.
[0038] In this embodiment, as an example, the facility 2 includes a plurality of sensors 21. The plurality of measurement data measured by the plurality of sensors 21 may include at least one of external environment data, feedback control data, operation state data, and consumption data.
[0039] The external environment data indicates a physical quantity that can act as interference to the control target device 20 (T). For example, the external environment data may indicate a physical quantity (or its change) that can act as interference to the control target device 20 (T). As an example, the external environment data may indicate the temperature and humidity of the air outside the facility 2, sunshine, wind direction, wind volume, precipitation, physical quantities that change due to the control of other devices 20, etc. The external environment data may be used to detect interference.
[0040] The feedback control data represents a physical quantity used to perform feedback control on each controlled device 20 (T). The feedback control data may represent a measured value measured for the controlled device 20 (T), for example, an output value of each controlled device 20 (T), or a value that changes according to the output value.
[0041] The operating state data indicates the operating state obtained as a result of controlling each control object device 20 (T). The operating state data may indicate a physical quantity that can be changed by controlling each control object device 20 (T), or may indicate an output value of each control object device 20 (T). The operating state data may be the same as the data for feedback control.
[0042] The consumption data indicates the consumption of at least one of energy and raw materials in the facility 2. The consumption data may indicate the consumption of electricity or fuel (for example, LPG (Liquefied Petroleum Gas)) as energy consumption.
[0043] [1-3. Control device 4]
[0044] The control device 4 controls each controlled device 20 (T). The control device 4 may be one or more computers, or may be composed of a PC, etc. The control device 4 includes a measurement data acquisition unit 40, an operation amount acquisition unit 41, a reward value acquisition unit 42, a learning processing unit 44, an AI control unit 45, a feedback control unit 46, a switching unit 47, and a control unit 49.
[0045] [1-3-1. Measurement Data Acquisition Unit 40]
[0046] The measurement data acquisition unit 40 is an example of an acquisition unit, and acquires measurement data measured by the sensor 21. The measurement data acquisition unit 40 may acquire measurement data measured by each of the plurality of sensors 21 provided in the facility 2. The measurement data may include a measurement value measured for each control target device 20 (T).
[0047] The measurement data acquisition unit 40 can acquire measurement data representing the average value of the measurement value within the control cycle in which the control device 4 controls each control object device 20 (T), or can acquire measurement data representing the measurement value of each control interval (i.e., the measurement value of the end timing of the control cycle). In the present embodiment, as an example, the control cycle of each control object device 20 (T) can be synchronized. The measurement data acquisition unit 40 can acquire the measurement data from the sensor 21, or can acquire the measurement data from the operator who has confirmed the sensor 21. The measurement data acquisition unit 40 can provide the acquired measurement data to the learning processing unit 44 and the AI control unit 45. In addition, the measurement data acquisition unit 40 can also provide the measurement value for each control object device 20 (T) (in the present embodiment, as an example, the output value of each control object device 20 (T) or the value that changes according to the output value) to the feedback control unit 46 and the switching unit 47.
[0048] [1-3-2. Operation Amount Acquisition Unit 41]
[0049] The operation amount acquisition unit 41 acquires the operation amount of each controlled device 20 (T). In the present embodiment, as an example, the operation amount acquisition unit 41 acquires the operation amount from the control unit 49, but it may also be acquired from the operator or from each controlled device 20 (T). The operation amount acquisition unit 41 may provide the acquired operation amount to the learning processing unit 44.
[0050] [1-3-3. Reward value acquisition unit 42]
[0051] The reward value acquisition unit 42 acquires a reward value for reinforcement learning in the learning processing unit 44. The reward value may be a value for evaluating the operating state of the facility 2, or a value determined by a preset reward function. Here, the so-called function is a mapping with a rule that makes each element of a certain set correspond to each element of another set one by one, and may be, for example, a mathematical expression or a table.
[0052] The reward function can output a reward value obtained by evaluating the state represented by the measurement data based on the input of the measurement data. The reward function can be a function in which the closer the measurement value measured for the control object device 20 (T) is to a target value, the higher the reward value. A target value can be a fixed value of a target value related to the measurement value measured for the control object device 20 (T), and similarly to the measurement value, it can represent the output value of each control object device 20 (T), or it can represent a value that changes according to the output value. As an example, when the control object device 20 (T) is a valve, and the target value (SV) and the measurement value (PV) represent the opening of the valve, the reward value R can be represented by the following reward function.
[0053] R=1.0-|SV-PV|*0.1
[0054] The reward function may be set by the operator. The reward value acquisition unit 42 may acquire the reward value from the operator who uses the reward function, or may input the measurement data from the sensor 21 into the reward function to acquire the reward value. When the reward value acquisition unit 42 inputs the measurement data into the reward function, the reward function may be stored inside the control device 4 or outside the control device 4.
[0055] [1-3-4. Learning Processing Unit 44]
[0056] The learning processing unit 44 performs a learning process of the model 450 provided in the AI control unit 45. The learning processing unit 44 performs a learning process of the model 450 using the measurement data acquired by the measurement data acquisition unit 40 and the learning data including the operation amount acquired by the operation amount acquisition unit 41. The learning processing unit 44 can perform a learning process of the model 450 using the reward value from the reward value acquisition unit 42.
[0057] [1-3-5. AI control unit 45]
[0058] The AI control unit 45 is an example of a second control unit, and uses a model 450 learned using learning data to output an operation amount of the control target device 20 (T) corresponding to the measured value for the control target device 20 (T). The AI control unit 45 can output the operation amount of each of the multiple control target devices 20 (T) corresponding to the measured value for each of the multiple control target devices 20 (T). The AI control unit 45 can input the operation amount to the feedback control unit 46.
[0059] Model 450 can output the recommended operation amount for increasing the reward value based on the input of the measurement data. The so-called operation amount for increasing the reward value can be an operation amount that makes the reward value higher than the base reward value when the reward value corresponding to the operating state of the facility 2 at a specified time point (for example, the current time) (for example, the reward value obtained by inputting the measurement data at the time point into the reward function) is used as the base reward value. The operation amount that increases the reward value as described above improves the operating state compared to the current time point, so it is recommended as a control for the control object device 20 (T). However, the base reward value can also be a fixed value (for example, a value obtained by subtracting the allowable value from the maximum value of the reward value).
[0060] In addition, in the present embodiment, as an example, a case where the AI control unit 45 has the model 450 built therein is described, but the model 450 may be stored in a server (eg, a cloud server) outside the control device 4 .
[0061] [1-3-6. Feedback control unit 46]
[0062] The feedback control unit 46 is an example of a first control unit, and outputs the operation amount of the control target device 20 (T) corresponding to the measured value for the control target device 20 (T) through feedback control. The feedback control unit 46 can output the operation amount of each of the multiple control target devices 20 (T) corresponding to the measured value for each of the multiple control target devices 20 (T). The feedback control unit 46 can operate in the automatic mode and the manual mode.
[0063] The automatic mode is a mode in which the operation amount of the control target device 20 (T) corresponding to the measured value is calculated and outputted according to the situation in which the measured value is input. In the automatic mode, the feedback control unit 46 can calculate the operation amount based on the measured value and the target value for feedback control. The feedback control unit 46 can also calculate the operation amount in a manner to reduce the difference between the target value and the current measured value according to the situation in which the target value is set by the operator or an external device. The target value set in the feedback control unit 46 can be a fixed value or can be changed appropriately.
[0064] The feedback control unit 46 can perform feedback control using at least one of proportional control (P control), integral control (I control), and differential control (D control), and in the present embodiment, performs PID control as an example.
[0065] The manual mode is a mode in which an operation amount to be output is output according to the input of the operation amount to be output. The operation amount input to the feedback control unit 46 may be supplied from the AI control unit 45 .
[0066] The feedback control unit 46 can provide the control unit 49 with the operation amount of the control target device 20 (T) in any mode.
[0067] [1-3-7. Switching unit 47]
[0068] The switching unit 47 switches which of the feedback control unit 46 and the AI control unit 45 controls the control target device 20 (T) (also referred to as control switching).
[0069] The switching unit 47 can switch the control by switching the mode of the feedback control unit 46. For example, the switching unit 47 can set the feedback control unit 46 to the automatic mode so that the feedback control unit 46 controls each control target device 20 (T). In addition, the switching unit 47 can also set the feedback control unit 46 to the manual mode so that the AI control unit 45 controls each control target device 20 (T).
[0070] [1-3-8. Control unit 49]
[0071] The control unit 49 controls each control target device 20 (T) using the supplied operation amount. The control unit 49 may supply the operation amount to each control target device 20 (T) to drive each control target device 20 (T) with the operation amount.
[0072] The control unit 49 can control each controlled device 20 (T) so that the output value of each controlled device 20 (T) is maintained within the control cycle. When the controlled device 20 (T) is feedback controlled, the control cycle may be longer than the cycle time of the feedback control.
[0073] In addition, the control unit 49 may further control each unit of the control device 4. For example, the control unit 49 may control the learning of the model 450.
[0074] According to the above system 1, since the feedback control unit 46 and the AI control unit 45 are switched to control the control target device 20 (T), when neither the feedback control unit 46 nor the AI control unit 45 can control the control target device 20 (T) well, the other one can control the control target device 20 (T) well. In addition, unlike the case where only the AI control unit 45 controls the control target device 20 (T), the feedback control unit 46 can share part of the control, so the learning of the model 450 can be simplified.
[0075] In addition, since the control switching is performed based on the difference between the measured value and the target value measured for the control object device 20 (T), when a large difference occurs during the rising period of the control object device 20 (T) or when a large difference occurs due to interference, etc., even if it takes time for the measured value to approach the target value in the feedback control unit 46, the measured value can be quickly approached to the target value through the AI control unit 45.
[0076] In addition, the measured data is input to the model 450 of the AI control unit 45, thereby outputting the operation amount recommended for increasing the reward value. Therefore, when the control is performed by the AI control unit 45, there is no need for a skilled operator to perform trial and error, and the controlled device 20 (T) can be controlled by an appropriate operation amount according to the situation.
[0077] In addition, the AI control unit 45 inputs the operation amount of the control target device 20 (T) to the feedback control unit 46, and the switching unit 47 switches the feedback control unit 46 between the automatic mode and the manual mode to switch the control. Therefore, the mode switching function of the feedback control unit 46 can be used to switch the control.
[0078] [2.Action]
[0079] [2-1. Learning stage]
[0080] Figure 2 The operation of the control device 4 according to the present embodiment in the learning phase is shown below. The control device 4 performs the process of steps S11 to S25 to learn the model 450 while operating the facility 2.
[0081] First, in step S11 , the measurement data acquisition unit 40 acquires measurement data measured by each sensor 21 . In this way, measurement data in an initial state is acquired. The measurement data acquisition unit 40 may store the measurement data in the learning processing unit 44 .
[0082] In step S13, the control unit 49 determines the operation amount of each control target device 20 (T). The control unit 49 can determine the operation amount in the next control cycle. In the present embodiment, as an example, the operation amount used when the next step S15 described later is performed can be determined. The determined operation amount can be an operation amount that increases the reward value, an operation amount that decreases the reward value, or an operation amount that is determined regardless of the reward value.
[0083] The control unit 49 may determine the operation amount in the next control cycle according to the operation of the operator, or may determine the operation amount output from the feedback control unit 46 to which the measured values for each controlled device 20 (T) are input as the operation amount in the next control cycle. Alternatively, the control unit 49 may determine the operation amount output from the model 450 as the operation amount in the next control cycle.
[0084] For example, when the process of step S13 is initially performed, the control unit 49 may determine the operation amount output from the model 450 as the operation amount in the next control cycle based on the input of the measurement data obtained in step S11 to the model 450. When the process of step S13 is performed multiple times by repeatedly performing the processes of steps S13 to S19, the control unit 49 may determine the operation amount output from the model 450 as the operation amount in the next control cycle based on the input of the measurement data obtained in the process of step S17 performed last to the model 450. When the process of step S13 is performed multiple times, different operation amounts may be determined between at least a part of the processes of multiple steps S13.
[0085] In step S15, the control unit 49 outputs the operation amount to each control target device 20 (T) to control each control target device 20 (T). The control unit 49 may store the operation amount in the learning processing unit 44 via the operation amount acquisition unit 41. The control unit 49 may also store the operation amount in the learning processing unit 44 in correspondence with the measurement data acquired by the measurement data acquisition unit 40 before controlling each control target device 20 (T). Thus, the learning data including the measurement data and the operation amount is stored in the learning processing unit 44.
[0086] In addition, when the process of step S15 is performed initially, the measurement data obtained before the control target device 20 (T) is controlled may be the measurement data obtained in the process of the above-mentioned step S11. In the case where the process of steps S13 to S19 is repeatedly performed and the process of step S15 is performed multiple times, the measurement data obtained before the control target device 20 (T) is controlled may be the measurement data obtained in the process of step S17 performed last.
[0087] In step S17, the measurement data acquisition unit 40 acquires the measurement data measured by each sensor 21. In this way, the measurement data when each control target device 20 (T) is controlled by the operation amount is acquired.
[0088] In step S19, the reward value acquisition unit 42 acquires a reward value determined by the reward function. Here, the measurement data acquired by the measurement data acquisition unit 40 may include the measurement data of the first group and the measurement data of the second group, respectively, and each group of measurement data may include at least one type of measurement data. The reward function may set the reward value to 0 regardless of the value of each measurement data of the second group when at least one of the measurement data of the first group does not satisfy the reference condition. In addition, the reward function may increase or decrease the reward value according to the value of each measurement data of the second group when the measurement data of the first group respectively satisfy the reference condition.
[0089] The measurement data of the first group may be operation status data, and the reference conditions of the measurement data of the first group may be conditions that should be achieved at the minimum in the facility 2. For example, in the case where the facility 2 is a manufacturing plant for products such as chemical products, the measurement data of the first group may represent the temperature and humidity in the plant, and the reference conditions of the measurement data may be the temperature range and humidity range that should be maintained in order to maintain the quality of the product. In addition, the measurement data of the second group may be consumption data. In this case, the more the consumption, the less the reward value. Thus, the learning process is performed in a manner of reducing the consumption.
[0090] The reward value acquisition unit 42 may store the acquired reward value in the learning processing unit 44. The reward value acquisition unit 42 may store the reward value in association with the learning data stored in the process of step S15 performed last.
[0091] In step S21, the control unit 49 determines whether the processing of steps S13 to S19 for the reference number of steps has been performed. When it is determined that the processing of the reference number of steps has not been performed (step S21: No), the processing is transferred to step S13. Thus, the learning data that is different in at least one of the measurement data or the operation amount is sampled by the reference number of steps and stored together with the reward value. In addition, when the processing of steps S13 to S19 is repeatedly performed, the cycle of step S13 (i.e., the control cycle) can be determined according to the time constant of the facility 2, and can be 5 minutes as an example. When it is determined in step S21 that the processing of the reference number of steps has been performed (step S21: Yes), the processing is transferred to step S23.
[0092] In step S23, the learning processing unit 44 performs learning processing of the model 450 using the corresponding stored learning data and reward value sets. Thus, the model 450 is updated. In addition, the learning processing unit 44 can perform learning processing based on known methods such as the steepest descent method, neural network, DQN (Deep Q-Network), Gaussian process, deep learning, etc. The learning processing unit 44 can perform learning processing of the model 450 so that the operation amount with a higher reward value is output as the recommended operation amount with higher priority.
[0093] In the model 450 after learning, weight coefficients may be stored corresponding to the learning data including the measurement data and the operation amount. The weight coefficients may be set according to the level of the reward value when the operation amount in the corresponding learning data is used for control, and may also be used to predict the reward value when the operation amount is used for control.
[0094] In step S25, the control unit 49 determines whether the processing of steps S13 to S23 has been performed for a reference number of iterations. If it is determined that the processing of the reference number of iterations has not been performed (step S25: No), the processing transfers to step S11. If it is determined that the processing of the reference number of iterations has been performed (step S25: Yes), the processing ends.
[0095] According to the above operation, when at least one of the measurement data of the first group does not satisfy the reference condition, the reward function sets the reward value to 0 regardless of the value of each of the measurement data of the second group, and when the measurement data of the first group respectively satisfy the reference condition, the reward value is increased or decreased according to the value of each of the measurement data of the second group. Therefore, the learning process of the model 450 can be performed so that the operation amount that increases the reward value is preferentially outputted under the premise that the measurement data of the first group satisfies the reference condition.
[0096] In addition, when the recommended operation amount output from the model 450 is determined as the operation amount in the next control cycle, each controlled device 20 (T) is controlled according to the recommended operation amount and measurement data corresponding to the control is obtained, so the learning process of the model 450 is performed using the learning data including the recommended operation amount and the reward value corresponding to the control result. Therefore, the learning process of the model 450 when the control is performed with the recommended operation amount can be sequentially performed to improve the learning accuracy.
[0097] [2-2. Operation phase]
[0098] Figure 3 The operation of the control device 4 according to the present embodiment in the operation phase is shown below: The control device 4 operates the facility 2 using the feedback control unit 46 and the AI control unit 45 by performing the processing of steps S31 to S37 .
[0099] In step S31, the measurement data acquisition unit 40 acquires the measurement data measured by each sensor 21. In this way, the measurement data in the initial state is acquired.
[0100] In step S33, the switching unit 47 determines which of the AI control unit 45 and the feedback control unit 46 controls the control object device 20 (T). When it is determined that the AI control unit 45 is to control (step S33: AI), the switching unit 47 can set the feedback control unit 46 to the manual mode. In this case, the control device 4 can transfer the processing to step S35. When it is determined that the feedback control unit 46 is to control (step S33: FB), the switching unit 47 can set the feedback control unit 46 to the automatic mode. In this case, the control device 4 can transfer the processing to step S37. The switching unit 47 can perform control switching (in the present embodiment, as an example, switching between the manual mode and the automatic mode of the feedback control unit 46) when it is determined that the control is to be performed by the feedback control unit 46 from the state where the AI control unit 45 is controlling, and when it is determined that the control is to be performed by the AI control unit 45 from the state where the feedback control unit 46 is controlling.
[0101] The switching unit 47 can switch the control according to the difference between the measured value and the target value for the control object device 20 (T). As an example, the switching unit 47 can switch the control from the control based on the feedback control unit 46 to the control based on the AI control unit 45 according to the case where the difference between the measured value and the target value is larger than the reference value, and switch the control from the control based on the AI control unit 45 to the control based on the feedback control unit 46 according to the case where the difference between the measured value and the target value is smaller than the reference value. The switching unit 47 can make the reference value when switching to the control based on the AI control unit 45 and the reference value when switching to the control based on the feedback control unit 46 have a hysteresis characteristic, or make the latter reference value smaller than the former reference value. The switching unit 47 can obtain the target value from the feedback control unit 46.
[0102] In addition, the switching unit 47 can switch the control from the control by the feedback control unit 46 to the control by the AI control unit 45 when the difference between the measured value and the target value of the control target device 20 (T) is larger than the reference value multiple times within the reference time window, that is, when the difference changes from a value below the reference value to a value larger than the reference value multiple times. As an example, the switching unit 47 can switch the control to the control by the AI control unit 45 when the measured value oscillates due to interference or the like. Any time width can be used as the reference time window, and any value can be used as the reference value.
[0103] In this case, the switching unit 47 may switch from the control by the AI control unit 45 to the control by the feedback control unit 46, based on the fact that the difference between the measured value and the target value is maintained smaller than the reference value within the reference time window. The switching unit 47 may make the reference value when switching to the control by the AI control unit 45 and the reference value when switching to the control by the feedback control unit 46 have hysteresis characteristics, or may make the latter reference value smaller than the former reference value.
[0104] In addition, the switching unit 47 may switch the control based on the comparison result between the threshold value and the measured value for the control target device 20 (T). The threshold value may be set based on a target value for the control target device 20 (T) included in the reward function. For example, the threshold value may be a value obtained by performing arithmetic operations such as four arithmetic operations on a target value, or may be a target value itself.
[0105] When the control is switched based on the comparison result between the threshold value and the measured value, the switching unit 47 may determine that the AI control unit 45 controls the control target device 20 (T) when the measured value is below the threshold value. In addition, the switching unit 47 may determine that the feedback control unit 46 controls the control target device 20 (T) when the measured value is greater than the threshold value. As an example, when the control target device 20 (T) is a valve and a target value indicating the opening is 30%, the threshold value may be set to 30% of the target value itself, and when the measured value is less than 30%, the AI control unit 45 may control the valve as the control target device 20 (T), and when the measured value is greater than 30%, the feedback control unit 46 may control the valve as the control target device 20 (T).
[0106] The switching unit 47 may provide a threshold value when switching to the control by the AI control unit 45 and a threshold value when switching to the control by the feedback control unit 46 with hysteresis characteristics, or may make the latter threshold value larger than the former threshold value.
[0107] In step S35, the control device 4 controls the control object device 20 (T) by the AI control unit 45. For example, the model 450 of the AI control unit 45 can output the operation amount recommended according to the situation that the measurement data is provided from the measurement data acquisition unit 40 to the control unit 49 via the feedback control unit 46. The control unit 49 can provide the input operation amount to the control object device 20 (T). As a result, the control object device 20 (T) is driven by the operation amount. After the processing of step S35 is completed, the control device 4 can transfer the processing to step S31.
[0108] In addition, in step S35, the model 450 may also calculate the reward value (also referred to as the predicted reward value) predicted when the operation amount is used for control for each operation amount included in the learning data. For example, the model 450 may extract each learning data containing one operation amount from a plurality of learning data. The model 450 may use the result obtained by weighted addition of each weight coefficient corresponding to each extracted learning data according to the distance between the measured data representing the state at the current time point (in this embodiment, as an example, the measured data obtained in the processing of the last step S31) and the measured data in the learning data as the predicted reward value for the one operation amount. The model 450 may set the weighted size such that the greater the distance between the measured data, the smaller the weight (i.e., the smaller the impact on the reward value). In the model 450, the higher the predicted reward value of the operation amount, the higher the priority is as the recommended operation amount. However, the model 450 may not necessarily use the operation amount with the highest predicted reward value as the recommended operation amount.
[0109] In step S37, the control device 4 controls the control object device 20 (T) by the feedback control unit 46. For example, the feedback control unit 46 may output the operation amount corresponding to the measured value to the control unit 49 according to the situation that the measured value for the control object device 20 (T) is input. The control unit 49 may provide the input operation amount to the control object device 20 (T). Thus, the control object device 20 (T) is driven by the operation amount. After the processing of step S37 is completed, the control device 4 may transfer the processing to step S31.
[0110] In step S37, when the feedback control unit 46 switches from the manual mode to the automatic mode, it controls the operation amount before and after the switching without disturbance (bump less), that is, it suppresses the sharp change of the operation amount before and after the switching. For example, the feedback control unit 46 can use the integral term that is inversely calculated from the operation amount output in the manual mode (that is, the operation amount provided by the AI control unit 45) to calculate the next operation amount. As an example, when the feedback control unit 46 performs PID control in the automatic mode, the operation amount MV can be calculated according to the following equations (1) and (2). When the feedback control unit 46 switches from the manual mode to the automatic mode, the integral term of the second term on the right side of equation (2) can be inversely calculated according to the operation amount output in the manual mode to calculate the next operation amount MV.
[0111]
Mathematical formula 1
[0112] Δv i =PV i -SV i (1)
[0113]
[0114] Here, in the formula, the subscripts i and i-1 are variables indicating control timing. PV is the measured value of the controlled device 20 (T), in other words, process data. SV is the target value, in other words, the set value. P, I, and D are proportional gain, integral gain, and differential gain.
[0115] According to the above operation, when the difference between the measured value and the target value becomes larger than the reference value multiple times within the reference time window, the control is switched from the control by the feedback control unit 46 to the control by the AI control unit 45. Therefore, when oscillation occurs due to the control by the feedback control unit 46, the measured value can be brought close to the target value while suppressing the oscillation.
[0116] In addition, in the reward function used by the AI control unit 45, the closer the measured value is to a target value, the higher the reward value is, and control switching is performed based on the comparison result between the threshold value based on the target value and the measured value. Therefore, when the measured value is included in the range that cannot be well controlled by the AI control unit 45, it can be well controlled by the feedback control unit 46.
[0117] In addition, since the threshold value when switching to control by the AI control unit 45 and the threshold value when switching to control by the feedback control unit 46 have hysteresis characteristics, it is possible to prevent the control subject from being frequently switched due to changes in the measured value, causing the operation amount to become unstable.
[0118] In addition, when the feedback control unit 46 switches from the manual mode to the automatic mode, the operation amount before and after the switching is controlled without disturbance, so that the discontinuity between the operation amount output from the feedback control unit 46 in the manual mode and the operation amount newly calculated by the feedback control unit 46 in the automatic mode can be suppressed, thereby suppressing the change.
[0119] [3. Application examples]
[0120] Figure 4 An example of application of the system 1 is shown. Figure 4 In FIG. 1 , the structure of the control device 4 is simplified.
[0121] In this application example, the facility 2 is an air conditioner for a factory, which introduces external air into the duct 200 and supplies the temperature- and humidity-controlled air to rooms in the factory or other air conditioners.
[0122] Valves B1 to B4 are provided as control target devices 20 (T) in facility 2. Valve B1 adjusts the heating amount in pipe 200, valve B2 adjusts the cooling amount in pipe 200, valve B3 adjusts the humidification amount in pipe 200, and valve B4 adjusts the dehumidification amount in pipe 200.
[0123] In addition, the facility 2 is provided with humidity sensors 21a, 21b, temperature sensors 21c, 21d, opening sensor 21e, sunshine sensor 21f, wind direction sensor 21g, air volume sensor 21h, power usage sensor 21i, LPG usage sensor 21j, etc. as sensors 21. The humidity sensor 21a and the temperature sensor 21c measure the humidity and temperature of the outside air introduced into the duct 200. The humidity sensor 21b and the temperature sensor 21d measure the humidity and temperature of the regulated air discharged from the duct 200. The opening sensor 21e measures the opening (output value) of the valves B1 to B4, respectively. The sunshine sensor 21f, the wind direction sensor 21g, and the air volume sensor 21h measure the sunshine amount, wind direction, and air volume outside the factory where the facility 2 is installed. The power usage sensor 21i measures the power usage of the facility 2. The LPG usage sensor 21j measures the LPG usage of the facility 2.
[0124] The learning processing unit 44 of the control device 4 uses the measurement data measured by these sensors 21a to 21j and the learning data of the operation amount of each valve B1 to B4 to perform the learning processing of the model 450 in the AI control unit 45. In this application example, as an example, the operation amount is related to the output value of the valve B1 to B4, that is, the opening. When the operation amount related to the opening is sent from the control device 4 through an electrical signal, etc., the valves B1 to B4 are opened and closed according to the operation amount. In the case where at least one of the temperature and humidity of the adjusted air is not maintained within the reference range, the reward value for the learning process can be set to 0. In the case where the temperature and humidity of the adjusted air are maintained within the reference range, the smaller the amount of electricity used and the amount of LPG used, the higher the reward value for the learning process can be.
[0125] The AI control unit 45 calculates the operation amount recommended for increasing the reward value based on the input of the measurement data measured by the sensors 21 a to 21 j.
[0126] The feedback control unit 46 calculates the operation amount based on the opening degree measured by the opening degree sensor 21 e and the target value of the opening degree.
[0127] The switching unit 47 performs control switching according to the difference between the opening measured by the opening sensor 21e and the target value of the opening. The switching unit 47 switches the feedback control unit 46 between the manual mode and the automatic mode, thereby providing the operation amount calculated by the AI control unit 45 and the operation amount calculated by the feedback control unit 46 from the feedback control unit 46 to the control unit 49.
[0128] The control unit 49 supplies the operation amounts to the valves B1 to B4 , thereby opening and closing the valves B1 to B4 according to the operation amounts.
[0129] [5. Modifications]
[0130] In addition, in the above embodiment, the system 1 is described as having a single control device 4, but it may also be provided with a plurality of control devices 4. In this case, the control target devices 20 (T) may be the same or different between the control devices 4. As an example, in the system 1, a control device 4 may be provided for each device 20 as the control target device 20 (T).
[0131] In addition, the control device 4 is described as having the operation amount acquisition unit 41, the reward value acquisition unit 42, the learning processing unit 44, and the control unit 49, but it may not have at least one of them. In the case where the control device 4 does not have the learning processing unit 44 or the operation amount acquisition unit 41, the control device 4 may not perform the learning process of the model 450, but use the model 450 after the learning process to control the control object device 20 (T).
[0132] Furthermore, although the case where the measurement data acquisition unit 40 acquires the measurement data measured by each of the plurality of sensors 21 has been described, it is also possible to acquire only the measurement value for the control target device 20 (T).
[0133] In addition, the case where the switching unit 47 switches the control according to the difference between the measured value and the target value of the control target device 20 (T) has been described, but the control switching may also be performed according to the time elapsed from the start of the control of the control target device 20 (T). For example, the switching unit 47 may perform the control by the AI control unit 45 before the reference time elapses from the start of the control of the control target device 20 (T), and perform the control by the feedback control unit 46 after the reference time elapses. Thus, it is possible to prevent overshoot or undershoot during the rising period of the control target device 20 (T) and make the measured value approach the target value quickly. In addition, since the control by the feedback control unit 46 is performed after the reference time elapses from the start of the control of the control target device 20 (T), the measured value can be stably controlled. The timing of starting the control of the control target device 20 (T) may be the timing of starting the control by the control device 4 when the control target device 20 (T) and the control device 4 are started, or it may be the timing of starting the control by the control device 4 by changing the target value set in the control device 4 once the control is started. As an example, the reference time may be a period from the start of control to the convergence of overshoot or undershoot when the control target device 20 (T) is controlled by the feedback control unit 46 .
[0134] In addition, the control device 4 has been described as having the feedback control unit 46 that outputs the operation amount corresponding to the measured value through feedback control, but it may also have a control unit that outputs the operation amount corresponding to the measured value through feedforward control based on the feedback control or instead of the feedback control.
[0135] In addition, the control device 4 has been described as having a single feedback control unit 46, but it may also have a plurality of feedback control units 46. The plurality of feedback control units 46 may be connected in multiple stages in order to perform cascade control in which feedback control is multiplexed. The measured value for the same control target device 20 (T) may be input to the feedback control unit 46 of each stage, and the operation amount output from the feedback control unit 46 of the previous stage may be input to the feedback control unit 46 of the next stage as a target value. In this case, the AI control unit 45 may provide the operation amount to any one of the feedback control units 46, and the feedback control unit 46 may switch the mode between the manual mode and the automatic mode through the switching unit 47. In addition, when the feedback control unit 46 is connected in multiple stages, the control device 4 may have a plurality of AI control units 45. These plurality of AI control units 45 may provide the operation amount to each feedback control unit 46, and these feedback control units 46 may switch the mode between the manual mode and the automatic mode through the switching unit 47. The models 450 of the plurality of AI control units 45 may implement the same learning process or different learning processes.
[0136] In addition, the case where the switching unit 47 switches the mode of the feedback control unit 46 is described, but the control switching can also be performed by other methods. For example, the AI control unit 45 and the feedback control unit 46 can provide the calculated operation amount to the switching unit 47 respectively, and the switching unit 47 can switch the supply source of the operation amount output to the control unit 49 between the AI control unit 45 and the feedback control unit 46 to perform the switching control.
[0137] In addition, various embodiments of the present invention may be described with reference to flow charts and block diagrams, wherein a block may represent (1) a stage of a process of performing an operation, or (2) a portion of a device having the function of performing an operation. Specific stages and portions may be implemented by at least one of a dedicated circuit, a programmable circuit provided together with computer-readable instructions stored on a computer-readable medium, and a processor provided together with computer-readable instructions stored on a computer-readable medium. The dedicated circuit may include a hardware circuit that is at least one of digital and analog, and may also include at least one of an integrated circuit (IC) and a discrete circuit. The programmable circuit may include a reconfigurable hardware circuit that includes logical AND, logical OR, logical XOR, logical NAND, logical NOR and other logical operations, memory elements such as flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc.
[0138] A computer-readable medium may include any tangible device capable of storing instructions executed by an appropriate device, with the result that a computer-readable medium having instructions stored therein has a product containing instructions that can be executed in order to create a unit for performing the operations specified in the flowchart or block diagram. As an example of a computer-readable medium, it may include an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, etc. As a more specific example of a computer-readable medium, it may include a floppy disk (registered trademark), a magnetic disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an electrically erasable programmable read-only memory (EEPROM), a static random access memory (SRAM), a compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a Blu-ray (RTM) disk, a memory stick, an integrated circuit card, etc.
[0139] Computer readable instructions may include assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, and any of codes or object codes written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, JAVA (registered trademark), C++, and existing procedural programming languages such as the "C" programming language or similar programming languages.
[0140] The computer readable instructions can be provided to a processor or programmable circuit of a general-purpose computer, a special-purpose computer or other programmable data processing device via a local or local area network (LAN), a wide area network (WAN) such as the Internet, etc., to create a unit for performing the operations specified in the flowchart or block diagram, and execute the computer readable instructions. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.
[0141] Figure 5 An example of a computer 2200 that can implement multiple aspects of the present invention in whole or in part is shown. The program installed in the computer 2200 can cause the computer 2200 to function as an operation associated with an apparatus involved in an embodiment of the present invention or one or more parts of the apparatus, or can cause the computer 2200 to perform the operation or the one or more parts, and in addition or instead, can cause the computer 200 to perform a process involved in an embodiment of the present invention or a stage of the process. Such a program can be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described in this specification.
[0142] The computer 2200 according to the present embodiment includes a CPU 2212, a RAM 2214, a graphic controller 2216, and a display device 2218, which are connected to each other through a host controller 2210. The computer 2200 also includes input-output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input-output controller 2220. The computer also includes conventional input-output units such as a ROM 2230 and a keyboard 2242, which are connected to the input-output controller 2220 via an input-output chip 2240.
[0143] The CPU 2212 controls each unit by operating according to the program stored in the ROM 2230 and the RAM 2214. The graphic controller 2216 obtains image data generated by the CPU in a frame buffer or the like provided in the RAM 2214 or in the graphic controller 2216 itself, and displays the image data on the display device 2218.
[0144] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from the IC card and, based on or instead of the programs and data, writes the programs and data to the IC card.
[0145] The ROM 2230 stores therein at least one of a boot program or the like executed by the computer 2200 at startup and a program dependent on the hardware of the computer 2200. The input-output chip 2240 can also connect various input-output units to the input-output controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, and the like.
[0146] The program is provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The program is read from the computer-readable medium, installed in a hard disk drive 2224, a RAM 2214, or a ROM 2230, which is also an example of a computer-readable medium, and executed by a CPU 2212. Information processing written in these programs is read by the computer 2200, and cooperation between the program and the various types of hardware resources described above is provided. The device or method can be constituted by implementing the operation or processing of information according to the use of the computer 2200.
[0147] For example, in the case where communication is performed between the computer 2200 and an external device, the CPU 2212 can execute a communication program loaded into the RAM 2214, and based on the processing written in the communication program, instruct the communication interface 2222 to perform communication processing. Under the control of the CPU 2212, the communication interface 2222 reads the transmission data stored in the transmission buffer processing area provided in the recording medium such as the RAM 2214, the hard disk drive 2224, the DVD-ROM 2201, or the IC card, and transmits the read transmission data to the network, or writes the reception data received from the network to the reception buffer processing area provided on the recording medium, etc.
[0148] In addition, the CPU 2212 may also read all or a necessary part of a file or database stored in an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), or an IC card into the RAM 2214, and perform various types of processing on the data on the RAM 2214. Then, the CPU 2212 writes the processed data back to the external recording medium.
[0149] Various types of information, such as various types of programs, data, tables, and databases, can be stored in a recording medium and receive information processing. CPU2212 can perform various types of processing on the data read from RAM2214 and write the results back to RAM2214, and the processing includes various types of operations, information processing, conditional judgments, conditional branches, unconditional branches, information retrieval, replacement, etc. specified by the instruction sequence of the program recorded in various places of this disclosure. In addition, CPU2212 can also retrieve information from files, databases, etc. in the recording medium. For example, in the case where multiple entries with attribute values of a first attribute respectively associated with attribute values of a second attribute are stored in a recording medium, CPU2212 can retrieve an entry that specifies an attribute value of the first attribute and is consistent with the condition from the multiple entries, and read the attribute value of the second attribute stored in the entry, thereby obtaining the attribute value of the second attribute associated with the first attribute that meets the predetermined condition.
[0150] The programs or software modules described above may be stored in a computer-readable medium on or near the computer 2200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet may also be used as a computer-readable medium, thereby providing the program to the computer 2200 via the network.
[0151] The present invention has been described above using the embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various changes or improvements can be made to the above embodiments. It can be clearly stated in the claims that the methods obtained by making such changes or improvements can also be included in the technical scope of the present invention.
[0152] It should be noted that the execution order of each process such as actions, processes, steps and stages in the device, system, program and method shown in the claims, specifications and drawings can be implemented in any order unless it is specifically stated as "before", "first", etc., and as long as the output of the previous process is not used for the subsequent process. Even if the action flow in the claims, specifications and drawings is described using "first", "next", etc. for convenience, it does not mean that it must be implemented in this order.
Claims
1. A control device comprising: an acquisition unit that acquires a measurement value measured for the control target device; a first control unit that outputs an operation amount of the control target device corresponding to the measured value through at least one of feedback control and feedforward control; a second control unit that outputs an operation amount of the control target device corresponding to the measured value using a model learned using the learning data; as well as a switching unit for switching which of the first control unit and the second control unit controls the control target device, The switching unit switches from the control by the first control unit to the control by the second control unit when the difference between the measured value and the target value is larger than a reference value.
2. The control device according to claim 1, wherein: The switching unit switches from control by the first control unit to control by the second control unit when the difference between the measured value and the target value becomes larger than a reference value a plurality of times within a reference time window.
3. A control device comprising: an acquisition unit that acquires a measurement value measured for the control target device; a first control unit that outputs an operation amount of the control target device corresponding to the measured value through at least one of feedback control and feedforward control; a second control unit that outputs an operation amount of the control target device corresponding to the measured value using a model learned using the learning data; as well as a switching unit for switching which of the first control unit and the second control unit controls the control target device, The switching unit switches from control by the second control unit to control by the first control unit when a difference between the measured value and the target value is smaller than a reference value.
4. The control device according to any one of claims 1 to 3, wherein: The first control unit outputs an operation amount calculated based on the measured value and the target value, The model of the second control unit is learned using measurement data including the measurement value and learning data including the operation amount of the control object device, and outputs the operation amount of the control object device recommended for improving the reward value determined by a predetermined reward function based on the input of the measurement data.
5. The control device according to claim 4, wherein: The reward function is a function in which the closer the measured value is to a target value, the higher the reward value is. The switching unit performs the switching based on a comparison result between a threshold value based on the one target value and the measured value.
6. The control device according to claim 5, wherein: The switching unit provides hysteresis characteristics to a threshold used for switching from control by the first control unit to control by the second control unit and a threshold used for switching from control by the second control unit to control by the first control unit.
7. A control device comprising: an acquisition unit that acquires a measurement value measured for the control target device; a first control unit that outputs an operation amount of the control target device corresponding to the measured value through at least one of feedback control and feedforward control; a second control unit that outputs an operation amount of the control target device corresponding to the measured value using a model learned using the learning data; as well as a switching unit for switching which of the first control unit and the second control unit controls the control target device, The switching unit causes the second control unit to perform control before a reference time elapses from the start of control of the control target device, and causes the first control unit to perform control after the reference time elapses from the start of control of the control target device.
8. The control device according to claim 7, wherein: The first control unit outputs an operation amount calculated based on the measured value and the target value, The model of the second control unit is learned using measurement data including the measurement value and learning data including the operation amount of the control object device, and outputs the operation amount of the control object device recommended for improving the reward value determined by a predetermined reward function based on the input of the measurement data.
9. The control device according to any one of claims 1 to 3, 7 to 8, wherein: The first control unit performs feedback control using at least one of proportional control, integral control, and differential control.
10. The control device according to any one of claims 1 to 3, 7 to 8, wherein: The first control unit can operate in an automatic mode and a manual mode. The automatic mode is a mode in which an operation amount of the controlled device corresponding to the measured value is calculated and output according to the input of the measured value, and the manual mode is a mode in which an operation amount to be output is output according to the input of the measured value. The second control unit inputs the operation amount of the control target device to the first control unit, The switching unit performs the switching by switching a mode of the first control unit.
11. The control device according to claim 10, wherein: The first control unit controls the operation amount before and after the switching without disturbance when switching from the manual mode to the automatic mode.
12. A control method, comprising: In the acquisition phase, the measured value measured for the controlled device is acquired; In a first control stage, an operation amount of the control target device corresponding to the measured value is outputted through at least one of feedback control and feedforward control; In a second control stage, a model obtained by learning using the learning data is used to output an operation amount of the controlled device corresponding to the measured value; as well as In a switching phase, a switch is performed to determine whether the first control phase or the second control phase is used to control the control target device. The switching phase includes a phase in which the switching from the control based on the first control phase to the control based on the second control phase is performed in response to a difference between the measured value and the target value being larger than a reference value.
13. A control method, comprising: In the acquisition phase, the measured value measured for the controlled device is acquired; In a first control stage, an operation amount of the control target device corresponding to the measured value is outputted through at least one of feedback control and feedforward control; In a second control stage, a model obtained by learning using the learning data is used to output an operation amount of the controlled device corresponding to the measured value; as well as In a switching phase, a switch is performed to determine whether the first control phase or the second control phase is used to control the control target device. The switching phase includes a phase of switching from control based on the second control phase to control based on the first control phase when the difference between the measured value and the target value is smaller than a reference value.
14. A control method, comprising: In the acquisition phase, the measured value measured for the controlled device is acquired; In a first control stage, an operation amount of the control target device corresponding to the measured value is outputted through at least one of feedback control and feedforward control; In a second control stage, a model obtained by learning using the learning data is used to output an operation amount of the controlled device corresponding to the measured value; as well as In a switching phase, a switch is performed to determine whether the first control phase or the second control phase is used to control the control target device. The switching phase performs control in the second control phase before a reference time elapses from the start of control of the control target device, and performs control in the first control phase after the reference time elapses from the start of control of the control target device.
15. A computer program product, comprising a computer program, wherein the computer program causes a computer to function as: an acquisition unit that acquires a measurement value measured for the control target device; a first control unit that outputs an operation amount of the control target device corresponding to the measured value through at least one of feedback control and feedforward control; a second control unit that outputs an operation amount of the control target device corresponding to the measured value using a model learned using the learning data; as well as a switching unit for switching which of the first control unit and the second control unit controls the control target device, The switching unit switches from the control by the first control unit to the control by the second control unit when the difference between the measured value and the target value is larger than a reference value.
16. A computer program product, comprising a computer program, wherein the computer program causes a computer to function as: an acquisition unit that acquires a measurement value measured for the control target device; a first control unit that outputs an operation amount of the control target device corresponding to the measured value through at least one of feedback control and feedforward control; a second control unit that outputs an operation amount of the control target device corresponding to the measured value using a model learned using the learning data; as well as a switching unit for switching which of the first control unit and the second control unit controls the control target device, The switching unit switches from control by the second control unit to control by the first control unit when a difference between the measured value and the target value is smaller than a reference value.
17. A computer program product, comprising a computer program, wherein the computer program causes a computer to function as: an acquisition unit that acquires a measurement value measured for the control target device; a first control unit that outputs an operation amount of the control target device corresponding to the measured value through at least one of feedback control and feedforward control; a second control unit that outputs an operation amount of the control target device corresponding to the measured value using a model learned using the learning data; as well as a switching unit for switching which of the first control unit and the second control unit controls the control target device, The switching unit causes the second control unit to perform control before a reference time elapses from the start of control of the control target device, and causes the first control unit to perform control after the reference time elapses from the start of control of the control target device.
