Remote control device, local control device, learning processing device, method, and program

Abstract

To provide a remote control device, a local control device, a learning processing device, a method, and a program.SOLUTION: Provided is a remote control device including: a measurement value receiving unit that receives a measurement value related to a facility, from a local control device that controls the facility; a calculation unit that uses a model for calculating a control value to be used for control of the facility in a case of occurrence of control delay including delay of communication with the local control device, from an amount of delay corresponding to the control delay and the measurement value, thereby calculating a control value corresponding to the received measurement value and the amount of delay; and a control value transmission unit that transmits the calculated control value to the local control device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a remote control device, a local control device, a learning processing device, a method, and a program. [Background technology]

[0002] Patent Document 1 states that "In facilities such as plants, a distributed control system (DCS) is constructed in which sensors and operating devices are connected to the control devices that control them via communication means, and a high level of automated operation is realized by the DCS." [Prior art document] [Patent documents] Patent Document 1: JP 2020-027556 A Summary of the Invention

[0003] A first aspect of the present invention provides a remote control device comprising: a measurement value receiving unit that receives measurement values related to equipment from a local control device that controls the equipment, a calculation unit that calculates a control value to be used for controlling the equipment when a control delay including a communication delay occurs between the local control device and the remote control device, using a model that calculates a control value corresponding to the received measurement values and a delay amount corresponding to the control delay, from the measurement values and a delay amount corresponding to the control delay, and a control value transmitting unit that transmits the calculated control value to the local control device.

[0004] In the remote control device, the calculation unit may use as the delay amount the amount by which a candidate value for the period from when the local control device transmits the measurement value to when the local control device receives the control value corresponding to the measurement value from the control value transmission unit exceeds a predetermined margin time.

[0005] In any of the remote control devices, the calculation unit may use the model to calculate a plurality of control values corresponding to each of the plurality of delay amounts from the received measurement values, and the control value transmission unit may transmit the plurality of control values to the local control device.

[0006] In the remote control device, the control value transmitting section may transmit data indicating a correspondence relationship between the plurality of control values and the plurality of delay amounts to the local control device.

[0007] In any of the remote control devices, the measurement value receiving unit may receive new measurement values obtained in response to controlling the equipment using the calculated control values and control delays corresponding to the control values used, and any of the remote control devices may be provided with a learning processing unit that updates the model using the received control delays and the new measurement values.

[0008] A second aspect of the present invention provides a local control device comprising: a measurement value transmitting unit that transmits measurement values related to equipment to a remote control device that calculates control values corresponding to the measurement values, a control value receiving unit that receives a plurality of control values corresponding to the transmitted measurement values from the remote control device, a selection unit that selects a control value to be used for controlling the equipment from the plurality of control values according to a control delay including a communication delay between the local control device and the remote control device, and a control unit that controls the equipment according to the selected control value.

[0009] The local control device may include a delay measurement unit that measures the period from when the measurement value transmission unit transmits the measurement value until the control value receiving unit receives the plurality of control values corresponding to the measurement value, and determines the control delay, and the selection unit may select a control value to be used for controlling the equipment from among the plurality of control values in accordance with the control delay determined by the delay measurement unit.

[0010] In any of the local control devices, the delay measurement unit may determine the control delay as the period from when the measurement value transmission unit transmits the measurement value until the control value receiving unit receives the plurality of control values corresponding to the measurement value, which period exceeds a predetermined margin time, and the control unit may control the equipment after at least the predetermined margin time has passed since the measurement value transmission unit transmitted the measurement value.

[0011] A third aspect of the present invention provides a learning processing device, comprising: a measurement value receiving unit that receives a control delay, including a communication delay between a remote control device and a local control device, and measurement values related to the equipment, the control delay being acquired by a local control device that controls the equipment in accordance with a control value received from the remote control device; and a learning processing unit that generates a model that calculates a control value to be used for controlling the equipment from a delay amount corresponding to the control delay and the measurement values.

[0012] A fourth aspect of the present invention provides a method, comprising: a measurement value receiving step of receiving measurement values related to equipment from a local control device that controls the equipment, a calculation step of calculating a control value to be used for controlling the equipment when a control delay including a communication delay occurs between the local control device and the equipment, using a model that calculates a control value corresponding to the received measurement values and a delay amount corresponding to the control delay from the measurement values, and a control value transmitting step of transmitting the calculated control value to the local control device.

[0013] A fifth aspect of the present invention provides a program that causes a computer to function as a measurement value receiving unit that receives measurement values related to equipment from a local control device that controls the equipment, a calculation unit that calculates a control value to be used for controlling the equipment in the event of a control delay including a communication delay between the local control device and the equipment, using a model that calculates a control value based on the received measurement values and a delay amount corresponding to the control delay, from the measurement values, and an amount of delay corresponding to the control delay, and a control value transmitting unit that transmits the calculated control value to the local control device.

[0014] In a sixth aspect of the present invention, there is provided a method, the method comprising: a measurement value transmitting step of transmitting a measurement value relating to a facility to a remote control device that calculates a control value in response to the measurement value; The method includes a control value receiving step of receiving, from the remote control device, a plurality of control values corresponding to the transmitted measurement values; a selection step of selecting, from the plurality of control values, a control value to be used for controlling the equipment according to a control delay including a communication delay with the remote control device; and a control step of controlling the equipment according to the selected control value.

[0015] A seventh aspect of the present invention provides a program that causes a computer to function as a measurement value transmitting unit that transmits measurement values related to equipment to a remote control device that calculates a control value corresponding to the measurement values, a control value receiving unit that receives a plurality of control values corresponding to the transmitted measurement values from the remote control device, a selection unit that selects a control value to be used for controlling the equipment from the plurality of control values according to a control delay including a communication delay between the remote control device and the computer, and a control unit that controls the equipment according to the selected control value.

[0016] An eighth aspect of the present invention provides a method, comprising: a measurement value receiving step of receiving, in a local control device that controls equipment in accordance with a control value received from a remote control device, a control delay including a communication delay between the remote control device and the local control device and measurement values related to the equipment, the control delay being acquired by the local control device; and a learning processing step of generating a model that calculates a control value to be used for controlling the equipment from a delay amount corresponding to the control delay and the measurement values.

[0017] A ninth aspect of the present invention provides a program that causes a computer to function as a measurement value receiving unit that receives a control delay, including a communication delay between a remote control device and a local control device, and measurement values related to the equipment, the control delay being acquired by a local control device that controls the equipment in accordance with a control value received from the remote control device, and a learning processing unit that generates a model that calculates a control value to be used for controlling the equipment from a delay amount corresponding to the control delay and the measurement values.

[0018] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0019] [Figure 1] 1 shows an example of a block diagram of a control system 10 according to the present embodiment. [Figure 2] FIG. 2 is an explanatory diagram of communication in the control system 10. [Figure 3] 2 shows an example of a processing flow of the remote control device 100 of this embodiment. [Figure 4] 10 shows an example of a control operation determination table. [Figure 5] An example of a weight table for Model 135 is shown below. [Figure 6] 10 shows an example of a table showing the correspondence between a plurality of control values and a plurality of delay amounts. [Figure 7] 10 shows an example of a processing flow of the local control device 200 of this embodiment. [Figure 8] An example of a flow of learning by the remote control device 100 in the control system 10 according to this embodiment will be shown. [Figure 9] 22 illustrates an example computer 2200 in which aspects of the present invention may be embodied, in whole or in part. DETAILED DESCRIPTION OF THE INVENTION

[0020] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0021] FIG. 1 shows an example of a block diagram of a control system 10 that may include a remote control device 100 and a local control device 200 according to this embodiment. Note that these blocks are functionally separated functional blocks and may not necessarily correspond to the actual device configuration. In other words, just because something is shown as a single block in this diagram does not necessarily mean that it is composed of a single device. Also, just because something is shown as separate blocks in this diagram does not necessarily mean that it is composed of separate devices. The same applies to the subsequent block diagrams.

[0022] As an example, the control system 10 performs maintenance management of a plant and includes a remote control device 100, a local control device 200, and equipment 300. When the control system 10 controls the opening and closing of valves and the like of the equipment 300 using PID (Proportional Integral Differential) or the like, the control system 10 performs the control for multiple cycles, with one cycle being from the transmission of measurement values of the equipment 300 by the local control device 200 to the acquisition of new measurement values for the equipment 300 controlled in accordance with the control values calculated by the remote control device 100. The control system 10 controls the equipment in consideration of control delays that occur due to the transmission and reception of data between the remote control device 100 and the local control device 200 during such control.

[0023] In the control system 10, the local control device 200 is installed in the facility 300 or near the controlled object of the facility 300, and may be, for example, a control device located at the site where a process is executed in a plant. The remote control device 100 is located at a location remote from the local control device 200, and may be, for example, a control instruction device or a control data calculation device located in a management center of the plant.

[0024] The facility 300 is a facility or device in which a device to be controlled is installed. For example, the facility 300 may be a plant, or may be a device installed within a plant. Examples of plants include industrial plants such as chemical and bio plants, plants that manage and control wellheads and surrounding areas of gas fields and oil fields, plants that manage and control power generation such as hydroelectric, thermal, and nuclear power, plants that manage and control environmental power generation such as solar and wind power, and plants that manage and control water supply and sewage systems, dams, etc.

[0025] The controlled object in the facility 300 may be, for example, an actuator, i.e., an operating element, such as a valve, heater, motor, fan, or switch, that controls at least one physical quantity, such as the amount, temperature, pressure, flow rate, speed, or pH, of an object in the process of the facility 300, and performs a given operation according to the operating amount.

[0026] The facility 300 may be provided with one or more sensors capable of measuring various conditions (physical quantities) inside and outside the facility 300. As an example, the sensors may output measurement values measuring the temperature, flow rate, etc. at various positions in the facility 300. The measurement values related to the facility 300 may include such measurement values. Furthermore, the measurement values related to the facility 300 may include an operation amount indicating the opening / closing degree of a valve of the facility 300. In addition to data indicating the operating state as a result of such control, the measurement values may also include consumption data indicating the amount of energy or raw material consumed in the facility 300, disturbance environment data indicating physical quantities that may act as disturbances to the control of the facility 300, and the like.

[0027] Here, the control value may indicate the control of the facility 300 performed by the local control device 200, and may indicate, for example, at least one of a control operation and a control amount for a control target of the facility 300. The control value may further indicate the control target of the facility 300 using an identifier or the like. As an example, the control value may indicate closing or opening a valve of the facility 300 by n% (n>0, n% may be the opening degree of the valve), increasing or decreasing the flow rate of a predetermined flow path of the facility 300 by n%, etc.

[0028] The remote control device 100 is connected to the local control device 200 and performs learning processing of the model 135 and transmits control values calculated using the model 135 to the local control device 200. The remote control device 100 may be a computer such as a PC, tablet PC, smartphone, workstation, server computer, or general-purpose computer, or may be a computer system in which multiple computers are connected. Such a computer system is also considered a computer in a broad sense. The remote control device 100 may also be implemented by one or more executable virtual computer environments within a computer. Alternatively, the remote control device 100 may be a dedicated computer designed for plant maintenance management or dedicated hardware realized by dedicated circuits. The remote control device 100 may also be realized by cloud computing.

[0029] The remote control device 100 includes a measurement value receiving unit 110, a learning processing unit 120, a model storage unit 130, a calculation unit 140, and a control value transmitting unit 150. The measurement value receiving unit 110 is connected to the local control device 200 and receives measurement values related to the equipment 300 from the local control device 200 via a network (for example, wireless / wired, the Internet, an intranet, etc., the same applies below). The measurement value receiving unit 110 may receive a control delay measured by the local control device 200 along with the measurement values. The measurement value receiving unit 110 supplies the received measurement values and control delay to the learning processing unit 120 and the calculation unit 140.

[0030] The learning processing unit 120 is connected to the model storage unit 130, and performs machine learning using the measurement values and control delays from the measurement value receiving unit 110 to generate and update a model 135. The learned model 135 calculates a control value to be used for controlling the equipment 300 when a control delay occurs, including a communication delay with the local control device 200, from the delay amount equivalent to the control delay and the measurement values. The learning processing unit 120 supplies the learned model 135 to the model storage unit 130.

[0031] The model storage unit 130 is connected to the calculation unit 140 and stores the model 135 learned by the learning processing unit 120. The model storage unit 130 supplies the model 135 to the calculation unit 140.

[0032] The calculation unit 140 is connected to the control value transmission unit 150, and uses the model 135 stored in the model storage unit 130 to calculate a control value according to the measurement value received by the measurement value reception unit 110 and the delay amount. The calculation unit 140 may acquire multiple delay amounts corresponding to control delays including communication delays with the local control device 200, input the multiple delay amounts and the measurement values to the model 135, and calculate multiple control values according to the multiple delay amounts, respectively. The calculation unit 140 supplies the control value to the control value transmission unit 150.

[0033] The control value transmitting unit 150 is connected to the local control device 200 and transmits the calculated control values to the local control device 200 via a network. The control value transmitting unit 150 may transmit the control values in association with the delay amounts used in the calculation, respectively.

[0034] The local control device 200 is connected to the equipment 300, controls the equipment 300 according to control values from the remote control device 100, and acquires and transmits measured values. Like the remote control device 100, the local control device 200 may also be a computer, such as a PC, tablet PC, smartphone, workstation, server computer, or general-purpose computer, or may be a computer system in which multiple computers are connected. The local control device 200 may also be implemented by one or more executable virtual computer environments within a computer. Alternatively, the local control device 200 may be a dedicated computer designed for plant maintenance management, or dedicated hardware realized by dedicated circuits. The local control device 200 may also be a controller using a microcontroller or the like.

[0035] The local control device 200 includes a measurement value transmitting section 210 , a control value receiving section 220 , a delay measuring section 230 , a selecting section 240 , and a control section 250 .

[0036] The measurement value transmitting unit 210 is connected to the remote control device 100, the control unit 250, and the delay measuring unit 230, acquires measurement values related to the equipment 300 from the control unit 250, and transmits the measurement values to the remote control device 100 via the network. The measurement value transmitting unit 210 may transmit the control delay measured by the delay measuring unit 230 to the remote control device 100 together with the measurement values. The measurement value transmitting unit 210 may supply information indicating the time at which the measurement values were transmitted to the remote control device 100 to the delay measuring unit 230.

[0037] The control value receiving unit 220 is connected to the delay measurement unit 230 and the selection unit 240, and receives from the remote control device 100, via the network, a plurality of control values corresponding to the measurement values transmitted by the measurement value transmission unit 210. The control value receiving unit 220 may receive a plurality of control values calculated from the measurement values transmitted immediately before by the measurement value transmission unit 210. The control value receiving unit 220 supplies the plurality of control values to the selection unit 240. The control value receiving unit 220 may supply, to the delay measurement unit 230, information indicating the time at which the control values were received.

[0038] The delay measurement unit 230 is connected to the selection unit 240 and may determine a control delay by measuring the period from when the measurement value transmission unit 210 transmits a measurement value until the control value reception unit 220 receives a plurality of control values corresponding to the measurement value. The delay measurement unit 230 may determine the control delay based on information from the control value reception unit 220 and the measurement value transmission unit 210. The delay measurement unit 230 provides the determined control delay to the selection unit 240. The delay measurement unit 230 may provide the determined control delay to the measurement value transmission unit 210.

[0039] The selection unit 240 is connected to the control unit 250 and selects a control value to be used for controlling the equipment 300 from among the plurality of control values in accordance with a control delay including a communication delay with the remote control device 100. The selection unit 240 may select a control value to be used for controlling the equipment 300 from among the plurality of control values in accordance with the control delay determined by the delay measurement unit 230. The selection unit 240 supplies control data indicating the selected control value to the control unit 250.

[0040] The control unit 250 is connected to the equipment 300 and controls the equipment 300 in accordance with the selected control value. The control unit 250 controls the equipment 300, such as opening and closing a valve, and may further acquire new measurement values for the equipment 300 after the control. The control unit 250 may acquire multiple types of measurement values measured by multiple different sensors related to the equipment 300. The control unit 250 supplies the acquired measurement values to the measurement value transmission unit 210.

[0041] 2 is an explanatory diagram of communication over one cycle in the control system 10. In Fig. 2, the dotted lines of the remote control device 100, the local control device 200, and the equipment 300 indicate the passage of time with downward dotted lines, the transmission and reception of data between the remote control device 100, the local control device 200, and the equipment 300 is indicated by solid arrows, and the measurement of the equipment 300 by the local control device 200 is indicated by a downward arrow. In Fig. 2, the local control device 200 transmits measurement values to the remote control device 100, the remote control device 100 calculates control values according to the measurement values and transmits them to the local control device 200, and the local control device 200 controls the equipment 300 according to the received control values to obtain new measurement values for the equipment 300.

[0042] In the control system 10, communication is required for sending and receiving data between the remote control device 100 and the local control device 200. This communication is subject to fluctuations, and the period from transmission of a measurement value to reception of a control value (the communication period shown in FIG. 2) varies, resulting in an indefinite control delay due to communication delay. As a result, a control operation according to a control value determined to be optimal may no longer be optimal at the timing when the control operation is performed on the equipment 300. Therefore, in the control system 10 of this embodiment, a model 135 that takes control delay into account is used to calculate a control value according to the amount of delay equivalent to the control delay.

[0043] 3 shows an example of a processing flow of the remote control device 100 of this embodiment. Note that this operation may be started in response to receiving a measurement value from the local control device 200.

[0044] In step S11, the measurement value receiving unit 110 receives the measurement value from the local control device 200. The measurement value receiving unit 110 supplies the measurement value to the calculation unit 140.

[0045] In step S12, the calculation unit 140 acquires multiple delay amounts. The calculation unit 140 may acquire multiple delay amounts via user input. The calculation unit 140 may use, as the delay amount, the amount by which a candidate value for a communication period from when the local control device 200 transmits a measurement value until when the local control device 200 receives a control value corresponding to the measurement value from the control value transmission unit 150 exceeds a preset margin time. The candidate value for the communication period may be set via user input based on the communication period of the cycle prior to the current cycle. The margin time may be set via user input to be less than the length of one control cycle from when the local control device 200 transmits a measurement value until just before transmitting the measurement value in the next cycle.

[0046] As an example, the average communication period measured over multiple cycles is approximately 50 ms, and the communication period can vary by up to 1 s above the average value due to communication fluctuations. If the control period is 200 ms, the slack time can be set to 100 ms and the delay amount can be set in increments of 200 ms to ensure sufficient time for the local control device 200 to acquire measurements. Therefore, the calculation unit 140 may use communication period candidate values of 100, 300, 500, 700, 900, and 1100 ms, and delay amounts of 0, 200, 400, 600, 800, and 1000 ms. Note that if the average communication period is close to 0 ms and communication fluctuations occur occasionally, the slack time may be set to 0 ms.

[0047] Furthermore, the calculation unit 140 may acquire the amount of delay based on the control delay received from the local control device 200 together with the measured value. The calculation unit 140 may use, as the amount of delay, a value obtained by adding a preset value (for example, 200 ms) to the received control delay, or a difference between the control delay and the preset value. The preset value may be set via a user input. The remote control device 100 may decide to use the slack time via a user input, and in this case, may transmit an indication to the local control device 200 indicating that the slack time will be used.

[0048] In step S13, the calculation unit 140 inputs each delay amount and the measurement value into the model 135, and calculates a control value as an output of the model 135. The calculation unit 140 may use the model 135 that has undergone reinforcement learning using a known algorithm such as Kernel Dynamic Policy Programming (KDPP), Temporal Difference Learning (TD learning), or the Monte Carlo method.

[0049] As an example, the calculation unit 140 uses a model 135 based on a kernel method such as KDPP. The calculation unit 140 generates a vector of a state s from the measurement values and each delay amount. Accordingly, the calculation unit 140 generates vectors of multiple states s1 to sn (n>1) corresponding to each of the multiple delay amounts. Next, the calculation unit 140 generates multiple control operation determination tables indicating combinations of each of the states s1 to sn and all possible control operations. The calculation unit 140 then inputs each of the control operation determination tables to the model 135. The model 135 may have a weight table in which weights are associated with sample data (states and control operations). In response to the input, kernel calculations are performed between each row of the control operation determination table and each sample data of the model 135, and the distance between each sample data is calculated. Then, the distances calculated for each sample data are multiplied by the respective weights and the results are sequentially added together to calculate a reward value for each control operation. The model 135 selects the control operation (control value) that produces the highest reward value calculated in this manner. In this way, the calculation section 140 can calculate the control value corresponding to each delay amount. The calculation section 140 may supply data (for example, a table) indicating the correspondence between the control value and each delay amount to the control value transmission section 150.

[0050] In step S14, the control value transmitting unit 150 transmits to the local control device 200 data indicating the correspondence between each control value and each delay amount.

[0051] In step S15, the remote control device 100 determines whether or not the control of the facility 300 by the local control device 200 has ended. If it has not ended (step S15; No), the process returns to step S11; if it has ended (step S15; Yes), the process ends.

[0052] 4 shows an example of a control operation decision table input to the model 135. The control operation decision table shows a state consisting of measurement values 1 and 2 measured in the same period and a delay amount Δt=200, as well as six possible control operations. The control operation 5 indicates that the valve is opened 5%, 3 indicates that the valve is opened 3%, 1 indicates that the valve is opened 1%, 0 indicates that the valve is maintained in its current state, -3 indicates that the valve is closed 3%, and -5 indicates that the valve is closed 5%. The delay amount Δt is the amount by which the candidate value for the communication period exceeds the 100 ms margin time, and the calculation unit 140 creates similar control operation decision tables for other delay amounts (for example, Δt=0, 400, 600, 800, and 1000 ms).

[0053] 5 shows an example of a weight table of the model 135. The model 135 has a weight table configured with sample data, which is a combination of a state s indicating a set of measured values and delay amounts of measured control delays and a control operation taken under each state, and weights calculated based on reward values. Note that such weights may be determined so that they become larger as the reward value determined by the reward function calculated by the learning processing unit 120 becomes larger.

[0054] FIG. 6 shows an example of a table showing the correspondence between a plurality of control values and a plurality of delay amounts. The table shows an index, a delay amount Δt, and a control operation. The delay amount Δt is the amount by which the candidate value for the communication period exceeds the margin time of 100 ms. In the table, each index is associated with a delay amount and a control operation indicated by the control value. The control value transmitting unit 150 may transmit a table such as that shown in FIG. 6 to the local control device 200.

[0055] 7 shows an example of a processing flow of the local control device 200 of this embodiment. Note that this operation may be started in response to a user's instruction input to the local control device 200. The user's instruction input may include input of an identifier (such as a device name or identification number) indicating the facility 300 to be controlled.

[0056] In step S21, the control unit 250 acquires measurement values from the facility 300, and the measurement value transmission unit 210 transmits the measurement values to the remote control device 100. The control unit 250 may acquire measurement values received directly from one or more sensors or the like of the facility 300, may acquire measurement values from a computer or the like arranged in the facility 300, or may acquire measurement values related to the facility 300 measured directly by the local control device 200. The measurement value transmission unit 210 may transmit the measurement values to the remote control device 100 together with an identifier indicating the type of the measurement value or the like. The measurement value transmission unit 210 supplies data indicating the time when the measurement value was transmitted (for example, a timestamp or the like) to the delay measurement unit 230.

[0057] In step S22, the control value receiving unit 220 receives a plurality of control values corresponding to the transmitted measurement values from the remote control device 100. The control value receiving unit 220 supplies data (such as a timestamp, for example) indicating the time at which the control values were received to the delay measurement unit 230.

[0058] In step S23, the delay measurement unit 230 determines the control delay for the current cycle. The delay measurement unit 230 may measure the period from the transmission time of the measurement value to the reception time of the control value, using data indicating the time at which the measurement value was transmitted from the measurement value transmission unit 210 and data indicating the time at which the control value was received from the control value reception unit 220, for the current cycle. The delay measurement unit 230 may also measure the period from the time at which the control unit 250 acquired the measurement value to the time at which the selection unit 240 received the control value from the control value reception unit 220.

[0059] If the remote control device 100 uses a portion exceeding the slack time as the delay amount in step S12, the delay measurement unit 230 may determine the portion of the measured period exceeding the preset slack time as the control delay. The slack time may be set via user input and may be the same as the slack time used in step S12 in the remote control device 100. The local control device 200 may determine to use the slack time via user input or by receiving an indication of the slack time from the remote control device 100. Alternatively, if the slack time is not used, the delay measurement unit 230 may determine the measured period as the control delay. The delay measurement unit 230 supplies the control delay to the selection unit 240.

[0060] In step S24, the selector 240 selects one control value from the received control values in accordance with the determined control delay. The selector 240 may compare the determined control delay with the multiple delay amounts in the table and select the control value corresponding to the delay amount closest to the determined control delay. Furthermore, if the control delay is a value between two delay amounts, the selector 240 may determine the control operation by adding the manipulated variable of the control value weighted according to the difference from the delay amount. For example, if the control delay is 20 ms, the selector 240 may select from the table of FIG. 6 a manipulated variable of 5% for the control value with a delay amount of 0 ms and a manipulated variable of 3% for the control value with a delay amount of 200 ms. Since (5% × 180 / 200) + (3% × 20 / 200) = 4.8%, the selector 240 may determine the control operation to open the valve by 4.8%. The selector 240 supplies the determined control operation to the controller 250.

[0061] In step S25, the control unit 250 performs the selected control operation on the equipment 300. The control unit 250 may output control data indicating the control operation to the control target of the equipment 300. Alternatively, the control unit 250 may directly perform a control operation on the control target according to the selected control value. When a preset margin time is used, the control unit 250 controls the equipment 300 after at least the preset margin time has elapsed since the measurement value transmission unit 210 transmitted the measurement value. Even if the control unit 250 has already received control data from the selection unit 240, the control unit 250 does not perform control until the margin time has elapsed after the transmission of the measurement value in step S21. If the control unit 250 receives control data from the selection unit 240 after the margin time has elapsed, the control unit 250 may immediately perform the control operation.

[0062] In step S26, the local control device 200 determines whether or not the control of the equipment 300 has been completed. If not (step S26; No), the process returns to step S21; if completed (step S26; Yes), the process ends.

[0063] The processing flow of the remote control device 100 shown in FIG. 3 and the processing flow of the local control device 200 shown in FIG. 7 may be executed in parallel, for example, in the order of step S21, step S11, step S12, step S13, step S14, step S15, step S22, step S23, step S24, step S25, and step S26.

[0064] 8 shows an example of a learning flow of the remote control device 100 in the control system 10 according to this embodiment. The learning processing unit 120 of the remote control device 100 may perform reinforcement learning of the model 135 using a known algorithm such as KDPP, TD learning, or the Monte Carlo method. An example of reinforcement learning using a kernel method such as KDPP is shown below.

[0065] In step S31, the learning processing unit 120 acquires a target value. The target value may be the same type of parameter as one of the measured values (for example, the water level of a tank), or may be the same type of parameter as the control operation amount indicated by the control value (for example, the opening of a valve). The learning processing unit 120 may acquire the target value via user input. Alternatively, the learning processing unit 120 may acquire the same target value as the target value used in the previous learning process.

[0066] In step S32, the learning processing unit 120 determines a reward function using the target value. The learning processing unit 120 may determine the reward function so that the reward value becomes high when the state of the equipment 300 controlled according to the control value calculated from the model 135 approaches a state corresponding to the target value. The learning processing unit 120 may also determine the reward function so that the reward value becomes high when the measurement value of the equipment 300 controlled by the model 135 satisfies the contents of the target value.

[0067] In step S33, similarly to steps S21 and S23, the control unit 250 acquires the measurement values and the control delay amount Δt related to the facility 300 and transmits them to the remote control device 100. For example, the control unit 250 may acquire the measurement values and the control delay amount Δt from the facility 300. When acquiring the first measurement values, the control unit 250 may set the control delay amount Δt to 0, for example. Furthermore, the remote control device 100 may acquire the measurement values and the control delay amount Δt from a simulator.

[0068] In step S34, similar to step S13, the calculation unit 140 determines the control value using the model 135. During learning, the calculation unit 140 may randomly determine the control value. The calculation unit 140 supplies the control value to the control value transmission unit 150, which transmits the control value to the local control device 200.

[0069] In step S35, the local control device 200 controls the equipment 300 according to the supplied control value. As in steps S23 and S24, the local control device 200 may select a control value according to the measured control delay. The local control device 200 may generate a control delay with a randomly set delay amount and select a control value according to the control delay. The local control device 200 may also cause a simulator to perform a simulation according to the supplied control value. The local control device 200 transmits to the remote control device 100 new measurement values obtained in response to controlling the equipment 300 using the control value, and the delay amount Δt of the measured control delay or the delay amount Δt used when the used control value was calculated by the model 135.

[0070] In step S36, the measurement value receiving unit 110 receives from the local control device 200 new measurement values obtained in response to controlling the equipment 300 using the calculated control values and the control delay Δt corresponding to the used control values. This allows the measurement values of the state after the change in response to the control operation of the equipment 300 using the determined control values to be acquired. The measurement value receiving unit 110 supplies the received measurement values and delay Δt to the learning processing unit 120.

[0071] In step S37, the learning processing unit 120 calculates a reward value based on the acquired measurement values and the amount of control delay. The learning processing unit 120 may calculate the reward value using the reward function determined in step S32.

[0072] In step S38, the learning processing unit 120 determines whether the process of acquiring the measurement value and the control delay amount corresponding to the control has exceeded a specified number of steps. Note that this number of steps may be specified in advance by the user, or may be determined based on the learning period (e.g., 10 days). If it is determined that the acquisition process has not exceeded the specified number of steps (step S38; No), the learning processing unit 120 returns the process to step S34 and continues the flow. As a result, the process of acquiring the measurement value and the control delay amount corresponding to the control is executed the specified number of steps.

[0073] In step S38, if it is determined that the above-mentioned process has been performed more than the designated number of steps (step S38; Yes), the learning processing unit 120 advances the process to step S39.

[0074] In step S39, the learning processing unit 120 calculates a weight for each sample data from the reward value, and updates the model 135. For example, the learning processing unit 120 overwrites the values of the weight column in the weight table of the model 135 shown in Fig. 5, and adds new sample data that has not been saved to the model 135.

[0075] In step S40, the learning processing unit 120 determines whether the update process of the model 135 has exceeded a specified number of repetitions. Note that such a number of repetitions may be specified in advance by the user, or may be determined depending on the validity of the model 135. If it is determined that the above process has not exceeded the specified number of repetitions (step S40; No), the learning processing unit 120 returns the process to step S33 and continues the flow.

[0076] If it is determined in step S40 that the above-described process has been repeated a specified number of times (step S40; Yes), the learning processing unit 120 ends the flow. In this way, for example, the learning processing unit 120 can generate a model 135 that outputs a control value according to the measurement values and control delays related to the equipment 300.

[0077] According to this embodiment, a control value can be calculated using a model 135 that takes into account the influence of communication fluctuations on control, and an optimal control operation of the equipment 300 can be executed according to the control value.

[0078] The model storage unit 130 may store a model 135 generated outside the remote control device 100. In this case, the remote control device 100 may not have the learning processing unit 120. A learning processing device that performs only the learning processing may include at least the measurement value receiving unit 110 and the learning processing unit 120 of this embodiment.

[0079] In addition, the calculation unit 140 may calculate a control value using the model 135 when the delay amount Δt is 0 and when the delay amount Δt > 0, and create a table in which the two control values correspond to bits 0 and 1 when the delay amount Δt is 0 and when the delay amount Δt > 0, respectively, and the control value transmission unit 150 may transmit the table to the local control device 200.

[0080] Various embodiments of the present invention may also be described with reference to flowcharts and block diagrams, where the blocks may represent (1) stages of a process in which operations are performed or (2) sections of apparatus responsible for performing the operations. Particular stages and sections may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable medium, and / or a processor provided with computer-readable instructions stored on a computer-readable medium. Dedicated circuitry may include digital and / or analog hardware circuitry, and may include integrated circuits (ICs) and / or discrete circuits. Programmable circuitry may include reconfigurable hardware circuitry including logical AND, OR, XOR, NAND, NOR, and other logic operations, flip-flops, registers, memory elements such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0081] A computer-readable medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that the computer-readable medium having instructions stored thereon comprises an article of manufacture containing instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray (RTM) disc, memory stick, integrated circuit card, etc.

[0082] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages such as the “C” programming language or similar programming languages.

[0083] The computer-readable instructions may be provided to a processor or programmable circuitry of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus 200 locally or over a wide-area network (WAN) such as a local area network (LAN), the Internet, etc., which executes the computer-readable instructions to create means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0084] 9 illustrates an example of a computer 2200 in which aspects of the present invention may be embodied, in whole or in part. Programs installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to embodiments of the present invention, and / or to perform a process or steps of a process according to embodiments of the present invention. Such programs may be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0085] A computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0086] The CPU 2212 operates according to programs stored in the ROM 2230 and RAM 2214, thereby controlling each unit. The graphics controller 2216 acquires image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into the graphics controller 2216 itself, and causes the image data to be displayed on the display device 2218.

[0087] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0088] The ROM 2230 stores therein a boot program or the like that is executed by the computer 2200 upon activation, and / or programs that depend on the hardware of the computer 2200. The input / output chip 2240 may also connect various input / output units to the input / output controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0089] The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. Information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by realizing information manipulation or processing in accordance with the use of the computer 2200.

[0090] For example, when communication is performed between the computer 2200 and an external device, the CPU 2212 may execute a communication program loaded into the RAM 2214 and instruct the communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 2212, the communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in the RAM 2214, the hard disk drive 2224, the DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0091] Furthermore, the CPU 2212 may cause all or a necessary portion of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

[0092] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and subjected to information processing. The CPU 2212 may perform various types of processing on data read from the RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 2214. The CPU 2212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored on the recording medium, the CPU 2212 may search for an entry that matches a condition specified by the attribute value of the first attribute from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0093] The above-described programs or software modules may be stored in a computer-readable medium on or near the computer 2200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable medium, thereby providing the programs to the computer 2200 via the network.

[0094] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0095] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0096] 10. Control System 100 Remote control device 110 Measurement value receiving unit 120 Learning processing unit 130 Model Memory Unit 135 model 140 Calculation Unit 150 Control value transmission unit 200 Local Control Device 210 Measurement value transmitter 220 Control value receiving unit 230 Delay measurement unit 240 Selection Section 250 control section 300 equipment 2200 Computer 2201 DVD-ROM 2210 host controller 2212 CPU 2214 RAM 2216 Graphics Controller 2218 Display Device 2220 Input / Output Controller 2222 communication interface 2224 hard disk drive 2226 DVD-ROM drive 2230 ROM 2240 I / O chip 2242 keyboard

Claims

1. a measurement value receiving unit that receives measurement values related to the equipment from a local control device that controls the equipment; a calculation unit that calculates a control value to be used for controlling the equipment when a control delay including a communication delay between the local control device and the equipment occurs, using a model that calculates a control value corresponding to the received measurement value and the delay amount from a delay amount corresponding to the control delay and the measurement value; a control value transmitting unit that transmits the calculated control value to the local control device, The calculation unit uses, as the delay amount, a portion of the candidate value for the communication period with the local control device that exceeds a preset margin time. Remote control device.

2. The calculation unit uses, as the delay amount, a portion of the candidate value of the communication period from when the local control device transmits the measurement value until when the local control device receives the control value corresponding to the measurement value from the control value transmission unit, which exceeds the preset margin time. The remote control device according to claim 1 .

3. the calculation unit calculates, using the model, a plurality of the control values corresponding to the plurality of delay amounts from the received measurement values; The control value transmitting unit transmits the plurality of control values to the local control device. The remote control device according to claim 1 .

4. The control value transmitting unit transmits data indicating a correspondence relationship between the plurality of control values and the plurality of delay amounts to the local control device.

4. The remote control device according to claim 3.

5. the measurement value receiving unit receives new measurement values obtained in response to controlling the equipment using the calculated control values and control delays corresponding to the used control values; a learning processing unit that updates the model using the received control delay and the new measurement value; The remote control device according to claim 1 .

6. a measurement value transmitting unit that transmits measurement values related to the equipment to a remote control device that calculates a control value according to the measurement values; a control value receiving unit that receives, from the remote control device, a plurality of control values corresponding to the transmitted measurement values; a selection unit that selects a control value to be used for controlling the equipment from the plurality of control values in accordance with a control delay including a communication delay with the remote control device; a control unit that controls the equipment in accordance with the selected control value, The selection unit uses, as the control delay, a time period during which communication with the remote control device exceeds a preset allowance time. Local control device.

7. a delay measurement unit that measures the communication period from when the measurement value transmitter transmits the measurement value until when the control value receiver receives the plurality of control values corresponding to the measurement value, and determines the control delay; The selector selects a control value to be used for controlling the equipment from among the plurality of control values in accordance with the control delay determined by the delay measurer. The local control device of claim 6 .

8. the delay measurement unit determines, as the control delay, a period during which the communication period from when the measurement value transmitter transmits the measurement value until when the control value receiver receives the plurality of control values corresponding to the measurement value exceeds the predetermined margin time; The control unit controls the equipment after at least the preset margin time has elapsed since the measurement value transmission unit transmitted the measurement value. The local control device of claim 7.

9. a measurement value receiving unit that receives a control delay, including a communication delay between the remote control device and the local control device, and a measurement value related to the equipment, the control delay being acquired in a local control device that controls the equipment according to a control value received from the remote control device; a learning processing unit that generates a model that calculates a control value to be used for controlling the equipment from a delay amount corresponding to the control delay and the measurement value, The learning processing unit uses, as the delay amount, a time period during which the communication period between the remote control device and the local control device exceeds a preset allowance time. Learning processing device.

10. a measurement value receiving step of receiving measurement values relating to the equipment from a local control device that controls the equipment; a calculation step of calculating a control value to be used for controlling the equipment when a control delay including a communication delay between the local control device and the equipment occurs, using a model that calculates a control value corresponding to the received measurement value and the delay amount from the measurement value and a delay amount corresponding to the control delay; a control value transmission step of transmitting the calculated control value to the local control device; In the calculation step, the amount of delay is determined by the amount of time that the candidate value for the communication period with the local control device exceeds a preset margin time. method.

11. Computer, a measurement value receiving unit that receives measurement values related to the equipment from a local control device that controls the equipment; a calculation unit that calculates a control value to be used for controlling the equipment when a control delay including a communication delay between the local control device and the equipment occurs, using a model that calculates a control value corresponding to the received measurement value and the delay amount from a delay amount corresponding to the control delay and the measurement value; a control value transmitting unit that transmits the calculated control value to the local control device; It functions as The calculation unit uses, as the delay amount, a portion of the candidate value for the communication period with the local control device that exceeds a preset margin time. program.

12. a measurement value transmission step of transmitting measurement values related to the equipment to a remote control device that calculates a control value according to the measurement values; a control value receiving step of receiving, from the remote control device, a plurality of control values corresponding to the transmitted measurement values; a selection step of selecting a control value to be used for controlling the equipment from the plurality of control values in accordance with a control delay including a communication delay with the remote control device; a control step of controlling the equipment in accordance with the selected control value; The selection step uses, as the control delay, a time period during which communication with the remote control device exceeds a preset allowance time. method.

13. Computer, a measurement value transmitting unit that transmits measurement values related to the equipment to a remote control device that calculates a control value according to the measurement values; a control value receiving unit that receives, from the remote control device, a plurality of control values corresponding to the transmitted measurement values; a selection unit that selects a control value to be used for controlling the equipment from the plurality of control values in accordance with a control delay including a communication delay with the remote control device; A control unit that controls the equipment in accordance with the selected control value It functions as The selection unit uses, as the control delay, a time period during which communication with the remote control device exceeds a preset allowance time. program.

14. a measurement value receiving step of receiving a control delay including a communication delay between the remote control device and the local control device and a measurement value related to the equipment, the control delay being acquired in a local control device that controls the equipment according to a control value received from the remote control device; a learning processing step for generating a model that calculates a control value to be used for controlling the equipment from a delay amount corresponding to the control delay and the measurement value, The learning process uses, as the delay amount, the amount of time that the communication period between the remote control device and the local control device exceeds a preset allowance time. method.

15. Computer, a measurement value receiving unit that receives a control delay, including a communication delay between the remote control device and the local control device, and a measurement value related to the equipment, the control delay being acquired in a local control device that controls the equipment according to a control value received from the remote control device; A learning processing unit that generates a model that calculates a control value to be used for controlling the equipment from a delay amount corresponding to the control delay and the measurement value. It functions as The learning processing unit uses, as the delay amount, a time period during which the communication period between the remote control device and the local control device exceeds a preset allowance time. program.

Images