Operation support system, operation support method, and program

Abstract

To provide a technology capable of materializing highly precise operation support even when sufficient data concerning an object plant is unavailable.SOLUTION: An operation support system in accordance with an embodiment of the present disclosure supports operation of an object plant, and includes an advance learning unit that learns a model which uses operation result data of a plant other than the object plant, inputs operation data representing a condition of the plant, and outputs a recommended set value for the plant, a fine tuning unit that uses the operation result data of the object plant to finely tune the model learned by the advance learning unit, a recommended set value calculation unit that uses the model to calculate the recommended set value from the operation data of the object plant every time the operation data is acquired from the object plant, and a proposal unit that proposes the recommended set value, which is calculated by the recommended set value calculation unit, to an operator of the object plant.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a driving assistance device, a driving assistance method, and a program. [Background technology]

[0002] In plants such as combustion furnaces, stable operation is achieved by operators changing the settings while taking into account past conditions and measurements of the furnace combustion state, etc. This places a heavy burden on the operator, and various techniques have been proposed to support the operator's operation. For example, Patent Document 1 describes a technique that uses a model created by a neural network to output recommended operations according to the operating state of the plant. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-159675 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, depending on the plant for which operation support is to be performed (hereinafter, referred to as the target plant), there may be cases where there is insufficient data representing the measured values ​​and set values ​​of the state of the target plant, etc. For this reason, for example, the technology described in Patent Document 1 may not be able to create a highly accurate model.

[0005] The present disclosure has been made in consideration of the above points, and provides a technology that can realize accurate operation assistance even when sufficient data regarding a target plant is not available. [Means for solving the problem]

[0006] An operation assistance device according to one aspect of the present disclosure is an operation assistance device for assisting in the operation of a target plant, and includes: a pre-learning unit that uses operating history data of plants other than the target plant to learn a model that takes operating data representing a plant state as input and outputs recommended setting values ​​for the plant; a fine-tuning unit that uses the operating history data of the target plant to fine-tune the model learned by the pre-learning unit; a recommended setting value calculation unit that calculates recommended setting values ​​from the operating data of the target plant using the model each time operating data is obtained from the target plant; and a proposal unit that proposes the recommended setting values ​​calculated by the recommended setting value calculation unit to an operator of the target plant. Effect of the Invention

[0007] A technology is provided that can realize accurate operation assistance even when sufficient data about the target plant is not available. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of an overall configuration of a plant control system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the driving assistance device according to the present embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a driving assistance device according to the present embodiment. [Figure 4] 10 is a flowchart illustrating an example of offline processing according to the embodiment. [Diagram 5] 10 is a flowchart illustrating an example of online processing according to the embodiment. [Figure 6] FIG. 1 is a diagram (part 1) showing an example of ranking of model outputs. [Figure 7] FIG. 2 is a diagram (part 2) showing an example of ranking of model outputs. [Figure 8] 1 is a schematic diagram of a waste incineration plant according to one embodiment; [Figure 9]FIG. 4 is a diagram illustrating recommended setting values ​​for each model in an embodiment. [Figure 10] FIG. 11 is a diagram illustrating settings of an operator in an embodiment. [Figure 11] FIG. 1 is a diagram illustrating a schematic diagram of learning a reward function in one embodiment. [Figure 12] FIG. 1 is a diagram illustrating model learning in an embodiment. [Figure 13] FIG. 10 is a diagram illustrating a result of model learning in one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An embodiment of the present invention will be described below. In the following embodiment, a plant control system 1 including an operation assistance device 10 that can realize accurate operation assistance even when sufficient data on a target plant (measured values ​​obtained by measuring the state of the target plant, settings of the target plant) is not available will be described. The target plant is a plant that is the subject of operation assistance by the operation assistance device 10.

[0010] Hereinafter, data consisting of measurements of the plant state will be referred to as "operational data." Operational data consists of measurements of physical quantities (e.g., temperature, pressure, flow rate, gas concentration, etc.) that represent various plant states, measured by various sensors. In other words, operational data is multivariate data consisting of variables (also called "state variables") that represent each physical quantity.

[0011] In the following, a set of operation data at a certain time and the set value set in the plant by the operator when the operation data was obtained (i.e., the set value at the time) will be referred to as "plant operation record data." For example, when the operation data at time t is expressed as x t , its operation data x t The setpoint value set by the operator to the plant when t Then, the plant operation record data is (x t ,y t) where the operation data at time t is x t may be multivariate data representing time series data of state variables measured between time t-1 and time t.

[0012] Furthermore, in the following, in order to use the plant operation record data of plants other than the target plant, the target plant will be represented by p' and the plants other than the target plant by p∈[P] to distinguish the target plant from the other plants, where P is the total number of plants other than the target plant.

[0013] <Overall configuration example of plant control system 1> An example of the overall configuration of a plant control system 1 according to this embodiment is shown in Fig. 1. As shown in Fig. 1, the plant control system 1 according to this embodiment includes an operation support device 10, an operator terminal 20, and a target plant 30. Here, the operation support device 10 and the operator terminal 20 are communicatively connected via an arbitrary communication network. Similarly, the operator terminal 20 and the target plant 30 are communicatively connected via an arbitrary communication network, and the target plant 30 and the operation support device 10 are communicatively connected via an arbitrary communication network.

[0014] The operation support device 10 learns a model (hereinafter also referred to as a "recommended setting value calculation model") that calculates recommended setting values ​​from operation data of the target plant 30 offline. In addition, every time the operation support device 10 acquires operation data from the target plant 30 online, it calculates recommended setting values ​​using the learned recommended setting value calculation model, proposes them to the operator, and dynamically re-learns the recommended setting value calculation model. Here, the recommended setting values ​​are setting values ​​recommended to the operator. In addition, the recommended setting value calculation model is a machine learning model that calculates and outputs recommended setting values ​​using operation data as input. Note that "online" refers to a state in which the target plant 30 is in operation. Meanwhile, "offline" refers to a state other than online, for example, a state before the target plant 30 starts operation or during operation stop.

[0015] The operator terminal 20 is any of various terminals (e.g., a PC (personal computer), a smartphone, a tablet terminal, a wearable device, etc.) operated by an operator of the target plant 30. The operator determines the setting values ​​to be actually set in the target plant 30 with reference to the recommended setting values ​​proposed by the operation support device 10, and then operates the operator terminal 20 to set the setting values ​​in the target plant 30. As a result, the operation of the target plant 30 is controlled by the setting values. Note that the setting values ​​refer to values ​​of operation quantities and the like that are set in the target plant 30.

[0016] The target plant 30 is a variety of plants that are targets of operation assistance. The target plant 30 is not limited to a specific plant, but an example is a waste incineration plant. The target plant 30 is not necessarily limited to a plant, and may be, for example, a supply and demand system such as an energy management system.

[0017] 1 is merely an example, and may have other configurations. For example, the operation support device 10 and the operator terminal 20 may be integrated. Also, a plurality of operator terminals 20 may be provided.

[0018] <Hardware configuration example of the driving support device 10> An example of a hardware configuration of the driving support device 10 according to this embodiment is shown in Fig. 2. As shown in Fig. 2, the driving support device 10 according to this embodiment has an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. Each of these pieces of hardware is connected to each other via a bus 109 so as to be able to communicate with each other.

[0019] The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, etc. The display device 102 is, for example, a display, a display panel, etc. Note that the driving assistance device 10 does not necessarily have to have at least one of the input device 101 and the display device 102.

[0020] The external I / F 103 is an interface with an external device such as a recording medium 103a. Examples of the recording medium 103a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0021] The communication I / F 104 is an interface for connecting the driving assistance device 10 to a communication network. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The auxiliary storage device 107 is a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD) and stores programs and data. The processor 108 is various types of arithmetic devices such as a central processing unit (CPU) or a graphics processing unit (GPU).

[0022] 2 is an example, and the driving support device 10 may have other hardware configurations. For example, the driving support device 10 may have multiple auxiliary storage devices 107 and multiple processors 108, or may have various hardware other than the hardware shown in the figure.

[0023] <Example of functional configuration of driving assistance device 10> An example of a functional configuration of the driving support device 10 according to this embodiment is shown in FIG. 3. As shown in FIG. 3, the driving support device 10 according to this embodiment has an offline processing unit 201 and an online processing unit 202. These units are realized, for example, by a process in which one or more programs installed in the driving support device 10 are executed by the processor 108 or the like. The driving support device 10 according to this embodiment also has a plant operation record storage unit 203 and a target plant operation record storage unit 204. These storage units are realized, for example, by a storage area of ​​a storage device such as the auxiliary storage device 107. At least one of the storage units of the plant operation record storage unit 203 and the target plant operation record storage unit 204 may be realized by a storage area of ​​a storage device connected to the driving support device 10 via a communication network.

[0024] The offline processing unit 201 learns a recommended setting value calculation model offline using plant operation record data of plants other than the target plant 30 and past plant operation record data of the target plant 30 (hereinafter also referred to as "target plant operation record data"). Here, the offline processing unit 201 includes a pre-learning unit 211 and a fine tuning unit 212. The pre-learning unit 211 pre-learns a recommended setting value calculation model using plant operation record data of plants other than the target plant 30. The fine tuning unit 212 fine-tunes the pre-learned recommended setting value calculation model using the target plant operation record data. As a result, the pre-learned recommended setting value calculation model is fine-tuned for the target plant 30, and a learned recommended setting value calculation model is obtained.

[0025] Every time operation data is acquired from the target plant 30 online, the online processing unit 202 uses the operation data and the learned recommended setting value calculation model to calculate recommended setting values ​​and re-learns the recommended setting value calculation model. Here, the online processing unit 202 includes a recommended setting value calculation unit 221, a proposal unit 222, a reward function learning unit 223, and a model learning unit 224. The recommended setting value calculation unit 221 calculates recommended setting values ​​using operation data acquired from the target plant 30 and the learned recommended setting value calculation model. The proposal unit 222 transmits the recommended setting value calculated by the recommended setting value calculation unit 221 to the operator terminal 20. As a result, the recommended setting value is proposed to the operator. The reward function learning unit 223 uses the target plant operation performance data acquired while online to learn a reward function that outputs a higher reward as the recommended setting value is closer to the setting value actually set by the operator in the target plant 30. The model learning unit 224 uses the target plant operation performance data obtained while online and the reward function learned by the reward function learning unit 223 to re-learn the recommended setting value calculation model so as to maximize the reward calculated by the reward function.

[0026] The plant operation record storage unit 203 stores plant operation record data of plants other than the target plant 30. The plant operation record data is provided to the operation support device 10 offline.

[0027] The target plant operation performance storage unit 204 stores the target plant operation performance data of the target plant 30. The target plant operation performance data is provided to the operation support device 10 offline, and is created from operation data and setting values ​​collected online. It is assumed that the target plant operation performance data provided to the operation support device 10 offline is smaller in amount than the plant operation performance data stored in the plant operation performance storage unit 203. In other words, it is assumed that only an insufficient amount of target plant operation performance data is provided offline to learn a highly accurate recommended setting value calculation model.

[0028] <Offline processing> The offline processing according to this embodiment will be described below with reference to Fig. 4. Note that the offline processing is executed before the online processing described later.

[0029] The pre-learning unit 211 of the offline processing unit 201 pre-learns a recommended setting value calculation model using actual plant operation data of plants other than the target plant 30 (step S101). For example, the pre-learning unit 211 uses a known optimization method to calculate parameters of the recommended setting value calculation model by the following formula (1). As a result, the recommended setting value calculation model in which these parameters are set is obtained as a pre-learned recommended setting value calculation model.

[0030]

number

[0031]

number

[0032] Also,

[0033]

number

[0034]

number

[0035] Note that N is an integer value of 1 or more, but is preferably 2 or more. This is because N recommended setting values ​​calculated by N recommended setting value calculation models are proposed to the operator, and so when N≧2, the operator can determine actual setting values ​​while comparing a plurality of recommended setting values.

[0036] Next, the fine tuning unit 212 of the offline processing unit 201 fine-tunes the pre-trained recommended setting value calculation model obtained in the above step S101 using the target plant operation record data (step S102). For example, the fine tuning unit 212 uses a known optimization method to calculate the parameters of the recommended setting value calculation model by the following formula (2). As a result, the recommended setting value calculation model in which these parameters are set is obtained as a trained recommended setting value calculation model.

[0037]

number

[0038]

number

[0039]

number

[0040] As described above, in the operation support device 10 according to this embodiment, the recommended setting value calculation model is pre-learned using the plant operation record data of plants other than the target plant 30, and then the recommended setting value calculation model is fine-tuned using the target plant operation record data of the target plant 30. As a result, knowledge of plants other than the target plant 30 is transferred to the recommended setting value calculation model, so that even when there is only a small amount of target plant operation record data (for example, when the target plant 30 has just started operation), a highly accurate recommended setting value calculation model can be obtained.

[0041] <Online processing> Hereinafter, the online processing according to this embodiment will be described with reference to FIG. 5. Steps S201 to S205 in FIG. 5 are repeatedly executed every time operation data is acquired from the target plant 30. Hereinafter, the online start time is referred to as t p' = 1, and the target plant operation record storage unit 204 stores the time t p' =1~T p' 2 The operation record data of the target plant up to time t p' =T p'2 A case where the above operational data is acquired from the target plant 30 will be described.

[0042] The recommended setting value calculation unit 221 of the online processing unit 202 calculates recommended setting values ​​by using the operation data acquired from the target plant 30 and the learned recommended setting value calculation model (step S201). That is, for n=1, . . . , N, the recommended setting value calculation unit 221 calculates the recommended setting values ​​by using the following formula:

[0043]

number

[0044] Next, the suggestion unit 222 of the online processing unit 202 transmits the N recommended setting values ​​calculated in the above step S201 to the operator terminal 20 (step S202). As a result, these N recommended setting values ​​are proposed to the operator.

[0045] Hereinafter, it is assumed that the operator refers to the N recommended setting values ​​to determine the setting values ​​that are actually to be set in the target plant 30, and then operates the operator terminal 20 to set the setting values ​​in the target plant 30. In addition, the setting values ​​are set at time t p' =T p' 2 The set value is transmitted from the operator terminal 20 to the driving support device 10. p' =T p' 2 Operation data and time t p' =T p' 2 The set value and the set value are paired at time t p' =T p' 2 The target plant operation record data is stored in the target plant operation record storage unit 204.

[0046] Next, the reward function learning unit 223 of the online processing unit 202 uses the target plant operation performance data obtained while online (i.e., at time t p' =1~T p' 2 The reward function is learned using the target plant operation performance data up to the target plant operation performance data (step S203). For example, the reward function learning unit 223 uses a known optimization method to calculate the parameters of the reward function by the following formula (3). As a result, the reward function in which the parameters are set is obtained as a learned reward function.

[0047]

number

[0048] Next, the model learning unit 224 of the online processing unit 202 uses the target plant operation record data obtained while online (i.e., time t p' =1~T p' 2 The recommended setting value calculation model is retrained using the target plant operation performance data up to ... n ←w n Then, using existing optimization methods and reinforcement learning methods, the parameter φ of the recommended setting value calculation model is calculated using the following formula (4). n This allows the parameter φ n The recommended setting value calculation model in which the above is set is obtained as a re-learned recommended setting value calculation model.

[0049]

number

[0050] Then, the model learning unit 224 of the online processing unit 202 calculates w n ←φ n (step S205). n is the parameter calculated in step S204. As a result, after new operation data is acquired from the target plant 30, the parameter w n The processes from step S201 onwards are carried out using the above.

[0051] As described above, the operation support device 10 according to this embodiment proposes recommended setting values ​​to the operator every time operation data is acquired while online, and dynamically learns a reward function and a recommended setting value calculation model using setting values ​​that are actually set in the target plant 30. This makes it possible to acquire a recommended setting value calculation model that accurately simulates the operation of the operator, and to propose accurate recommended setting values ​​to the operator.

[0052] In the online processing shown in FIG. 5, the reward function and the recommended setting value calculation model are learned for each repetition, but after a certain predetermined condition is met, only steps S201 to S202 may be executed without learning the reward function and the recommended setting value calculation model. The condition may be, for example, when the number of repetitions of steps S201 to S205 exceeds a predetermined number (including the case of once), each parameter w n has converged, etc.

[0053] <Other examples of how to learn reward functions> Hereinafter, another example of the reward function learning method in step S203 of FIG. 5 will be described.

[0054] Another example of how to learn a reward function: Part 1 Instead of the above equation (3), the parameters of the reward function may be calculated by the following equation (5).

[0055]

number

[0056] Another example of how to learn a reward function: Part 2 Each time t p' For each combination of any two recommended setting value calculation models, the outputs (i.e., recommended setting values) of the two recommended setting value calculation models may be ranked according to a certain predetermined criterion to learn a reward function.

[0057] More specifically, at each time t p' When comparing the outputs of any two recommended setting value calculation models, the recommended setting value with the higher ranking is calculated by f good , the recommended setting value with the lower ranking is f bad As, f good and f bad Then, an index d is assigned to each pair of f starting from 1 to identify the pair. Then, {(d,f good ,f bad )|d=1,···,D}(where D is f good and f bad Using the total number of pairs of (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32,

[0058]

number

[0059] Here, various criteria can be used as the ranking criteria, but for example, it is conceivable that the ranking is determined to be higher based on the recommended setting value.

[0060] As an example, N=3, and the ranking is determined by the larger recommended setting value. In this case, t p' =1 and f 2 >f 3 >f 1 , t p' =2 and f 1 >f 3 >f 2 In the case where (d,f good ,f bad ) is shown in Figure 6. As shown in Figure 6, t p' When =1, f 2 >f 3 >f 1 Therefore, (d,f good ,f bad )=(1,f 2 ,f 1 ),(2,f 3 ,f 1 ),(3,f 2 ,f 3 ) Similarly, t p' When = 2, f 1 >f 3 >f 2 Therefore, (d,f good ,f bad )=(4,f 1 ,f 2 ),(5,f 1 ,f 3 ),(6,f 3 ,f 2 )

[0061] Another example of how to learn a reward function: Part 3 In the second example of the reward function learning method, the outputs of the two recommended setting value calculation models may be allowed to have the same rank. p' It is acceptable to rank only the outputs of two specific models for calculating recommended setting values ​​(in other words, at a certain time t p' There may be combinations of recommended setting value calculation models that cannot be ranked in the order listed.

[0062] In this case, {(d,a,f good ,f bad )|d=1, . . . , D}, the parameters of the reward function may be calculated using the following equation (7).

[0063]

number

[0064] As an example, N=3, and the ranking is determined by the larger recommended setting value. In this case, t p' =1 and f 2 >f 3 =f 1 , t p' =2 and f 1 >f 3 >f 2 and t p' =3 for f 1 and f 2 Only f can be ranked. 1 >f 2 In the case where (d, a, f good ,f bad ) is shown in Figure 7. As shown in Figure 7, t p' When =1, f 2 >f 3 =f 1 Therefore, (d, a, f good ,f bad )=(1,1 / 2,f 2 ,f 1 ),(2,1 / 2,f 2 ,f 3 ) Similarly, t p' When = 2, f 1 >f 3 >f 2 Therefore, (d, a, f good ,fbad )=(3,1 / 3,f 1 ,f 2 ),(4,1 / 3,f 1 ,f 3 ),(5,1 / 3,f 3 ,f 2 ) Also, t p' When =3, f 1 and f 2 Only f can be ranked. 1 >f 2 Therefore, (d, a, f good ,f bad )=(6,1,f 1 ,f 2 )

[0065] <Another example of the recommended setting value calculation model learning method> Hereinafter, another example of the method for re-learning the recommended setting value calculation model in step S204 of FIG. 5 will be described.

[0066] - Method 1 for learning the model for calculating recommended settings When re-learning the recommended setting value calculation model, virtual driving data and recommended setting values ​​obtained by inputting the virtual driving data into the recommended setting value calculation model may be used instead of actual driving data and recommended setting values. In this case, the virtual driving data may be generated by, for example, sampling from the distribution of actual driving data.

[0067] - Method 2 for learning the model for calculating recommended settings

[0068] Let x be the distribution of the actual operating data, and instead of the above formula (4), the parameter φ of the recommended setting value calculation model is calculated using the following formula (8). n may be calculated.

[0069]

number

[0070] <Example> An example of this embodiment will be described below.

[0071] In this embodiment, a waste incineration plant is assumed as the target plant 30. A schematic diagram of a waste incineration plant is shown in FIG. 8. As shown in FIG. 8, in a waste incineration plant, waste and air are fed into a combustion furnace, and the heat generated by the combustion is converted into steam, and steam and exhaust gases such as carbon monoxide (CO) are output. In general, steam is used for power generation and the like, so it is required to increase the amount of steam generated and stabilize it. On the other hand, in order to increase the amount of steam generated, it is necessary to increase the amount of waste input and the air flow rate, but this may cause incomplete combustion, resulting in an increase in the CO concentration. For this reason, it is necessary to appropriately control the amount of waste input and the air flow rate. The amount of waste input is controlled by the operating speed of equipment called a feeder, and the air flow rate is controlled by the opening and closing angle of a valve, etc. In addition, the steam flow rate and exhaust gas concentration (CO concentration) are measured by a sensor, etc.

[0072] Therefore, the state variables of the waste incineration plant are x1: feeder speed, x2: air flow rate, x3: steam flow rate, and x4: exhaust gas concentration. p' The operation data of is p' -1~time t p' The time series data of each state variable x1, x2, x3, and x4 from time t p' Two models f that use the operation data of the model as input to calculate the time series data of the recommended setting values ​​x1 and x2 at future times 1 and f 2 Assume the following.

[0073] The model f that was pre-trained and fine-tuned in this embodiment 1 and f 2 The input and output of the model f are shown in Fig. 9. 1 and f 2 is the time t p'The model f uses the operation data as input and calculates and outputs the time series data of the recommended setting values ​​x1 and x2 (i.e., feeder speed and air flow rate) at future times. As shown in Figure 9, 1 While the recommended setting values ​​are calculated while maintaining the past setting values, model f 2 In this example, recommended setting values ​​are calculated that are higher than the previous settings.

[0074] In this embodiment, the setting values ​​set by the operator in the waste incineration plant are shown in FIG. 10. As shown in FIG. 10, the operator sets the model f 1 Recommended settings and f 2 Based on the recommended settings and the user's own experience, the user selects an operation that reduces both the feeder speed and the air flow rate.

[0075] In this example, the reward function r θ The state of learning is shown in Figure 11. As shown in Figure 11, the reward function r θ In the learning, the parameter θ of the reward function is learned so that the reward is high for the setting value actually set by the operator.

[0076] In this embodiment, the recommended setting value calculation model f 1 The re-learning state is shown in Figure 12. As shown in Figure 12, the initial value of the parameter φ1 is set to w1, and the reward function r θ Using the model f 1 The parameter φ1 of the recommended setting value calculation model f 2 Similarly, for the parameter φ2, the initial value is set to w2, and the reward function r θ Using the model f 2 The parameter φ2 is learned.

[0077] In this embodiment, the recommended setting value calculation model f 1 and f 2 The results of learning are shown in Fig. 13. As shown in Fig. 13, re-learning is performed online, so if driving data representing the same state is subsequently acquired, the model f 1This makes it possible to calculate recommended setting values ​​close to the setting values ​​that the operator actually sets. This shows that a recommended setting value calculation model that simulates the actual operation of the operator can be obtained.

[0078] <Summary> As described above, the operation support device 10 according to this embodiment pre-learns the recommended setting value calculation model offline using the plant operation record data of plants other than the target plant 30, and then fine-tunes the recommended setting value calculation model using the target plant operation record data of the target plant 30. Moreover, the operation support device 10 according to this embodiment proposes recommended setting values ​​calculated by the recommended setting value calculation model to an operator online every time operation data is acquired, and dynamically learns the reward function and the recommended setting value calculation model using setting values ​​actually set in the target plant 30.

[0079] As a result, in the operation support device 10 according to this embodiment, even if there is only a small amount of target plant operation record data, it is possible to obtain a highly accurate recommended setting value calculation model offline. Furthermore, in the operation support device 10 according to this embodiment, the recommended setting value calculation model is re-learned online from setting values ​​that are actually set in the target plant 30, so that a recommended setting value calculation model that accurately simulates the actual operation of the operator can be obtained. Furthermore, in the operation support device 10 according to this embodiment, the re-learning of the recommended setting value calculation model can be stabilized by estimating a reward function online.

[0080] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, combinations with known technologies, etc. are possible without departing from the scope of the claims. [Explanation of symbols]

[0081] 1 Plant control system 10 Driving assistance devices 20 Operator terminal 30 Target plants 101 Input Device 102 Display device 103 External I / F 103a Recording media 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage 108 processors 109 Bus 201 Offline processing unit 202 Online Processing Unit 203 Plant operation record memory unit 204 Target plant operation record memory unit 211 Pre-Learning Section 212 Fine Tuning Section 221 Recommended setting value calculation section 222 Proposal Department 223 Reward function learning part 224 Model Learning Department

Claims

1. An operation support device for supporting the operation of a target plant, a pre-learning unit that uses operation record data of a plant other than the target plant to learn a model that receives operation data representing a state of the plant as an input and outputs recommended setting values ​​for the plant; a fine tuning unit that fine-tunes the model trained by the pre-training unit by using operational record data of the target plant; a recommended setting value calculation unit that calculates recommended setting values ​​from the operation data of the target plant using the model every time operation data is acquired from the target plant; a proposal unit that proposes the recommended setting value calculated by the recommended setting value calculation unit to an operator of the target plant; A driving assistance device having the above configuration.

2. a reward function learning unit that learns a reward function that outputs a higher reward as the recommended setting value is closer to the setting value, using operating data acquired from the target plant, the recommended setting value calculated by the recommended setting value calculation unit, and a setting value that is actually set in the target plant by an operator of the target plant; a model learning unit that re-learns the model by using operation data acquired from the target plant, recommended setting values ​​calculated by inputting the operation data acquired from the target plant into the model, and a reward function learned by the reward function learning unit, so as to maximize a sum of rewards output by the reward function; The driving assistance device according to claim 1 .

3. The model learning unit includes:

3. The driving assistance device according to claim 2, wherein, instead of the driving data acquired from the target plant, virtual driving data sampled from a distribution of the driving data acquired from the target plant is used, and, instead of recommended setting values ​​calculated by inputting the driving data acquired from the target plant into the model, the recommended setting values ​​calculated by inputting the virtual driving data into the model are used to re-learn the model.

4. The model learning unit includes: The driving assistance device according to claim 2 , further comprising: re-learning the model so as to minimize a Kullback-Leibler divergence between distributions of the driving data before and after re-learning the model, and to maximize a log-likelihood of the model.

5. The reward function learning unit is 5. The driving assistance device according to claim 2, wherein the reward function is learned so as to maximize a difference between a reward calculated by the reward function from the operating data acquired from the target plant and the recommended setting value, and a reward calculated by the reward function from the operating data acquired from the target plant and the setting value.

6. The reward function learning unit is The driving assistance device according to claim 5 , wherein the reward function is learned so as to maximize a logarithm of a sigmoid function value of the difference.

7. The reward function learning unit is ranking the recommended setting values ​​calculated by the models for each combination of two of the models; 5. The driving assistance device according to claim 2, wherein the reward function is learned so as to maximize a difference between a reward calculated by the reward function from a recommended setting value having a high ranking and a reward calculated by the reward function from a recommended setting value having a low ranking.

8. The reward function learning unit is The driving assistance device according to claim 7 , wherein the reward function is learned so as to maximize an expectation or an average of the difference.

9. An operation support device for supporting the operation of the target plant, a pre-learning step of learning a model that uses operation record data of a plant other than the target plant to input operation data representing a state of the plant and outputs recommended setting values ​​for the plant; a fine-tuning step of fine-tuning the model trained by the pre-training step using operational performance data of the target plant; a recommended setting value calculation step of calculating recommended setting values ​​from the operation data of the target plant using the model each time operation data is acquired from the target plant; a proposal step of proposing the recommended setting values ​​calculated by the recommended setting value calculation step to an operator of the target plant; A driving assistance method that performs the above.

10. The operation support device for supporting the operation of the target plant includes: a pre-learning step of learning a model that uses operation record data of a plant other than the target plant to input operation data representing a state of the plant and outputs recommended setting values ​​for the plant; a fine-tuning step of fine-tuning the model trained by the pre-training step using operational performance data of the target plant; a recommended setting value calculation step of calculating recommended setting values ​​from the operation data of the target plant using the model each time operation data is acquired from the target plant; a proposal step of proposing the recommended setting values ​​calculated by the recommended setting value calculation step to an operator of the target plant; A program that executes the following.

Images