Information processing apparatus, creation method, and creation program

Abstract

To realize the creation of a data driving type simulator which can suppress error propagation.SOLUTION: An information processing apparatus comprises: an acquisition unit for acquiring time-series data on an operation amount for a process and a process value expressing the state of the process; a creation unit for creating an input-output data set with the operation amount and the process value in a first time as an input sample and with the process value in a second time following the first time as an output sample; an addition unit for adding virtual error data per process value of the input sample; and a training unit for training a machine learning model for outputting the process value in the second time with the operation amount and the process value in the first time as input by using the input-output data set with the virtual error data added.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a generation method, and a generation program. [Background technology]

[0002] In the field of plant process control, an AI (Artificial Intelligence) control method has been proposed that acquires optimal control policies through reinforcement learning.

[0003] Reinforcement learning requires repeated trial and error to find a solution for the control object, which makes it unrealistic to implement it in a production environment such as a petrochemical plant, where safety is of the utmost importance.

[0004] From this perspective, reinforcement learning is performed on a simulator that mimics the processes in the production environment, and the AI ​​control strategies that have been verified as effective in the simulation are often transferred to the real environment.

[0005] An example of a simulator that implements reinforcement learning is a simulator that uses a physical model that is constructed based on the physical characteristics of the controlled object.

[0006] Simulators that use physical models are expected to improve simulator accuracy because the physical characteristics of the controlled object are strictly defined. However, defining the physical characteristics of the controlled object requires a high level of specialized knowledge, which means that it is highly dependent on the individual, and building a physical model requires a huge amount of effort.

[0007] As another example, a statistical model that represents the dynamic input / output relationship of a process is constructed from operational data of the process in a production environment, and the constructed statistical model is used in a simulation.

[0008] When a process is a linear one-input, one-output system, a system identification method that estimates the coefficients of each variable in a mathematical equation that defines the relationship between the input and output, such as the ARX (Auto-Regressive with eXogenou) model, is used. However, when a process is a nonlinear multi-input, multi-output system, it can be difficult to build a model based on a mathematical equation.

[0009] As a further example, a dynamic predictive model may be trained using machine learning techniques such as deep learning, using process operation data to predict the next state of the process using the operation on the process and the previous state of the process as input, and used in the simulation.

[0010] Models generated by deep learning are known to be highly useful for processes with strong nonlinearities. Furthermore, even without domain knowledge of the process to be controlled, models can be easily generated using deep learning by preparing process operation data (input / output data) as training data. This makes it possible to create data-driven simulators that are suitable for both linear and nonlinear processes, as well as single-input / single-output and multi-input / multi-output processes.

[0011] However, the prediction accuracy of the dynamic prediction model described above depends on the quantity and quality of the training data, and therefore there may be domains where the prediction is accurate and domains where the prediction is inaccurate.

[0012] From this perspective, a method has been proposed in which multiple models are used to perform ensemble predictions, thereby minimizing the impact of uncertain prediction results on reinforcement learning control strategies. [Prior art documents] [Patent documents]

[0013] [Patent Document 1] Japanese Patent Publication No. 2022-25859 [Patent Document 2] Japanese Patent Application Publication No. 2023-48962 Summary of the Invention [Problem to be solved by the invention]

[0014] However, it is unrealistic to completely suppress prediction errors in the dynamic prediction model. For example, even when performing the ensemble prediction, if the number of models or the number of reinforcement learning training sessions is insufficient, prediction accuracy and generalization performance will decrease.

[0015] In the data-driven simulator, the output of the dynamic prediction model is repeatedly used as the input for the next time, and therefore, the prediction error propagates from one time to the next every time a prediction is made by the dynamic prediction model. As a result, it is difficult to prevent the prediction error from increasing over time in the data-driven simulator.

[0016] An object of the present invention is to realize the generation of a data-driven simulator capable of suppressing error propagation. [Means for solving the problem]

[0017] According to one aspect, an information processing device includes an acquisition unit that acquires time-series data relating to an operation variable for a process and a process value that represents a state of the process; a creation unit that creates an input / output data set in which the operation variable and the process value at a first time are input samples and the process value at a second time subsequent to the first time are output samples; an addition unit that adds virtual error data to each process value of the input sample; and a training unit that uses the input / output data set to which the virtual error data has been added to train a machine learning model that inputs the operation variable and the process value at the first time and outputs a process value at the second time.

[0018] In one aspect, the generation method is performed by a computer to perform the following steps: acquire time-series data on manipulated variables for a process and process values ​​representing the state of the process; create an input-output data set in which the manipulated variables and process values ​​at a first time are input samples and the process values ​​at a second time subsequent to the first time are output samples; add virtual error data to each process value of the input samples; and train a machine learning model that uses the input-output data set to which the virtual error data has been added, and that inputs the manipulated variables and process values ​​at the first time and outputs the process values ​​at the second time.

[0019] A generation program according to one aspect causes a computer to execute the following steps: acquire time-series data on manipulated variables for a process and process values ​​representing a state of the process; create an input-output dataset in which the manipulated variables and process values ​​at a first time are input samples and the process values ​​at a second time subsequent to the first time are output samples; add virtual error data to each process value of the input samples; and train a machine learning model that uses the input-output dataset to which the virtual error data has been added, and that inputs the manipulated variables and process values ​​at the first time and outputs the process values ​​at the second time. [Effects of the Invention]

[0020] According to one embodiment, it is possible to generate a data-driven simulator that can suppress error propagation. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a block diagram illustrating an example of a functional configuration of an information processing device. [Figure 2] FIG. 2 is a schematic diagram illustrating one aspect of the problem. [Figure 3] FIG. 3 is a diagram illustrating an example of error propagation. [Figure 4] FIG. 4 is a block diagram illustrating an example of the internal configuration of the simulator generation unit. [Figure 5] FIG. 5 is a schematic diagram showing an example of generating input and output data sets. [Figure 6] FIG. 6 is a diagram illustrating an example of the association of inputs and outputs. [Figure 7] FIG. 7 is a diagram showing an example of an input / output data set after adding an error. [Figure 8] FIG. 8 is a schematic diagram illustrating an example of training a dynamic prediction model. [Figure 9] FIG. 9 is a schematic diagram showing an example of the simulation. [Figure 10] FIG. 10 is a diagram illustrating an example of suppressing error propagation. [Figure 11] FIG. 11 is a flowchart showing the procedure of the overall process of the simulator generation unit. [Figure 12] FIG. 12 is a flowchart showing the procedure of the error addition process. [Figure 13] FIG. 13 is a diagram showing an example of a simulation result. [Figure 14] FIG. 14 is a diagram illustrating an example of a simulation result. [Figure 15] FIG. 15 is a block diagram showing an example of the functional configuration of an information processing device according to an application example. [Figure 16] FIG. 16 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, an information processing device, a generation method, and a generation program according to the present disclosure will be described with reference to the accompanying drawings. Note that the embodiment merely illustrates one example or aspect, and the structure, action, function, properties, characteristics, methods, uses, etc. according to the present disclosure are not limited by such an example.

[0023] <Embodiment 1> <Example of use case> Fig. 1 is a block diagram showing an example of the functional configuration of an information processing device. Fig. 1 shows, as an example, an information processing device 10 that provides a generation function for generating a dynamic prediction model used in a data-driven simulator, in particular, a dynamic prediction model that can suppress the propagation of errors during simulation execution.

[0024] The term "data-driven simulator" as used here may refer to a simulator that can simulate the state transition of a process, which is an example of a dynamic system, and is applicable to linear and nonlinear processes, as well as processes of one-input, one-output systems and multi-input, multi-output systems.

[0025] Furthermore, a "dynamic predictive model" refers to a machine learning model that predicts the next state of a process using the operation of the process and the previous state of the process as input. For example, in a dynamic predictive model, the dynamic input-output relationship of a process can be trained using machine learning techniques such as deep learning using operational data of the process in a production environment. An example of such a process state is the value of a process variable, or so-called process value.

[0026] In one embodiment, the information processing device 10 may be realized as a server that provides the above-mentioned generation function on-premise. In another embodiment, the information processing device 10 may be realized as a SaaS (Software as a Service) type application, thereby providing a service corresponding to the above-mentioned generation function as a cloud service.

[0027] A client terminal 30 may be connected to such an information processing device 10 via an arbitrary network NW. For example, the network NW may be realized by any technology, whether wired or wireless, such as an intranet, the Internet, or a low-power wireless communication standard for IoT (Internet of Things). Note that the network NW may not be a single network, but may be a network in which an intranet and the Internet are connected via a network device such as a gateway.

[0028] The client terminal 30 is a terminal device that receives the above-described generation function. This client terminal 30 may be used by all plant personnel, such as process engineers, operators, and workers. For example, the client terminal 30 may be realized by any computer, such as a personal computer, a smartphone, a tablet terminal, or a wearable terminal.

[0029] Although FIG. 1 shows an example in which the above generation is provided by a client-server system, this is merely an example, and the above generation function may also be provided as a standalone function.

[0030] <One aspect of the issue> As explained in the Background Art section above, it is unrealistic to completely suppress the occurrence of prediction errors in the dynamic prediction model. For example, even when performing the ensemble prediction, if the number of models or the number of reinforcement learning training sessions is insufficient, prediction accuracy and generalization performance will decrease.

[0031] In the current situation, the data-driven simulator repeats a process of recursively using the output of the dynamic prediction model at a previous time as the input at the next time, so that prediction errors propagate from one time to the next every time a prediction is made by the dynamic prediction model. As a result, it is difficult for the data-driven simulator to suppress the prediction error from increasing over time.

[0032] Fig. 2 is a schematic diagram illustrating one aspect of the problem. Fig. 2 illustrates, as a conventional example, a dynamic prediction model 2A that receives manipulated variables for a process and a previous state of the process as input and outputs the next state of the process, and a simulator 2 that uses the dynamic prediction model 2A.

[0033] This dynamic predictive model 2A is trained using machine learning techniques such as deep learning, using operational data of the process in a production environment as training data. For example, the parameters of the dynamic predictive model 2A are trained according to a deep learning algorithm, using the manipulated variable for the process and the previous state of the process as explanatory variables, and the next state of the process as the objective variable. This generates a trained dynamic predictive model 2A.

[0034] When such a dynamic prediction model 2A is applied to a production environment, i.e., in the inference phase, the problem of error propagation does not occur as long as the process state is observed in real time and the state at the next time is always predicted based on the actual measured values ​​up to the previous time.

[0035] However, when the simulator 2 executes a simulation using the dynamic prediction model 2A, it is separated from the real environment, and therefore there are cases where it is not possible to obtain actual measured values ​​of the process state as observation data to be input to the dynamic prediction model 2A.

[0036] Therefore, in order for the simulator 2 to repeatedly predict the state of the process, the predicted value of the process state output by the dynamic prediction model 2A as the next state at the previous time is recursively used as an input to the dynamic prediction model 2A at the subsequent time.

[0037] In this case, errors in the predicted value of the process state at the preceding time are also recursively used as inputs to the dynamic prediction model 2A at the subsequent time. As this recursion is repeated, the errors are propagated from one time to the next, resulting in the prediction error increasing over time.

[0038] The error propagation problem becomes particularly serious when the process to be controlled has multiple outputs and the predicted values ​​of multiple types of outputs are recursively used as the next input, or when there is a large delay in the process input / output and the predicted values ​​of the state several points in time prior must be used as the input.

[0039] Fig. 3 is a diagram showing an example of error propagation. For example, in Fig. 3, predicted values ​​of the process state output in a time series by simulator 2 according to the conventional example are plotted as dashed lines, while actual measured values ​​of the process state are plotted as solid lines as a reference. Note that the vertical axis of the graph shown in Fig. 3 indicates the process state, for example, the value of a process variable, and the horizontal axis of the graph shown in Fig. 3 indicates time.

[0040] As shown in Figure 3, at the start of the simulation, there is not much difference between the predicted and measured values, but it is clear that the error between the predicted and measured values ​​increases as time passes.

[0041] If reinforcement learning is performed on simulator 2, where error propagation occurs, and the manipulated variable is continuously changed to simulate state transitions, the prediction error will continue to widen as the state transitions, and there is a risk that the reinforcement learning will be adversely affected when the control policy is updated using the prediction value, which generates a large error, as feedback.

[0042] <Configuration of information processing device> Next, a functional configuration of an information processing device 10 having a generation function that solves the above-mentioned problems will be described. In Fig. 1, blocks related to the functions of the information processing device 10 are diagrammed.

[0043] As shown in FIG. 1, the information processing device 10 includes a simulator generation unit 11, a simulator 12, an AI control model generation unit 13, an AI control model 14, and a report generation unit 15.

[0044] The simulator generation unit 11 is a processing unit that generates the above-mentioned data-driven simulator as simulator 12. To generate such simulator 12, driving data in a real environment and input / output definition information of simulator 12 can be used. Hereinafter, driving data in a real environment may be referred to as "actual driving data." The input of this actual driving data and input / output definition information can be received from client terminal 30, for example.

[0045] The AI ​​control model generation unit 13 is a processing unit that generates the AI ​​control model 14. In one embodiment, the AI ​​control model generation unit 13 performs reinforcement learning on the simulator 12 generated by the simulator generation unit 11. For example, the AI ​​control model generation unit 13 inputs manipulated variables to the simulator 12 according to a control strategy determined by a reinforcement learning algorithm, for example, an input pattern of manipulated variables. The AI ​​control model generation unit 13 then updates the input pattern of manipulated variables based on a predicted value of the process state output from the simulator 12. For example, the update of the input pattern of manipulated variables is repeated until optimization of an objective function that calculates an evaluation value related to a goal, for example, effectiveness, efficiency, cost, etc., is achieved. The AI ​​control model 14 is generated, which holds the input pattern of manipulated variables obtained by such optimization as a control strategy.

[0046] The report generation unit 15 is a processing unit that runs a process control simulation on the simulator 12 based on the control measures of the AI ​​control model 14 and outputs the results of the AI ​​control as a report. One example of a destination for such a report is the client terminal 30. For example, the effectiveness of the AI ​​control can be confirmed by comparing the results of the AI ​​control with the results of control measures that have been used in process control in the past.

[0047] In this way, the simulator 12 according to this embodiment may be used to perform reinforcement learning to acquire an AI control model in a simulation, and further to perform a simulation of process control using an AI control model.

[0048] 1 shows an example in which information processing device 10 has three functional units: simulator generation unit 11, AI control model generation unit 13, and report generation unit 15. However, these three functional units do not necessarily have to be provided as a package. For example, AI control model generation unit 13 and report generation unit 15 may be realized as external devices, services, or applications.

[0049] <Internal structure of the simulator generation unit> Next, a description will be given of the internal configuration of the simulator generation unit 11. Fig. 4 is a block diagram showing an example of the internal configuration of the simulator generation unit 11. As shown in Fig. 4, the inputs and outputs of the simulator generation unit 11 are as follows.

[0050] Input 1: Actual operation data The actual operation data may be past time-series operation data (sensor values ​​indicating the state of the process, such as temperature and pressure, and manipulated variables for the process, such as valve opening) collected from a real environment. For example, in a field such as a plant, measuring devices such as sensors that measure the state of the process, e.g., temperature and pressure, and operating devices such as actuators that operate the process, e.g., valve opening and closing, are installed as field devices (not shown). The actual operation data may be time-series data of the state of the process collected in time series from such measuring devices or time-series data of the manipulated variables for the process collected in time series from operating devices. Note that the state of the process and the manipulated variables for the process do not have to be collected directly from the measuring devices or operating devices, but may be acquired from recorders or controllers to which the measuring devices or operating devices are connected.

[0051] Input 2: Input / output definition information for simulator 12 The input / output definition information may include "input tags," "output tags," and "delay time between input and output." For example, an "input tag" refers to the name or identifier of element data (e.g., a control valve) that changes the state of a process. An "output tag" refers to the name or identifier of element data (e.g., temperature, pressure, etc.) that represents the state of a process. Furthermore, the "delay time between input and output" refers to the time difference between when the input data changes and when a change appears in the output data.

[0052] Output 1: Dynamic prediction model (trained) A dynamic predictive model is a machine learning model that has the ability to dynamically predict the next state of a process to be controlled based on the previous state and operation.

[0053] As shown in FIG. 4, the simulator generation unit 11 includes a data acquisition unit 11A, a learning period selection unit 11B, a data set creation unit 11C, an error addition unit 11D, and a model training unit 11E.

[0054] The data acquisition unit 11A is a processing unit that acquires data used to train the dynamic prediction model. In one embodiment, the data acquisition unit 11A can acquire actual driving data and input / output definition information for the simulator 12 from the client terminal 30. In this case, the actual driving data does not necessarily have to be acquired as a single data file, but may be acquired as multiple data files. Then, the data acquisition unit 11A extracts, from the actual driving data, input data corresponding to input tags defined in the input / output definition information and output data corresponding to output tags. Hereinafter, when it is not necessary to distinguish between the input data and the output data, the input data and the output data may be referred to as "input / output data."

[0055] The learning period selection unit 11B is a processing unit that selects a learning period to be subjected to machine learning for training the dynamic prediction model from among all sections of the input / output data acquired by the data acquisition unit 11A.

[0056] As one example, the learning period selection unit 11B can accept a learning period specification by user definition via the client terminal 30. Alternatively, the learning period selection unit 11B may accept a learning period specification by system definition set by a developer who develops the generation function or an administrator of the information processing device 10. The learning period specification may be realized by selecting a start point and an end point, or by selecting a start point or an end point and a section length.

[0057] As another example, the learning period selection unit 11B can automatically select a learning period using machine learning technology. For example, the learning period selection unit 11B divides the input / output data for the entire interval into multiple intervals. Then, the learning period selection unit 11B clusters the segment data based on the feature values ​​of each of the segment data divided into multiple intervals, such as the mean, variance, and frequency characteristics. For example, such clustering can be performed according to an algorithm such as the K-means method. Then, the learning period selection unit 11B extracts the segment data belonging to each cluster at a predetermined ratio, for example, an equal ratio.

[0058] This prevents the training data from being biased toward the distribution of features corresponding to specific operating states of the plant. In other words, during the actual operation of a process in a real plant, stable periods are far more frequent than unstable periods. Therefore, the entire range of input / output data may inevitably have a population with a biased ratio of steady-state to non-low-state, such as 9:1. In this case, randomly extracting training data from the population can result in bias toward training data with steady-state characteristics. Therefore, by extracting segment data belonging to each cluster at an equal ratio, bias toward training data with steady-state characteristics can be prevented.

[0059] The data set creation unit 11C is a processing unit that creates an input / output data set from input / output data corresponding to the learning period selected by the learning period selection unit 11B. In one aspect, the data set creation unit 11C associates input samples and output samples included in the input / output data corresponding to the learning period based on the delay time between input and output defined in the input / output definition information.

[0060] That is, from the viewpoint of constructing a dynamic prediction model that dynamically predicts the next state from the previous state and operation while taking into account the delay time between the input and output of a process in a plant, the data set creation unit 11C generates an input / output data set based on the time series data of actual operation, in which the actual measured values ​​of the operation at multiple points in time corresponding to the delay time and the actual measured values ​​of the previous state at multiple points in time are used as input samples, and the actual measured values ​​of the state at the next point in time are used as output samples, and this data set is used as training data for the dynamic prediction model.

[0061] Fig. 5 is a schematic diagram showing an example of generating an input / output data set. Fig. 5 shows an example in which, as input / output definition information for the simulator 12, "U1" is defined as an input tag, "Y1" is defined as an output tag, and "2" is defined as a delay time between input and output.

[0062] 5, the input / output data corresponding to the input tag "U1" and the output tag "Y1" from the actual operation data is extracted by the data acquisition unit 11A. Furthermore, from the entire period of the input / output data corresponding to the input tag "U1" and the output tag "Y1", the input / output data corresponding to the learning period selected by the learning period selection unit 11B is extracted.

[0063] After the input / output data corresponding to the learning period is extracted in this manner, the dataset creation unit 11C creates an input / output dataset by performing the following data operations. That is, the dataset creation unit 11C extracts, from the input / output data corresponding to the learning period, entries that match the input tag "U1" of the input / output definition information in chronological order from the first time to the time just before the last time. This extracts an "operation" column that forms the input / output dataset TR. The dataset creation unit 11C also extracts, from the input / output data corresponding to the learning period, entries that have the output tag "Y1" of the input / output definition information in chronological order from the first time to the time just before the last time. This extracts a "previous state" column that forms the input / output dataset TR. Furthermore, the dataset creation unit 11C extracts, from the input / output data corresponding to the learning period, entries that match the output tag "Y1" in chronological order from the time shifted from the first time by the delay time between input and output defined in the input / output definition information to the last time. This extracts a "next state" column that forms the input / output dataset TR. As a result, the input / output dataset TR shown in FIG. 6 is created. Thereafter, the data set creation unit 11C associates the inputs and outputs included in the input / output data set TR based on the delay time between the inputs and outputs defined in the input / output definition information.

[0064] Fig. 6 is a diagram showing an example of the association of inputs and outputs. Fig. 6 shows an example in which an input corresponding to an output at time "3" is associated as one aspect, but it goes without saying that inputs corresponding to outputs at other times can also be associated in a similar manner. Note that in Fig. 6, as in Fig. 5, the delay time between inputs and outputs defined in the input / output definition information is assumed to be "2".

[0065] As shown in Figure 6, the output data record at time "3" is associated with the input data records up to two time periods earlier, which correspond to the delay time "2" between input and output. That is, the input samples are the actual measurement value of the operation at time "1," "0.5," and the actual measurement value of the previous state, "6.9," and the actual measurement value of the operation at time "2," "1.24," and the actual measurement value of the previous state, "7.2," at time "2," and the actual measurement value of the next state at time "3," "8.2," is the output sample (correct label), and the input and output samples are associated with each other. Hereinafter, a pair of associated input and output samples may be referred to as an input / output sample.

[0066] 6 shows an example in which two input data records for times "1" and "2", which correspond to a time difference of "2" between when the input changes and when the output changes, are associated with an output data record for time "3", but multiple input data records do not necessarily have to be associated with an output data record. For example, only one of the input data records for times "1" and "2" may be associated with the output data record for time "3".

[0067] Although the input / output data set TR is in a table format, this is merely an example, and the data structure is not limited to a relational database. For example, the data may be written in a tag format using a markup language such as XML (Extensible Markup Language), or may be written using commas and line breaks such as CSV (Comma-Separated Values).

[0068] The error addition unit 11D is a processing unit that adds an error to the process state of the input sample for each input / output sample included in the input / output dataset. In one embodiment, the error addition unit 11D adds, as virtual error data, random numbers generated according to a predetermined probability distribution, such as a normal distribution with a mean of 0 and a predetermined standard deviation, to the actual measured value of the "previous state" of the input sample. Hereinafter, to distinguish between the input / output dataset created by the dataset creation unit 11C and the input / output dataset in which error data is added to each input sample by the error addition unit 11D, the latter may be referred to as the "input / output dataset after error addition." In this case, the error addition unit 11D can generate multiple types of input / output datasets after error addition from a single input / output dataset. In this case, the error addition unit 11D can change random number generation parameters, such as the probability distribution and the random number seed, used to generate virtual error data for each input / output dataset after error addition.

[0069] More specifically, the error addition unit 11D reads the input / output data set created by the data set creation unit 11C. The error addition unit 11D then repeats the following process a number of times corresponding to the number M of error-added input / output data sets to be generated from one input / output data set. That is, the error addition unit 11D sets random number generation parameters to be used to generate the mth error data. The error addition unit 11D then repeats the process of generating virtual error data by generating random numbers according to a predetermined probability distribution and adding the error data to the previous state of each input sample until error data is added to all input samples included in the input / output data set. This generates the mth error-added input / output data set. The above-described process, i.e., setting the random number generation parameters and adding error data to the "previous state" of each input sample, is then repeated until the Mth error-added input / output data set is generated.

[0070] Fig. 7 is a diagram showing an example of input / output data sets after error addition. For example, Fig. 7 shows an excerpt of the first input / output data set TR1 and the second input / output data set TR2 after error addition from the M input / output data sets after error addition. As shown in Fig. 7, M input / output data sets after error addition can be generated from the single input / output data set TR shown in Fig. 6, including the input / output data set TR1 after error addition with added input / output data set TR1 having added thereto an error A generated by generating random numbers based on a random number generation parameter a, and the input / output data set TR2 after error addition with added input / output data set TR2 having added thereto an error B generated by generating random numbers based on a random number generation parameter b.

[0071] According to such an input-output data set after adding errors, the value of the "previous state" can be regarded not as an actual measurement value but as a predicted value with error that follows a probability distribution centered on the actual measurement value. Therefore, the input-output data set after adding errors can be an input-output data set that takes the actual measurement value of the operation on the process and the hypothetical predicted value with error of the previous state as input, and outputs the actual measurement value of the next state. Therefore, by performing machine learning by the model training unit 11E, the next state can be correctly predicted even if there is a prediction error in the previous state data.

[0072] The model training unit 11E is a processing unit that trains the dynamic prediction model. Hereinafter, in order to distinguish between the pre-trained dynamic prediction model and the trained dynamic prediction model, the former may be referred to as the “dynamic prediction model 12a” and the latter as the “dynamic prediction model 12A.”

[0073] FIG. 8 is a schematic diagram illustrating an example of training a dynamic prediction model. As shown in FIG. 8, the model training unit 11E can perform machine learning, such as deep learning, of the dynamic prediction model 12a using the M error-added input / output data sets TR1 to TRM generated by the error addition unit 11D as training data sets. Such a dynamic prediction model 12a may be realized by any machine learning model, such as a neural network or a deep neural network (DNN), a support vector machine, or gradient boosting. For example, the model training unit 11E uses the manipulated variable for the process and the previous state of the process as explanatory variables of the dynamic prediction model 12a, and the next state of the process as the objective variable of the dynamic prediction model 12a, and trains the parameters of the dynamic prediction model 12a according to a deep learning algorithm. This generates a trained dynamic prediction model 12A.

[0074] Here, an example has been given in which M error-added input / output data sets TR1 to TRM are used for machine learning of the dynamic prediction model 12a, but it is also possible to use an input / output data set created by the data set creation unit 11C. Also, here, an example has been given in which multiple data sets are used for machine learning, but it goes without saying that machine learning can be performed with at least one or more data sets.

[0075] The dynamic prediction model 12A generated in this manner is used as a simulator 12 in the implementation environment of the reinforcement learning. Fig. 9 is a schematic diagram showing an example of a simulation. Fig. 9 illustrates the dynamic prediction model 12A generated by the simulator generation unit 11 according to this embodiment and the simulator 12 that uses the dynamic prediction model 12A.

[0076] As shown in Figure 9, the simulator 12 queries the dynamic prediction model 12A using a predetermined operation and initial values ​​of state data, and recursively uses the predicted value of the next state output from the model 12A to simulate state transitions due to changes in the manipulated variable. Furthermore, the dynamic prediction model 12A is trained to accurately predict the next state regardless of whether there is an error in the previous state, thereby reducing the impact of errors in the predicted value of the previous state on the prediction of the next state. As a result, it is possible to suppress error propagation.

[0077] Fig. 10 is a diagram showing an example of suppressing error propagation. For example, in Fig. 10, predicted values ​​of the process state output in a time series by the simulator 12 according to this embodiment are plotted as dashed lines, while actual measured values ​​of the process state are plotted as solid lines as a reference. Note that the vertical axis of the graph shown in Fig. 10 indicates the process state, for example, the value of a process variable, and the horizontal axis of the graph shown in Fig. 10 indicates time.

[0078] As shown in Figure 10, there is little difference between the predicted and measured values ​​not only at the start of the simulation but also until the end of the simulation, which clearly shows that the propagation of errors between the predicted and measured values ​​has been suppressed throughout the entire simulation period.

[0079] <Processing flow> Next, a process flow of the information processing device 10 according to this embodiment will be described. Here, (1) Overall process executed by the simulator generating unit 11 will be described, followed by (2) Error addition process executed as a subroutine of the overall process.

[0080] (1) Overall processing Fig. 11 is a flowchart showing the procedure of the overall processing of the simulator generation unit 11. As shown in Fig. 11, the data acquisition unit 11A acquires actual operation data and input / output definition information of the simulator 12 from the client terminal 30 (step S101).

[0081] Next, the data acquiring unit 11A extracts input data corresponding to the input tags and output data corresponding to the output tags defined in the input / output definition information from the actual operation data acquired in step S101 (step S102).

[0082] Then, the learning period selection unit 11B selects a learning period to be subjected to machine learning for training the dynamic prediction model 12a (step S103).

[0083] Thereafter, the data set creation unit 11C creates an input / output data set from the input / output data corresponding to the learning period selected in step S103 among all sections of the input / output data extracted in step S102 (step S104).

[0084] Then, the error addition unit 11D executes an "error addition process" in which, for each input / output sample included in the input / output data set created in step S104, a random number generated according to a predetermined probability distribution is added as virtual error data to the actual measured value of the "previous state" of the input sample (step S105).

[0085] As a result of the error addition process executed in step S105, M input / output data sets TR1 to TRM after adding errors are generated.

[0086] Thereafter, the model training unit 11E performs machine learning, for example, deep learning, of the dynamic prediction model 12a using the M error-added input / output data sets TR1 to TRM generated as a result of step S105 as training data sets (step S106).

[0087] As a result of the machine learning performed in step S106, a trained dynamic prediction model 12A is generated.

[0088] (2) Error addition processing Fig. 12 is a flowchart showing the procedure of the error addition process. This process corresponds to the process of step S105 shown in Fig. 11. As shown in Fig. 12, the error addition unit 11D reads the input / output data set created in step S104 (step S301).

[0089] Then, the error addition unit 11D executes loop processing 1, which repeats the processing from step S302 to step S304 described below, the number of times corresponding to the number M of input / output data sets after error addition to be generated from one input / output data set. Note that the processing from step S302 to step S304 described below does not necessarily have to be executed repeatedly, and may be executed in parallel.

[0090] That is, the error adding unit 11D sets parameters for generating random numbers used to generate the m-th error data, such as a probability distribution and a random number seed (step S302).

[0091] Thereafter, the error addition unit 11D executes loop processing 2, which repeats the processing of the following step S303 and the processing of the following step S304 the number of times corresponding to the number N of input / output samples included in the input / output data set. Note that the processing of the following step S303 and the processing of the following step S304 do not necessarily have to be executed repeatedly, and may be executed in parallel.

[0092] That is, the error adding unit 11D generates virtual error data by generating random numbers using the random number generation parameters set in step S302 (step S303). Subsequently, the error adding unit 11D adds the error data generated in step S303 to the state before the input sample of the n-th input / output sample (step S304).

[0093] By repeating this loop process 2, error data is added to the previous states of the N input samples included in the input / output data set, resulting in the mth input / output data set after adding errors.Furthermore, by repeating loop process 1, M input / output data sets after adding errors are completed.

[0094] <Effects of the First Embodiment> As described above, the information processing device 10 according to this embodiment performs machine learning of the dynamic prediction model 12a using an input / output data set in which virtual error data is added to input samples of time-series data of input and output related to process control.

[0095] The dynamic prediction model 12A generated by such machine learning is trained to accurately predict the next state regardless of whether there is an error in the previous state. Therefore, when simulating state transitions as the simulator 12, the adverse effect of errors in the predicted value of the previous state on the prediction of the next state can be reduced.

[0096] Therefore, the information processing device 10 according to this embodiment can generate a data-driven simulator capable of suppressing error propagation.

[0097] Here, in order to explain the improvement effect of the dynamic prediction model 12A according to this embodiment, the simulation results are compared between a simulator 2 using a dynamic prediction model 2A according to a conventional example and a simulator 12 using the dynamic prediction model 12A according to this embodiment.

[0098] 13 and 14 are diagrams showing examples of simulation results. For example, in FIG. 13, predicted values ​​of the process state output in a time series by a simulator 2 using a dynamic prediction model 2A according to a conventional example are plotted as a dashed line. On the other hand, in FIG. 14, predicted values ​​of the process state output in a time series by a simulator 12 using a dynamic prediction model 12A according to this embodiment are plotted as a dashed line. Furthermore, in FIGS. 13 and 14, actual measured values ​​of the process state are plotted as a reference as a solid line. Note that the vertical axis of the graphs shown in FIGS. 13 and 14 indicates the process state, for example, the value of a process variable, and the horizontal axis of the graphs shown in FIGS. 13 and 14 indicates time.

[0099] For example, when a simulation is performed using a conventional dynamic prediction model 2A, errors in the predicted values ​​at the previous time point have a significant impact on the predicted values ​​at the next time point. As a result, as shown in Figure 13, it is clear that the prediction errors propagate and grow as the state transitions. With this conventional simulator 2, the agreement rate between the reference actual measurement values ​​and the simulation predicted values ​​is only 3%.

[0100] On the other hand, when a simulation is performed using the dynamic prediction model 12A according to this embodiment, the propagation of errors can be suppressed. Therefore, even if an error occurs temporarily during the simulation, the error does not tend to increase, as shown in FIG. 14. As described above, according to the simulator 12 according to this embodiment, the agreement rate between the reference actual measurement value and the simulation predicted value reaches 89%, which is a higher agreement rate than the conventional example.

[0101] Therefore, when learning an AI control strategy by repeating trial and error through reinforcement learning on the simulator 12 according to this embodiment, by suppressing error propagation, the adverse effect of prediction error on the reinforcement learning of the control strategy can be reduced, which can contribute to verifying the effectiveness of the AI ​​control simulation.

[0102] <Other embodiments> Although the embodiments of the present disclosure have been described above, various applications are possible, and the present disclosure may be implemented in various different forms other than the above-described embodiments.

[0103] <Simulator for different driving conditions> Processes in plants and other facilities are not necessarily controlled under the same operating conditions. For example, operating conditions in petrochemical processes are subject to change. Because different operating conditions can lead to different process input / output relationships, it can be difficult to represent processes under multiple operating conditions with a single dynamic predictive model. Therefore, it is possible to generate a data-driven simulator for each operating condition, taking into account the multiple operating conditions involved.

[0104] Fig. 15 is a block diagram showing an example of the functional configuration of an information processing device 20 according to an application example. In Fig. 15, blocks that perform the same functions as those of the information processing device 10 shown in Fig. 1 are assigned the same reference numerals.

[0105] As shown in FIG. 15, the information processing device 20 receives input of operating condition information (the period of each operating condition) from the client terminal 30 in addition to actual operation data and input / output definition information. For example, the operating condition information may correspond to the period during which the process was controlled under each operating condition. Based on such operating condition information, the data allocator 21 allocates segment data of the operating data for each operating condition, for each operating condition. The simulator generator 11 then executes the processes shown in FIGS. 11 and 12 using the segment data allocated to each operating condition. This generates a simulator corresponding to each operating condition. Furthermore, the AI ​​control model generator 13 may generate AI control models for each operating condition for a simulator for each operating condition. Alternatively, the AI ​​control model generator 13 may alternately learn from simulators for multiple operating conditions to generate a single AI control model capable of handling multiple operating conditions. Furthermore, the report generation unit 15 can simulate AI control on a simulator for each operating condition based on an AI control model for each operating condition or an AI control model that can handle multiple operating conditions, and verify the effectiveness of AI control under multiple operating conditions.

[0106] <Numbers, etc.> The matters described in the above embodiment, such as the number of client terminals 30, the number of input / output data sets after adding errors, and specific examples of parameters used for random number generation, are merely examples and can be changed. Also, the order of processing in the flowcharts described in the embodiment can be changed within a consistent range.

[0107] <System> The information including the processing procedures, control procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, any one or more of the functional units of the simulator generation unit 11, the AI ​​control model generation unit 13, and the report generation unit 15 of the information processing device 10 may be configured as separate devices.

[0108] Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown. In other words, all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Note that each configuration may also be a physical configuration.

[0109] Furthermore, each processing function performed by each device can be realized, in whole or in part, by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0110] <Hardware> Next, an example of the hardware configuration of the computer described in the above embodiment will be described. Fig. 16 is a diagram showing an example of the hardware configuration. As shown in Fig. 16, an information processing device 10 has a communication device 10a, a storage device 10b, a memory 10c, and a processor 10d. Note that the components shown in Fig. 16 may be connected to each other via a bus or the like.

[0111] The communication device 10a is a network interface card, etc. The storage device 10b is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). For example, the storage device 10b stores programs and databases that operate the functions shown in FIGS. 1 and 4.

[0112] The processor 10d reads out a program that executes the same processing as the processing unit shown in FIG. 1 from the storage device 10b or the like and loads it into the memory 10c, thereby operating a process that executes the functions described in FIG.

[0113] Such a process realizes the same functions as the processing units of the information processing device 10. For example, the processor 10d reads from the storage device 10b or the like a program having the same functions as the simulator generation unit 11, the AI ​​control model generation unit 13, the report generation unit 15, etc. Then, the processor 10d executes a process that executes the same processing as the simulator generation unit 11, the AI ​​control model generation unit 13, the report generation unit 15, etc.

[0114] In this way, the information processing device 10 operates as an information processing device that executes a calculation method by reading and executing a program. The information processing device 10 can also realize functions similar to those of the above-described embodiment by reading the program from a recording medium using a medium reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the information processing device 10. For example, the present invention can also be applied in the same way to cases where another computer or server executes the program, or where these execute the program in cooperation with each other.

[0115] The above program can be distributed via a network such as the Internet. The above program can also be recorded on any recording medium and executed by a computer by reading it from the recording medium. For example, the recording medium can be a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), a digital versatile disk (DVD), or the like.

[0116] <Other> Some examples of combinations of the disclosed technical features are set out below.

[0117] (1) an acquisition unit that acquires time-series data relating to a manipulated variable for a process and a process value that represents a state of the process; a creation unit that creates an input / output data set in which a manipulated variable and a process value at a first time are input samples and a process value at a second time subsequent to the first time are output samples; an adding unit that adds virtual error data to each process value of the input sample; a training unit that uses the input / output data set to which the virtual error data has been added to train a machine learning model that receives an operation amount and a process value at the first time as input and outputs a process value at the second time; An information processing device comprising:

[0118] (2) The information processing device according to (1), wherein the adding unit includes a process for generating the virtual error data by generating random numbers according to a predetermined probability distribution.

[0119] (3) The information processing device according to (2), wherein the probability distribution is a normal distribution.

[0120] (4) The information processing device according to (3), wherein the normal distribution has a mean of 0 and a standard deviation of a predetermined value.

[0121] (5) The information processing device described in any one of (1) to (4), characterized in that the creation unit includes a process of setting the first time and the second time based on the time difference between when the manipulated variable changes and when a change in the process value appears.

[0122] (6) An extraction unit that clusters the segment data based on feature amounts of each of the segment data obtained by dividing the time series data into a plurality of sections, and extracts segment data belonging to each cluster at a predetermined ratio; The information processing device according to any one of (1) to (4), wherein the creation unit includes a process of creating an input / output data set using the segment data extracted by the extraction unit.

[0123] (7) A distribution unit distributes segment data of the time-series data for each operating condition based on operating condition information in which a period during which the process was controlled under each operating condition is associated with each operating condition of the process, The information processing device according to any one of (1) to (4), wherein the creation unit, the addition unit, and the training unit perform processing using segment data assigned to each of the driving conditions.

[0124] (8) The information processing device according to any one of (1) to (4), wherein the machine learning model is realized by a neural network.

[0125] (9) acquiring time-series data relating to a manipulated variable for a process and a process value representing a state of the process; creating an input / output data set having a manipulated variable and a process value at a first time as input samples and a process value at a second time subsequent to the first time as output samples; adding virtual error data to each process value of the input sample; training a machine learning model that receives as input the manipulated variable and the process value at the first time and outputs the process value at the second time, using the input / output data set to which the virtual error data has been added; A generating method characterized in that the processing is carried out by a computer.

[0126] (10) acquiring time-series data relating to a manipulated variable for a process and a process value representing a state of the process; creating an input / output data set having a manipulated variable and a process value at a first time as input samples and a process value at a second time subsequent to the first time as output samples; adding virtual error data to each process value of the input sample; training a machine learning model that receives as input the manipulated variable and the process value at the first time and outputs the process value at the second time, using the input / output data set to which the virtual error data has been added; A generating program that causes a computer to execute a process. [Explanation of symbols]

[0127] 10. Information processing equipment 11 Simulator generation unit 11A Data acquisition section 11B Study Period Selection Section 11C Dataset Creation Department 11D Error addition section 11E Model Training Department 12A Dynamic Prediction Model 12 Simulator 13 AI control model generation unit 14 AI Control Model 15 Report Generation 30 client terminals

Claims

1. an acquisition unit that acquires time-series data relating to a manipulated variable for a process and a process value that represents a state of the process; a creating unit that creates an input / output data set in which a manipulated variable and a process value at a first time are input samples and a process value at a second time subsequent to the first time are output samples; an adding unit that adds virtual error data to each process value of the input sample; a training unit that uses the input / output data set to which the virtual error data has been added to train a machine learning model that receives an operation amount and a process value at the first time as input and outputs a process value at the second time; An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, wherein the adding unit generates the virtual error data by generating random numbers according to a predetermined probability distribution.

3. 3. The information processing apparatus according to claim 2, wherein the probability distribution is a normal distribution.

4. 4. The information processing apparatus according to claim 3, wherein the normal distribution has a mean of 0 and a standard deviation of a predetermined value.

5. 2. The information processing apparatus according to claim 1, wherein the creation unit includes a process for setting the first time and the second time based on a time difference between when the manipulated variable changes and when a change in the process value appears.

6. The time-series data is divided into a plurality of segments, and the segment data is clustered based on the feature amount of each segment data, and segment data belonging to each cluster is extracted at a predetermined ratio.

2. The information processing apparatus according to claim 1, wherein the creation unit includes a process for creating an input / output data set using the segment data extracted by the extraction unit.

7. a distribution unit that distributes, for each operating condition, segment data of the time-series data for a period corresponding to the operating condition, based on operating condition information in which a period during which the process was controlled under the operating condition is associated with each operating condition of the process; The information processing device according to claim 1 , wherein the creating unit, the adding unit, and the training unit execute processing using segment data assigned to each of the driving conditions.

8. The information processing device according to claim 1 , wherein the machine learning model is realized by a neural network.

9. acquiring time-series data relating to a manipulated variable for a process and a process value representing a state of the process; creating an input / output data set having manipulated variables and process values ​​at a first time as input samples and process values ​​at a second time subsequent to the first time as output samples; adding virtual error data to each process value of the input sample; training a machine learning model that receives an input of an manipulated variable and a process value at the first time and outputs a process value at the second time, using the input / output data set to which the virtual error data has been added; A generating method characterized in that the processing is carried out by a computer.

10. acquiring time-series data relating to a manipulated variable for a process and a process value representing a state of the process; creating an input / output data set having manipulated variables and process values ​​at a first time as input samples and process values ​​at a second time subsequent to the first time as output samples; adding virtual error data to each process value of the input sample; training a machine learning model that receives an input of an manipulated variable and a process value at the first time and outputs a process value at the second time, using the input / output data set to which the virtual error data has been added; A generating program that causes a computer to execute a process.

Images