Information processing apparatus, generation method, and computer-readable recording medium

Abstract

The present invention relates to an information processing device, a generation method, and a computer-readable recording medium for generating a data-driven simulator capable of suppressing error transmission. The information processing apparatus includes: an acquisition unit that acquires time-series data relating to an operation amount for a process and a process value indicating a state of the process; a creation unit that creates an input / output data set in which an operation amount and a process value at a first time point are set as input samples and a process value at a second time point subsequent to the first time point is set as output samples; an addition unit that adds virtual error data for each process value of the input sample; and a training unit that, using the input / output data set to which the virtual error data is added, trains a machine learning model that outputs the process value at the second time using the operation amount at the first time and the process value as inputs.

Description

Technical Field

[0001] The present invention relates to an information processing device, a generation method, and a computer-readable recording medium. Background Art

[0002] In fields such as workshop process control, an AI (Artificial Intelligence) control method has been proposed to obtain the optimal control strategy through reinforcement learning.

[0003] Reinforcement learning requires repeated trial and error to find a strategy for the control object, so there are aspects that are not realistic in formal environments such as petrochemical plants where safety is of paramount importance.

[0004] According to this aspect, as a result of executing reinforcement learning on a simulator that simulates the process of a formal environment, the AI control strategy whose effectiveness has been verified through simulation is often transferred to the actual environment.

[0005] As an example of a simulator that implements reinforcement learning, there is a simulator that uses a physical model constructed based on the physical characteristics of a control object.

[0006] Simulators that utilize physical models rigorously define the physical properties of the controlled object, leading to expectations of improved simulator accuracy. However, defining the physical properties of the controlled object requires a high level of specialized knowledge, resulting in a high degree of reliance on individual skills and a significant amount of effort involved in constructing the physical model.

[0007] As another example, a statistical model representing the dynamic input-output relationship of a process may be constructed based on operational data of a process in a formal environment, and the constructed statistical model may be used for simulation.

[0008] When a process is linear, such as a single-input, single-output system, a system identification method, such as the ARX (Auto-Regressive with eXogenou) model, is used to estimate the coefficients of each variable from a mathematical formula that defines the relationship between the input and output. However, when a process is nonlinear, such as a multi-input, multi-output system, constructing a model based on the mathematical formula is difficult.

[0009] As another example, process operation data is sometimes used to train a dynamic prediction model using machine learning techniques such as deep learning. The model uses the process operation and the previous state as input to predict the next state, and then uses this model for simulation.

[0010] Models generated through deep learning are known to be highly useful even for processes with strong nonlinearities. Furthermore, even without domain knowledge of the process being controlled, models can be easily generated through deep learning by preparing process operating data (input and output data) as training data. This enables the realization of a data-driven simulator suitable for both linear and nonlinear processes, as well as single-input, single-output systems and multi-input, multi-output systems.

[0011] However, the prediction accuracy of the above dynamic prediction model depends on the quantity and quality of training data, so there may be areas where the prediction is accurate and areas where the prediction is inaccurate.

[0012] Based on this aspect, a method has been proposed in which a plurality of models are prepared for collective prediction so that unreliable prediction results have as little impact on the control strategy of reinforcement learning as possible.

[0013] Patent Document 1: Japanese Patent Application Laid-Open No. 2022-25859

[0014] Patent Document 2: Japanese Patent Application Laid-Open No. 2023-48962 Summary of the Invention

[0015] However, it is not realistic to completely suppress prediction errors in the dynamic prediction model described above. For example, even when performing ensemble prediction, if the number of models or the number of reinforcement learning cycles is insufficient, prediction accuracy and general performance will decline.

[0016] Under these circumstances, data-driven simulators repeatedly recursively use the output of the dynamic prediction model as input at the next moment. Consequently, prediction errors are continuously propagated downwards each time predictions are executed using the dynamic prediction model. Consequently, it is difficult to prevent prediction errors from increasing over time with data-driven simulators.

[0017] An object of the present invention is to realize the creation of a data-driven simulator capable of suppressing error propagation.

[0018] An information processing device according to one aspect comprises: an acquisition unit that acquires time series data related to an operation amount for a process and a process value representing a state of the process; a creation unit that creates an input-output data set in which the operation amount and process value at a first moment are set as input samples and the process value at a second moment subsequent to the first moment is set as output samples; an addition unit that adds virtual error data to each process value of the input sample; and a training unit that uses the input-output data set to which the virtual error data is added to train a machine learning model that takes the operation amount and process value at the first moment as input and outputs the process value at the second moment.

[0019] Regarding a generation method involved in one aspect, a computer performs the following processing: obtaining time series data related to the operation quantity for a process and the process value representing the state of the process; creating an input-output data set with the operation quantity and process value at the first moment as input samples and the process value at the second moment subsequent to the first moment as output samples; appending virtual error data to each process value of the input sample; and using the input-output data set to which the virtual error data is appended, training a machine learning model that takes the operation quantity and process value at the first moment as input and outputs the process value at the second moment.

[0020] One aspect involves a computer-readable recording medium having recorded thereon a generation program, which causes a computer to perform the following processing: obtaining time series data related to an operation amount for a process and a process value representing a state of the process; creating an input-output data set in which the operation amount and process value at a first moment are set as input samples and the process value at a second moment subsequent to the first moment is set as output samples; appending virtual error data to each process value of the input sample; and using the input-output data set to which the virtual error data is appended, training a machine learning model in which the operation amount and process value at the first moment are set as input and the process value at the second moment is output.

[0021] Effects of the Invention

[0022] According to one embodiment, it is possible to generate a data-driven simulator that can suppress error propagation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a block diagram showing an example of the functional configuration of an information processing device.

[0024] Figure 2 This is a schematic diagram showing one aspect of the subject.

[0025] Figure 3 This is a diagram showing an example of error propagation.

[0026] Figure 4 This is a block diagram showing an example of the internal structure of a simulator generation unit.

[0027] Figure 5 This is a schematic diagram showing an example of generating input and output data sets.

[0028] Figure 6 This is a diagram showing an example of establishing a relationship between input and output.

[0029] Figure 7This figure shows an example of an input-output data set after error addition.

[0030] Figure 8 is a schematic diagram showing a training example of a dynamic prediction model.

[0031] Figure 9 This is a schematic diagram showing an example of a simulation.

[0032] Figure 10 This is a diagram showing an example of suppressing error propagation.

[0033] Figure 11 This is a flowchart showing the overall processing flow of the simulator generation unit.

[0034] Figure 12 This is a flowchart showing the flow of error addition processing.

[0035] Figure 13 This is a diagram showing an example of simulation results.

[0036] Figure 14 This is a diagram showing an example of simulation results.

[0037] Figure 15 This is a block diagram showing a functional configuration example of an information processing device according to an application example.

[0038] Figure 16 This is a diagram showing an example of a hardware configuration. DETAILED DESCRIPTION

[0039] The following describes a method for implementing the information processing device, generation method, and generation program according to the present invention (hereinafter referred to as an "embodiment") with reference to the accompanying drawings. This embodiment merely illustrates one example or aspect, and the structure, action, function, nature, characteristics, method, and application of the present invention are not limited by this example.

[0040] <Implementation Method 1>

[0041] <Application Examples>

[0042] Figure 1 This is a block diagram showing an example of the functional structure of an information processing device. Figure 1 , an information processing device 10 is shown as an example of a dynamic prediction model used in a data-driven simulator, and in particular, provides a function of generating a dynamic prediction model capable of suppressing the propagation of errors during simulation execution.

[0043] The "data-driven simulator" mentioned here can refer to an example of a dynamic system, that is, a simulator that simulates the state transition of a process and can also adapt to linear and nonlinear processes, as well as single-input single-output systems and multi-input multi-output systems.

[0044] Furthermore, a "dynamic prediction model" refers to a machine learning model that uses the operation of a process and the previous state of the process as input to predict the next state. For example, a dynamic prediction model can use operational data from a process in a real environment to train the dynamic input-output relationship of the process using machine learning techniques such as deep learning. An example of such a process state is the value of a process variable, or process value.

[0045] In one embodiment, the information processing device 10 can be implemented as a server that provides the above-mentioned generation function to an internal company environment. In another embodiment, the information processing device 10 can be implemented as a SaaS (Software as a Service) type application, thereby also providing services corresponding to the above-mentioned generation function as a cloud service.

[0046] The client terminal 30 can connect to the information processing device 10 via any network NW. For example, whether wired or wireless, the network NW can be implemented using any technology, such as a local area network (LAN), the Internet, or a power-saving wireless communication standard for the IoT (Internet of Things). Furthermore, the network NW is not a single network; a local area network and the Internet can be connected via a network device such as a gateway.

[0047] The client terminal 30 is a terminal device that receives the aforementioned generation function. This client terminal 30 can be used by all stakeholders in the plant, such as process engineers, operators, and workers. For example, the client terminal 30 can be implemented as any computer, including a personal computer, a smartphone, a tablet computer, a wearable device, and so on.

[0048] In addition, Figure 1 The example in which the above generation is provided by a client-server system is given, but this is only an example, and the above generation function may be provided by an independent system.

[0049] <One aspect of the topic>

[0050] As explained in the Background Art section above, it is not realistic to completely suppress prediction errors in the dynamic prediction model. For example, even when performing ensemble prediction, if the number of models and the number of reinforcement learning cycles are insufficient, prediction accuracy and general performance will decline.

[0051] Under these circumstances, data-driven simulators repeatedly recursively use the output of the dynamic prediction model at the previous moment as the input at the next moment. This causes prediction errors to continue to propagate each time predictions are executed using the dynamic prediction model. Consequently, data-driven simulators have difficulty preventing prediction errors from increasing over time.

[0052] Figure 2 This is a schematic diagram showing one aspect of the subject. Figure 2 1 and 2 show, as conventional examples, a dynamic prediction model 2A that takes an operation amount for a process and a previous state of the process as input and outputs a next state of the process, and a simulator 2 using the dynamic prediction model 2A.

[0053] This dynamic prediction model 2A uses operational data from processes in a production environment as training data and is trained using machine learning techniques such as deep learning. For example, the process's operational data and the previous state are used as explanatory variables, and the next state is used as the target variable. The parameters of the dynamic prediction model 2A are trained using a deep learning algorithm. This results in a trained dynamic prediction model 2A.

[0054] When this dynamic prediction model 2A is applied to a formal environment, that is, in the inference stage, the state of the process is observed in real time. As long as the state at the next moment is always predicted based on the measured values up to the previous moment, there will be no error transmission problem.

[0055] However, when the simulator 2 performs simulation using the dynamic prediction model 2A, it is disconnected from the actual environment, and therefore sometimes cannot obtain actual measured values of the state of the process as input to the dynamic prediction model 2A as observation data.

[0056] Therefore, in order for the simulator 2 to repeatedly and continuously predict the state of the process, the dynamic prediction model 2A is recursively inputted with the predicted value of the state of the process output as the next state at the previous time as input to the dynamic prediction model 2A at the subsequent time.

[0057] In this case, the error in the predicted value of the process state at the previous time is also recursively input into the dynamic prediction model 2A at the subsequent time. As a result of this recursion being repeated and the error being passed down, the prediction error increases over time.

[0058] In particular, the error transmission problem becomes more serious when the process of the control object has multiple outputs so that the predicted values of multiple outputs are recursively used as the next input. In addition, when there is a large lag between the input and output of the process and the predicted values of the state multiple time points ago need to be used as input.

[0059] Figure 3 is a diagram showing an example of error propagation. For example, Figure 3 In FIG, the predicted values of the state of the process outputted in time series by the simulator 2 according to the conventional example are drawn by the dotted line, while the actual measured values of the state of the process are drawn by the solid line for reference. Figure 3 The vertical axis of the graph shown represents the state of the process, for example the value of a process variable. Figure 3 The horizontal axis of the graph shown represents time.

[0060] like Figure 3 As shown in FIG. 1 , there is no significant difference between the predicted value and the measured value at the start of the simulation, but it can be said that the error between the predicted value and the measured value increases with the passage of time.

[0061] When reinforcement learning is implemented on the simulator 2 that generates error transmission in this way, if the operation amount is continuously changed to perform the simulation of state migration, the prediction error will continue to expand with the state migration, which may have an adverse effect on the reinforcement learning when the control strategy is updated by feeding back the predicted value that generates a large error.

[0062] <Structure of Information Processing Device>

[0063] Next, the functional structure of the information processing device 10 having a generation function for solving the above-mentioned problems will be described. Figure 1 Schematically shows modules related to the functions of the information processing device 10.

[0064] like Figure 1 As shown, the information processing device 10 includes a simulator generation unit 11 , a simulator 12 , an AI control model generation unit 13 , an AI control model 14 , and a report generation unit 15 .

[0065] The simulator generator 11 is a processing unit that generates the aforementioned data-driven simulator as the simulator 12. This simulator 12 can be generated using actual environment operating data and the simulator's 12 input / output definition information. Below, the actual environment operating data may be referred to as "actual operating data." For example, the actual operating data and input / output definition information can be input from the client terminal 30.

[0066] The AI control model generation unit 13 is a processing unit that generates the AI control model 14. As one embodiment, the AI control model generation unit 13 performs reinforcement learning on the simulator 12 generated by the simulator generation unit 11. For example, the AI control model generation unit 13 inputs the operation amount to the simulator 12 according to the control strategy determined by the reinforcement learning algorithm, such as the input pattern of the operation amount. Moreover, the AI control model generation unit 13 updates the input pattern of the operation amount based on the predicted value of the state of the process output from the simulator 12. For example, the input pattern of the operation amount is repeatedly updated until the optimization of the objective function for calculating the evaluation value related to the target, such as effect, efficiency, cost, etc. is achieved. The AI control model 14 is generated and the input pattern of the operation amount obtained by this optimization is saved as a control strategy.

[0067] The report generator 15 is a processing unit that executes a process control simulation based on the control strategy of the AI control model 14 on the simulator 12 and outputs the results of the AI control as a report. An example of a destination for such a report is the client terminal 30. For example, the effectiveness of the AI control can be confirmed by comparing the results of the AI control with the results of a control strategy that has previously achieved successful process control implementation.

[0068] In this way, the simulator 12 according to this embodiment can be used to perform reinforcement learning to obtain an AI control model during simulation and to perform simulation of process control based on the AI control model.

[0069] In addition, Figure 1 The example in the figure 1 illustrates an information processing device 10 having three functional units: a simulator generator 11, an AI control model generator 13, and a report generator 15. However, these three functional units can also be provided separately. For example, the AI control model generator 13 and the report generator 15 do not interfere with the implementation of external devices, services, or applications.

[0070] Internal structure of the simulator generation unit

[0071] Next, the internal structure of the simulator generation unit 11 will be described. Figure 4 1 is a block diagram showing an example of the internal structure of the simulator generation unit 11. Figure 4 As shown, the input and output of the simulator generation unit 11 are as follows.

[0072] ○Input 1: Actual operation data

[0073] The actual operation data can be the past time series operation data collected from the actual environment (sensor values such as temperature and pressure that represent the state of the process, and operation quantities for the process such as the opening of the valve). For example, at a site such as a workshop, measuring instruments such as sensors that measure the state of the process, such as temperature and pressure, and operating instruments such as actuators that perform operations for the process, such as the opening and closing of valves, are set as field instruments (not shown). It can be time series data of the state of the process collected in time series from such measuring instruments, and time series data of the operation quantities for the process collected in time series from operating instruments. In addition, the state of the process and the operation quantities for the process may not be collected directly from the measuring instruments and operating instruments, but may also be obtained from recorders and controllers connected to the measuring instruments and operating instruments.

[0074] ○ Input 2: Input and output definition information of simulator 12

[0075] Input / output definition information can include "input labels," "output labels," and "lag time between input and output." For example, an "input label" refers to the name or identifier of the data element that changes the state of a process (e.g., a control valve). Furthermore, an "output label" refers to the name or identifier of the data element that indicates the state of a process (e.g., temperature, pressure, etc.). Furthermore, the "lag time between input and output" refers to the time difference between a change in input data and a change in output data.

[0076] ○ Output 1: Dynamic prediction model (training completed)

[0077] The dynamic prediction model is a machine learning model that has the function of dynamically predicting the previous state and the next state after operation of a process that is a control target.

[0078] like Figure 4 As shown, the simulator generation unit 11 includes a data acquisition unit 11A, a learning period selection unit 11B, a data set creation unit 11C, an error addition unit 11D, and a model training unit 11E.

[0079] The data acquisition unit 11A is a processing unit that acquires data used to train the dynamic prediction model. As one embodiment, the data acquisition unit 11A can acquire actual operating data and the input / output definition information of the simulator 12 from the client terminal 30. In this case, the actual operating data can be acquired not as a single data file but as multiple data files. Based on this, the data acquisition unit 11A extracts input data corresponding to input labels defined by the input / output definition information and output data corresponding to output labels from the actual operating data. Below, when input data and output data do not need to be distinguished, they may sometimes be referred to as "input / output data."

[0080] The learning period selection unit 11B is a processing unit that selects a learning period to be used as a target of machine learning for training the dynamic prediction model, from the entire interval of the input and output data acquired by the data acquisition unit 11A.

[0081] As one example, the learning period selection unit 11B can accept the designation of a learning period through user definition via the client terminal 30. Alternatively, the learning period selection unit 11B can accept the designation of a learning period through a system definition set by a developer who develops the aforementioned generation function, an administrator of the information processing device 10, or the like. The designation of the learning period can be achieved by selecting a start point and an end point, or by selecting a start point or an end point and a section length.

[0082] As another example, the learning period selection unit 11B can also automatically select a learning period using machine learning techniques. For example, the learning period selection unit 11B divides the input and output data of the entire interval into multiple intervals. Furthermore, the learning period selection unit 11B clusters the segmented data based on various characteristic quantities of the segmented data, such as mean, variance, and frequency characteristics. As an example, this clustering can be performed using an algorithm such as the K-means method. Based on this, the learning period selection unit 11B extracts the segmented data belonging to each cluster at a predetermined ratio, such as an equal ratio.

[0083] This makes it possible to suppress the distribution of training data that is biased toward characteristic quantities corresponding to a specific operating state of a plant. Specifically, in the actual operation of processes in an actual plant, periods of stability are much longer than periods of instability. Consequently, there are cases where the entire range of input and output data inevitably becomes a population with a ratio of 9 to 1 between stable and unstable states. In this case, when training data is randomly extracted from the population, there is an aspect that favors training data with characteristics of a stable state. Based on this aspect, by extracting fragmented data belonging to each cluster at an equal ratio, it is possible to suppress the bias toward training data with characteristics of a stable state.

[0084] Dataset creation unit 11C is a processing unit that creates an input / output dataset based on the input / output data corresponding to the learning period selected by learning period selection unit 11B. In one embodiment, dataset creation unit 11C associates input samples and output samples included in the input / output data corresponding to the learning period based on the lag time between inputs and outputs defined in the input / output definition information.

[0085] That is, from the perspective of constructing a dynamic prediction model that dynamically predicts the next state based on the previous state and operation by considering the lag time between the input and output of the workshop process, the data set creation unit 11C generates an input-output data set based on the time series data of actual operation, which uses the measured values of the operation at multiple time points of the lag time and the measured values of the state before multiple time points as input samples, and uses the measured value of the state at the next time point as output samples, and sets it as training data for the dynamic prediction model.

[0086] Figure 5 This is a diagram showing an example of generating input and output data sets. Figure 5 , as the input / output definition information of the simulator 12 , an example is shown in which “U1” is defined as an input label, “Y1” is defined as an output label, and “2” is defined as a lag time between input and output.

[0087] like Figure 5 As shown, the data acquisition unit 11A extracts the input and output data corresponding to the input label "U1" and the output label "Y1" from the actual operating data. Furthermore, the input and output data corresponding to the learning period selected by the learning period selection unit 11B is extracted from the entire period of the input and output data corresponding to the input label "U1" and the output label "Y1."

[0088] After the input and output data corresponding to the learning period are extracted in this way, the data set creation unit 11C creates an input and output data set through the following data operation. That is, the data set creation unit 11C extracts the entries that match the input label "U1" of the input and output definition information from the input and output data corresponding to the learning period in a time series from the initial moment until one moment before the last moment. Thus, a column of "operation" forming the input and output data set TR is extracted. In addition, the data set creation unit 11C extracts the entries that match the output label "Y1" set as the input and output definition information from the input and output data corresponding to the learning period in a time series from the initial moment until one moment before the last moment. Thus, a column of "previous state" forming the input and output data set TR is extracted. Furthermore, the data set creation unit 11C extracts the lag time between the input and output of the entries that match the output label "Y1" defined in the input and output definition information from the moment offset from the initial moment until the last moment in a time series. Thus, a column of "next state" forming the input and output data set TR is extracted. As a result, Figure 6 Then, the dataset creation unit 11C associates the inputs and outputs included in the input-output dataset TR based on the lag time between the inputs and outputs defined in the input-output definition information.

[0089] Figure 6 This is a diagram showing an example of establishing a relationship between input and output. Figure 6 As one aspect, the example of establishing association with the input corresponding to the output at time "3" is shown, but it is of course possible to establish association with the input corresponding to the output at other times in the same manner. Figure 6 In, with Figure 5 Similarly, the lag time between inputs and outputs defined in the input / output definition information is set to "2".

[0090] like Figure 6 As shown, the records in the input data up to two moments prior to the lag time "2" between the input and output are associated with the record at time "3" in the output data. Specifically, the measured value "0.5" of the operation at time "1" and the measured value "6.9" of the previous state, and the measured value "1.24" of the operation at time "2" and the measured value "7.2" of the previous state are used as input samples, and the measured value "8.2" of the next state at time "3" is used as the output sample (correct label), thereby associating the input and output samples. Below, pairs of input and output samples that are associated with each other may sometimes be referred to as input and output samples.

[0091] also, Figure 6 In the example above, the input data records at time points "1" and "2," corresponding to the time difference "2" between the input change and the output change, are associated with the output data record at time point "3." However, it is not necessary to associate multiple input data records with the output data record. For example, only one of the input data records at time points "1" and "2" can be associated with the output data record at time point "3."

[0092] While the example of the input / output dataset TR being in table format is shown, this is merely an example, and the data structure is not limited to that of a relational database. For example, data may be expressed in a tag format using a markup language such as XML (Extensible Markup Language), or data may be expressed using commas and line breaks, such as in CSV (Comma-Separated Values).

[0093] The error adding unit 11D is a processing unit that adds an error to the state of the input sample process for each input / output sample included in the input / output dataset. As one embodiment, the error adding unit 11D adds a random number generated according to a predetermined probability distribution, such as a normal distribution with a mean of 0 and a predetermined standard deviation, as virtual error data to the actual measured value of the "previous state" of the input sample. Below, to distinguish between the input / output dataset created by the dataset creation unit 11C and the input / output dataset to which the error adding unit 11D has added error data for each input sample, the latter will sometimes be referred to as the "input / output dataset after error addition." In this case, the error adding unit 11D can generate multiple input / output datasets after error addition from a single input / output dataset. In this case, the error adding unit 11D can change the parameters for generating random numbers used to generate virtual error data, such as the probability distribution and random number seed, for each input / output dataset after error addition.

[0094] More specifically, the error adding unit 11D reads the input / output dataset created by the dataset creating unit 11C. Furthermore, the error adding unit 11D repeatedly performs the following process a number of times corresponding to the number M of error-added input / output datasets to be generated based on one input / output dataset. Specifically, the error adding unit 11D sets parameters for generating random numbers used to generate the mth error data. Then, until error data has been added to all input samples included in the input / output dataset, the error adding unit 11D generates virtual error data by generating random numbers based on a predetermined probability distribution, and repeatedly performs the process of adding this error data to the previous state of the process in the input sample. This generates the mth error-added input / output dataset. The above-described process, i.e., setting parameters for generating random numbers and adding error data for the "previous state" of each input sample, is repeated until the Mth error-added input / output dataset is generated.

[0095] Figure 7 is a diagram showing an example of an input-output data set after error addition. For example, Figure 7 FIG. 1 shows the first input-output data set TR1 and the second input-output data set TR2 after error addition among the M input-output data sets after error addition. Figure 7 As shown, it is possible to Figure 6The 1 input-output data set TR shown generates M error-added input-output data sets, including an error-added input-output data set TR1 with an error A added to a random number generated based on a parameter a for generating a random number, and an error-added input-output data set TR2 with an error B added to a random number generated based on a parameter b for generating a random number.

[0096] According to this error-added input / output dataset, the value of the "previous state" is not the actual measured value, but rather a predicted value with error based on a probability distribution centered on the measured value. Therefore, the error-added input / output dataset can be formed as an input dataset that uses the measured values of the process operation and the predicted value of the previous state with virtual error as input, and the measured value of the next state as output. Therefore, by executing machine learning using the model training unit 11E, even if there is a prediction error in the previous state data, the next state can be accurately predicted.

[0097] The model training unit 11E is a processing unit that trains a dynamic prediction model. Hereinafter, the dynamic prediction model before training and the trained dynamic prediction model may be identified as "dynamic prediction model 12a" and the dynamic prediction model after training as "dynamic prediction model 12A".

[0098] Figure 8 is a schematic diagram showing the training examples of the dynamic prediction model. Figure 8 As shown, the model training unit 11E can use the M error-added input and output data sets TR1 to TRM generated by the error adding unit 11D as training data sets to perform machine learning, such as deep learning, on the dynamic prediction model 12a. This dynamic prediction model 12a can be implemented by any machine learning model, such as a support vector machine represented by a neural network or DNN (Deep Neural Network), or gradient boosting. For example, the model training unit 11E sets the operation amount for the process and the previous state of the process as the explanatory variables of the dynamic prediction model 12a, and sets the next state of the process as the target variable of the dynamic prediction model 12a, and trains the parameters of the dynamic prediction model 12a according to the deep learning algorithm. In this way, a trained dynamic prediction model 12A is generated.

[0099] While the example of using the input-output datasets TR1 to TRM after adding M errors for machine learning in the dynamic prediction model 12a is presented here, the input-output dataset created by the dataset creation unit 11C can also be used. While the example of using multiple datasets for machine learning is presented here, machine learning can naturally be performed as long as there is at least one dataset or more.

[0100] The dynamic prediction model 12A generated in this way is used as the simulator 12 of the execution environment of the above-mentioned reinforcement learning. Figure 9 is a schematic diagram showing an example of simulation. Figure 9 2 shows an example of a dynamic prediction model 12A generated by the simulator generation unit 11 according to the present embodiment and a simulator 12 using the dynamic prediction model 12A.

[0101] like Figure 9 As shown, simulator 12 queries dynamic prediction model 12A using predetermined initial values for operation and state data. The predicted value for the next state output by model 12A is recursively utilized, thereby simulating state transitions based on changes in the manipulated variable. Furthermore, dynamic prediction model 12A is trained to accurately predict the next state regardless of the presence or absence of errors in the previous state. This reduces the impact of errors in the previous state's predicted value on the prediction of the next state. Consequently, the propagation of errors can be suppressed.

[0102] Figure 10 is a diagram showing an example of error propagation suppression. For example, Figure 10 In FIG. 1 , the predicted values of the state of the process outputted in time series by the simulator 12 according to the present embodiment are drawn by the dotted line, while the actual measured values of the state of the process are drawn by the solid line as a reference. Figure 10 The vertical axis of the graph shown is the state of the process, for example the value of a process variable. Figure 10 The horizontal axis of the graph shown represents time.

[0103] like Figure 10 As shown, not only at the beginning of the simulation, but also until the end of the simulation, there is no significant difference between the predicted value and the measured value, and it is clear that the propagation of the error between the predicted value and the measured value can be suppressed throughout the entire simulation period.

[0104] <Processing Flow>

[0105] Next, the flow of processing by the information processing device 10 according to this embodiment will be described. Here, after describing (1) the overall processing executed by the simulator generation unit 11, (2) the error addition processing executed as a sub-flow thereof will be described.

[0106] (1) Overall processing

[0107] Figure 11 1 is a flowchart showing the overall processing flow of the simulator generation unit 11. Figure 11 As shown, the data acquisition unit 11A acquires actual operation data and input / output definition information of the simulator 12 from the client terminal 30 (step S101 ).

[0108] Next, the data acquisition unit 11A extracts input data corresponding to input tags defined by the input / output definition information and output data corresponding to output tags from the actual operation data acquired in step S101 (step S102 ).

[0109] Then, the learning period selection unit 11B selects a learning period to be used as a target of machine learning for training the dynamic prediction model 12 a (step S103 ).

[0110] Then, the data set creation unit 11C creates an input-output data set based on the input-output data corresponding to the learning period selected in step S103 in the entire interval of the input-output data extracted in step S102 (step S104 ).

[0111] Moreover, the error adding unit 11D performs the following "error adding processing", that is, for each input and output sample contained in the input and output data set created in step S104, a random number generated according to the specified probability distribution is added to the measured value of the "previous state" of the input sample as virtual error data (step S105).

[0112] As a result of the error addition process executed in step S105 , M error-added input / output data sets TR1 to TRM are generated.

[0113] Then, the model training unit 11E uses the M error-added input-output data sets TR1 to TRM generated as a result of step S105 as training data sets to perform machine learning, such as deep learning, on the dynamic prediction model 12a (step S106).

[0114] As a result of the machine learning performed in step S106 , a trained dynamic prediction model 12A is generated.

[0115] (2) Additional error processing

[0116] Figure 12 is a flowchart showing the process of error addition processing. Figure 11 The processing of step S105 shown in FIG. Figure 12 As shown, the error adding unit 11D reads the input / output data set created in step S104 (step S301).

[0117] The error adding unit 11D then executes a loop process 1, repeatedly executing steps S302 through S304 as many times as the number M of error-added input / output datasets to be generated from one input / output dataset. Alternatively, steps S302 through S304 may be executed in parallel, rather than repeatedly.

[0118] That is, the error adding unit 11D sets parameters for generating random numbers used for generating the m-th error data, such as probability distribution and random number seeds (step S302 ).

[0119] The error adding unit 11D then performs loop processing 2, repeating steps S303 and S304 as many times as the number N of input / output samples included in the input / output data set. Alternatively, steps S303 and S304 may be performed in parallel, rather than repeatedly.

[0120] That is, the error adding unit 11D generates a random number using the random number generation parameters set in step S302, thereby generating virtual error data (step S303). Next, the error adding unit 11D adds the error data generated in step S303 to the previous state of the input sample in the n-th input / output sample (step S304).

[0121] By repeatedly executing this loop process 2, error data is added to the previous state of N input samples included in the input / output dataset, resulting in the completion of the mth error-added input / output dataset. Furthermore, by repeatedly executing loop process 1, M error-added input / output datasets are completed.

[0122] <Effects of Embodiment 1>

[0123] As described above, the information processing device 10 according to this embodiment executes machine learning of the dynamic prediction model 12a using an input / output dataset in which virtual error data is added to input samples in time-series data of input / output related to process control.

[0124] The dynamic prediction model 12A generated by this machine learning is trained in a manner that accurately predicts the next state regardless of the presence or absence of errors in the previous state. Therefore, when the simulator 12 simulates the transition of states, it can reduce the adverse effects of the errors in the predicted values of the previous state on the prediction of the next state.

[0125] Therefore, according to the information processing device 10 according to the present embodiment, it is possible to create a data-driven simulator that can suppress error propagation.

[0126] Here, in order to explain the improvement effect based on the dynamic prediction model 12A according to this embodiment, simulation results are compared between the simulator 2 using the conventional dynamic prediction model 2A and the simulator 12 using the dynamic prediction model 12A according to this embodiment.

[0127] Figure 13 and Figure 14 is a graph showing an example of simulation results. For example, Figure 13 In FIG, the predicted value of the state of the process outputted in time series by the simulator 2 using the dynamic prediction model 2A according to the conventional example is drawn by the dotted line. Figure 14 In FIG, the predicted value of the state of the process output in time series by the simulator 12 using the dynamic prediction model 12A of this embodiment is drawn by the dotted line. Figure 13 and Figure 14 In FIG, the actual measured value of the process state is drawn by the solid line as a reference. Figure 13 and Figure 14 The vertical axis of the graph shown represents the state of the process, for example the value of a process variable. Figure 13 and Figure 14 The horizontal axis of the graph shown represents time.

[0128] For example, when the dynamic prediction model 2A of the prior art is used for simulation, the error of the prediction value at the previous moment will have a significant impact on the prediction value at the next moment. Figure 13 As shown, it is clear that the prediction error is propagated and expanded along with the state transition. According to the simulator 2 of this conventional example, the agreement rate between the reference measured value and the simulated predicted value is only 3%.

[0129] On the other hand, when the dynamic prediction model 12A according to this embodiment is used for simulation, the propagation of errors can be suppressed. Figure 14 As shown, even if an error temporarily occurs during the simulation, there is no tendency for the error to expand. Thus, according to the simulator 12 of this embodiment, the agreement rate between the reference measured value and the simulated predicted value is increased to 89%, which is a higher agreement rate than the conventional example.

[0130] Therefore, when the AI control strategy is learned by repeatedly performing trial and error based on reinforcement learning on the simulator 12 involved in this embodiment, the propagation of errors can be suppressed, thereby reducing the adverse effects of prediction errors on the reinforcement learning of the control strategy, and as a result, it can contribute to the verification of the effect of AI control simulation.

[0131] <Other Implementation Methods>

[0132] While the embodiments of the present invention have been described above, various applications are possible, and the present invention can be implemented in various different forms in addition to the above-described embodiments.

[0133] Simulators for different operating conditions

[0134] Processes in plants and other areas are not always controlled under the same operating conditions. For example, the operating conditions of petrochemical processes may change. Consequently, different operating conditions can lead to different input-output relationships in the process, making it difficult to represent processes under multiple operating conditions using a single dynamic prediction model. Therefore, based on aspects related to multiple operating conditions, a data-driven simulator can be generated for each operating condition.

[0135] Figure 15 1 is a block diagram showing an example of the functional configuration of an information processing device 20 according to an application example. Figure 15 In the Figure 1 Modules having the same functions in the information processing device 10 shown are denoted by the same reference numerals.

[0136] like Figure 15 As shown, in the information processing device 20, in addition to the actual operating data and input / output definition information, operating condition information (periods of each operating condition) is also input from the client terminal 30. For example, for each operating condition of the process, the period during which the process is controlled under the operating condition is associated with the operating condition information. Based on this operating condition information, for each operating condition, the fragment data of the period corresponding to the operating condition in the operating data is assigned to the data assignment unit 21. Then, the simulator generation unit 11 simply executes the simulation for each operating condition using the fragment data assigned to the operating condition. Figure 11 、 Figure 12 The processing shown is sufficient. Thus, a simulator corresponding to each operating condition is generated for the operating condition. Furthermore, the AI control model generation unit 13 can generate AI control models for different operating conditions for simulators of different operating conditions. Alternatively, the AI control model generation unit 13 can alternately learn simulators of multiple operating conditions to generate one AI control model that can correspond to multiple operating conditions. Furthermore, the report generation unit 15 can simulate AI control on simulators of different operating conditions based on the AI control models for each different operating condition or one AI control model that can correspond to multiple operating conditions, and verify the effect of AI control under multiple operating conditions.

[0137] <Numerical values, etc.>

[0138] The matters described in the above embodiments, such as the number of client terminals 30, the number of input and output data sets after error addition, and the specific examples of parameters used to generate random numbers, are merely examples and may be modified. Furthermore, the processing order of the flowcharts described in the embodiments may be modified within a range consistent with the present disclosure.

[0139] <System>

[0140] The information including the processing flow, control flow, specific names, various data, and parameters described above and shown in the accompanying drawings can be arbitrarily changed, except where otherwise noted. For example, any one or more functional units of the simulator generation unit 11, AI control model generation unit 13, and report generation unit 15 of the information processing device 10 may be composed of different devices.

[0141] Furthermore, the components of the devices shown in the diagrams are functional concepts and do not necessarily need to be physically configured as shown. Specifically, the specific methods of distributing and integrating the devices are not limited to those shown. Specifically, all or part of the components can be functionally or physically distributed / integrated in arbitrary units based on various loads, usage conditions, and other factors. Furthermore, each structure may also be a physical structure.

[0142] Furthermore, all or any part of each processing function performed by each device may be realized by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or may be realized as hardware based on wired logic.

[0143] Hardware

[0144] Next, a hardware configuration example of the computer described in the above embodiment will be described. Figure 16 This is a diagram showing an example of a hardware configuration. Figure 16 As shown, the information processing device 10 includes a communication device 10a, a storage device 10b, a memory 10c, and a processor 10d. Figure 16 The components shown are connected to each other by a bus or the like.

[0145] The communication device 10a is a network interface card, etc. The storage device 10b is a storage device such as HDD (Hard Disk Drive) or SSD (Solid State Drive). Figure 1 、 Figure 4 The program, DB, etc. that execute the functions shown are stored.

[0146] The processor 10d reads and executes the data from the storage device 10b and the like. Figure 1 The same processing unit as shown in FIG. 1 is expanded in the memory 10c, thereby making the execution Figure 1 The process of executing the functions described in the above etc.

[0147] This process implements the same functions as the processing unit of the information processing device 10. For example, the processor 10d reads a program having the same functions as the simulator generation unit 11, the AI control model generation unit 13, and the report generation unit 15 from the storage device 10b. The processor 10d then executes a process that performs the same processing as the simulator generation unit 11, the AI control model generation unit 13, and the report generation unit 15.

[0148] In this manner, the information processing device 10 operates as an information processing device that reads and executes a program to perform a computational method. Furthermore, the information processing device 10 can also read the program from a recording medium using a media reader and execute the program, thereby achieving the same functionality as the aforementioned embodiment. Furthermore, the programs described in other embodiments are not limited to being executed by the information processing device 10. For example, the present invention can also be applied to other computers or servers executing the program, or to other computers or servers operating in collaboration to execute the program.

[0149] The above program can be distributed via a network such as the Internet. In addition, the above program can be recorded on any recording medium and read from the recording medium by a computer for execution. For example, the recording medium can be implemented by a hard disk, a floppy disk (FD), a CD-ROM, an MO (Magneto-Optical disk), a DVD (Digital Versatile Disc), etc.

[0150] <Other>

[0151] Several examples of combinations of disclosed technical features are described below.

[0152] (1) An information processing device, characterized in that:

[0153] The information processing device comprises:

[0154] an acquisition unit that acquires time-series data related to an operation amount for a process and a process value indicating a state of the process;

[0155] a creating unit that creates an input-output data set in which an operation amount and a process value at a first moment are set as input samples, and a process value at a second moment subsequent to the first moment is set as an output sample;

[0156] an appending section that appends virtual error data to each process value of the input sample; and

[0157] A training unit trains a machine learning model that takes the manipulated variable and the process value at the first moment as input and outputs the process value at the second moment using the input-output data set to which the virtual error data is added.

[0158] (2) The information processing device according to (1), wherein the adding unit includes a process of generating the virtual error data by generating a random number based on a predetermined probability distribution.

[0159] (3) The information processing device according to (2), characterized in that the probability distribution is a normal distribution.

[0160] (4) The information processing device according to (3), wherein the mean of the normal distribution is 0 and the standard deviation is a predetermined value.

[0161] (5) The information processing device according to any one of (1) to (4) is characterized in that the creation unit includes the following processing, namely, setting the first moment and the second moment based on the time difference from the change of the operation amount to the change of the process value.

[0162] (6) The information processing device according to any one of (1) to (4), characterized in that the information processing device further comprises an extraction unit that clusters the segment data based on the feature values of each segment data after the time series data is divided into a plurality of intervals, and extracts the segment data belonging to each cluster at a predetermined ratio.

[0163] The creation unit includes a process of creating an input / output data set using the fragment data extracted by the extraction unit.

[0164] (7) The information processing device according to any one of (1) to (4), further comprising an allocating unit that allocates, for each operating condition, fragment data of a period corresponding to the operating condition in the time series data, based on operating condition information that associates, for each operating condition of the process, the period during which the process is controlled under the operating condition.

[0165] The creation unit, the addition unit, and the training unit execute processing using the segment data allocated for each of the operating conditions.

[0166] (8) The information processing device according to any one of (1) to (4), characterized in that the machine learning model is implemented by a neural network.

[0167] (9) A generation method, characterized in that:

[0168] The computer performs the following processing:

[0169] acquiring time series data related to an operation quantity for a process and a process value representing a state of the process;

[0170] Creating an input-output data set in which the operation amount and process value at a first moment are set as input samples, and the process value at a second moment subsequent to the first moment is set as output samples;

[0171] appending fictitious error data for each process value of the input sample; and

[0172] The input-output data set to which the fictitious error data is added is used to train a machine learning model that takes the manipulated variable and the process value at the first moment as input and outputs the process value at the second moment.

[0173] (10) A generation program, characterized in that

[0174] The generation program causes the computer to execute the following processing:

[0175] acquiring time series data related to an operation quantity for a process and a process value representing a state of the process;

[0176] Creating an input-output data set in which the operation amount and the process value at a first moment are set as input samples and the process value at a second moment subsequent to the first moment is set as output samples;

[0177] appending fictitious error data for each process value of the input sample; and

[0178] The input-output data set to which the fictitious error data is added is used to train a machine learning model that takes the manipulated variable and the process value at the first moment as input and outputs the process value at the second moment.

[0179] Description of the label

[0180] 10 Information processing device

[0181] 11 Simulator Generation Unit

[0182] 11A Data Acquisition Department

[0183] 11B Study Period Selection

[0184] 11C Dataset Creation Department

[0185] 11D Error Addition Unit

[0186] 11E Model Training Department

[0187] 12A dynamic prediction model

[0188] 12 Simulator

[0189] 13 AI control model generation department

[0190] 14 AI Control Model

[0191] 15 Report Generation Department

[0192] 30 Client Terminal

Claims

1. An information processing device, characterized in that The information processing device comprises: an acquisition unit that acquires time-series data related to an operation amount for a process and a process value indicating a state of the process; a creating unit that creates an input-output data set in which an operation amount and a process value at a first moment are set as input samples, and a process value at a second moment subsequent to the first moment is set as an output sample; an appending unit for appending virtual error data to each process value of the input sample; as well as A training unit trains a machine learning model that takes the manipulated variable and the process value at the first moment as input and outputs the process value at the second moment using the input-output data set to which the virtual error data is added.

2. The information processing device according to claim 1, wherein The adding unit includes a process of generating the virtual error data by generating a random number according to a predetermined probability distribution.

3. The information processing device according to claim 2, wherein The probability distribution is a normal distribution.

4. The information processing device according to claim 3, wherein The normal distribution has a mean of 0 and a standard deviation of a specified value.

5. The information processing device according to claim 1, wherein The creation unit includes a process of setting the first time and the second time based on a time difference from when the manipulated variable changes until when the process value changes.

6. The information processing device according to claim 1, wherein The information processing device further includes an extraction unit that clusters the segment data based on the feature amount of each segment data after the time series data is divided into a plurality of intervals, and extracts the segment data belonging to each cluster at a predetermined ratio. The creation unit includes a process of creating an input / output data set using the fragment data extracted by the extraction unit.

7. The information processing device according to claim 1, wherein The information processing device further includes an allocating unit that allocates, for each operating condition of the process, segment data of a period corresponding to the operating condition in the time series data based on operating condition information that associates, for each operating condition of the process, a period during which the process is controlled under the operating condition. The creation unit, the addition unit, and the training unit execute processing using the segment data allocated for each of the operating conditions.

8. The information processing device according to claim 1, wherein The machine learning model is implemented by a neural network.

9. A generation method, characterized in that, The computer performs the following processing: acquiring time series data related to an operation quantity for a process and a process value representing a state of the process; Creating an input-output data set in which the operation amount and process value at a first moment are set as input samples, and the process value at a second moment subsequent to the first moment is set as output samples; For each process value of the input sample, appending virtual error data; as well as The input-output data set to which the fictitious error data is added is used to train a machine learning model that takes the manipulated variable and the process value at the first moment as input and outputs the process value at the second moment.

10. A computer-readable recording medium having a generation program recorded thereon, characterized in that: The generation program causes the computer to execute the following processing: acquiring time series data related to an operation quantity for a process and a process value representing a state of the process; Creating an input-output data set in which the operation amount and the process value at a first moment are set as input samples and the process value at a second moment subsequent to the first moment is set as output samples; For each process value of the input sample, appending virtual error data; as well as The input-output data set to which the fictitious error data is added is used to train a machine learning model that takes the manipulated variable and the process value at the first moment as input and outputs the process value at the second moment.

Images