Method and apparatus for monitoring machine learning models

Abstract

The invention relates to the field of monitoring or controlling industrial processes, in particular by means of artificial neural networks. The method comprises the following steps: training a first control model (10) by means of a first input data set (31) as a first input (11), resulting in a trained first control model (10T); copying the trained first control model (10T) to a second control model (20), wherein after the copying the second input layer (22) and the plurality of second hidden layers (24) are identical to the plurality of first hidden layers (14) and the first output layer (18) is replaced by a second output layer (28); freezing the plurality of second hidden layers (24); training the second control model (20) by means of the first input data set (31) as a second input (21), resulting in a trained second control model (20T); and running the trained second control model (20T) by means of a second input data set (32) as a second input (21), wherein a second output (29) outputs a quality measure (qm) of the first control model (10).

Description

Technical Field

[0001] This invention relates to the field of monitoring or controlling industrial processes, particularly using artificial neural networks (ANNs). The invention also relates to an apparatus, a computer program product, a computer-readable storage medium, and its applications. Background Technology

[0002] Machine learning models (such as artificial neural networks (ANNs)) can be used in industrial processes, for example, to perform monitoring and / or control tasks. However, at least some machine learning models may degrade over time because the processes that generate the relevant data may change with the duration of use. In at least some applications, degradation is difficult to measure. Summary of the Invention

[0003] Therefore, the object of the present invention is to provide a method that at least partially improves this situation. This object is achieved by the subject matter of the independent claims. Other embodiments will be apparent from the dependent claims and the following description.

[0004] One aspect relates to a method for determining a quality metric of a first control model to monitor or control an industrial process, wherein the first control model is an artificial neural network (ANN) including a first input, a first input layer, a plurality of first hidden layers, a first output layer, and a first output. The method includes the following steps:

[0005] The first control model is trained by using the first input dataset as the first input, resulting in a trained first control model.

[0006] The trained first control model is copied to a second control model, wherein the second control model includes a second input, a second input layer, multiple second hidden layers, a second output layer, and a second output, the second output being configured to output a quality metric of the first control model. After copying, the second input layer is identical to the first input layer, the multiple second hidden layers are identical to the multiple first hidden layers, and the first output layer is replaced by the second output layer.

[0007] Freeze at least a portion of multiple second hidden layers.

[0008] The second control model is trained by using the first input dataset as the second input, resulting in a trained second control model.

[0009] A trained second control model is run using a second input dataset as a second input, where the second output outputs a quality metric of the first control model.

[0010] The first control model may be referred to as a "primary model." An industrial process can be any process that produces, manufactures, and / or alters any commodity, such as those of chemical, mechanical, and / or electrical nature. The first control model can be used to directly or indirectly control the industrial process, and / or to monitor it, for example, by delivering information about the process and / or its behavior. The first control model can be or includes an artificial neural network, possibly combined with standard computers and / or other computing components, simply referred to as an "ANN." Each layer in the first control model may include a set of neurons connected by weighted edge connections. Values ​​generated by the first output layer can be output to the industrial process and / or displays, alarm devices, etc., via a first output. The values ​​output to the industrial process may contribute to controlling and / or guiding the industrial process.

[0011] Before the productive use of the first control model, it is trained or applied to a first input dataset as the first input. The first input dataset may include historical data from one or more known monitoring or control conditions; that is, both the input and output (e.g., the desired action or response) are known. The conditions may include well-performing or very well-performing conditions and / or poorly performing conditions. Training means that the first control model is trained to change, specifically its multiple first hidden layers, its neurons, and / or the weights of its weighted edges can be altered. Training the first control model may include considering a cost function and / or reward when the monitored or control output is within the desired range. This may include some kind of prediction of how an industrial process will perform in the near future (e.g., in a second, a minute, an hour, a day, etc.). The prediction timeframe may depend on the type and / or other details of the industrial process.

[0012] Training the first control model produces a trained first control model, which is then ready to execute its (multiple) functions. Note that productive use of the trained first control model may alter it, causing it to degrade over time.

[0013] Immediately after (initial) training, the trained first control model is copied to a second control model using the first input dataset. This second control model can be referred to as a "surrogate model." The second control model is then "frozen," meaning at least some of the second hidden layers remain unchanged. The second control model (e.g., the second input layer and multiple second hidden layers) is then very similar to or identical to the trained first control model, except for its second output layer. The second output layer can be trained to output a quality metric for the first control model, i.e., used to evaluate the current quality of the first control model. The cost function and / or reward of the second control model can include quality-related values, thereby altering the second output layer. These quality-related values ​​may consider the direct response to the industrial process, but also (additionally or alternatively) consider predictions of the industrial process in the near future (e.g., as defined above). This prediction may include long-term effects or responses to the industrial process, some delay time (dead time), and / or other effects or responses. In other words, the second output is configured to output a quality metric for the first control model. Therefore, after copying, the first output layer is replaced by the second output layer. The trained second control model can then be fixed or "frozen." This "freezing" can include fixing the weights between neurons in different layers, making it impossible to adjust the fixed weights of the frozen layers during training. This "freezing" might also involve not fixing all hidden layers, but rather keeping some layers unfrozen.

[0014] During the productive use of the first control model, it is further trained using a second input dataset. This training may include making and / or using predictions. The second input dataset may be “real-time data” from the industrial process. In parallel, the trained second control model (i.e., untrained and therefore unchanged) is run using the second input dataset as a second input. Since the second output of the trained second control model is configured to output a quality metric of the first control model, any degradation of the first control model (i.e., its quality and / or performance) can be measured in this way. Once available, the degradation metric can be used in various ways, such as to alert service personnel, to improve the industrial process and / or its control, and / or to run custom applications.

[0015] In various embodiments, the final step (particularly running a trained second control model with the aid of a second input dataset) is repeated periodically and / or upon request. The periodicity of "periodically" may depend on the industrial process to be monitored or controlled. For some industrial processes, repetition may be performed monthly, weekly, daily, hourly, minutely, secondly, and / or less frequently or more frequently. Additionally or alternatively, the user and / or machine may request the repetition of the final step. Repetition can advantageously contribute to high-quality monitoring of the industrial process and / or timely execution.

[0016] In various embodiments, the method further includes the following steps:

[0017] Compare quality metrics with predefined metrics; and

[0018] If the quality metric is within a predefined metric, then the predefined action is executed.

[0019] Predefined metrics can be a range of metrics and / or can include a set of predefined metrics for comparison with a range of quality metrics. Being outside of a predefined metric may mean being below or even above a predefined metric. A predefined action can be a single action or multiple and / or a series of actions. When a quality metric is within a predefined metric, another predefined action can be performed, such as sending a message, typing a record in a journal, another action, or no action at all.

[0020] In various embodiments, the predefined actions include at least one of the following: outputting an alarm and / or retraining the first control model. Outputting an alarm may include sending any kind of message to a user and / or machine and / or journal. Retraining the first control model can produce a corrected, trained first control model, which may improve and / or otherwise alter the industrial process and / or its quality.

[0021] Retraining the first control model includes the following steps:

[0022] Unfreeze the hidden layers of the trained second control model;

[0023] The second control model is trained by using a third input dataset as the second input, where the third input dataset is a historical dataset selected from multiple second input datasets, and the second output is an output quality metric.

[0024] Freeze the hidden layers of the second control model;

[0025] Compare quality metrics with predefined metrics; and

[0026] If the quality metric is within a predefined range, the first control model is trained using a third input dataset as the first input, resulting in a corrected and trained first control model.

[0027] Unfreezing the hidden layers of the trained second control model prepares it for further training, i.e., for modification. Training is performed using a third input dataset as the second input. The third input dataset is a collection or subset of historical data that may have already been used as input data for the first and / or second inputs. Historical data may include multiple and / or a series of second input datasets (or at least one second input dataset). Historical data can be generated by storing a series of real-time data from the industrial process. The third input dataset can be selected in any manner, or by using criteria such as: optimal performance of the industrial process, highest stability, and / or other criteria. The length of the third input dataset (e.g., a time window extracted from historical data) may be similar to the length of the first input dataset used for initial training.

[0028] The third input dataset, selected as described, is then applied to the second input. Therefore, the second output measures the quality, and the second control model is trained (and modified) using this third input dataset. The hidden layers of the trained second control model are then frozen.

[0029] Subsequently, the quality metric is compared with a predefined metric. If the quality metric falls within the predefined range, the trained second control model remains frozen, and the first control model uses a third input dataset as its first input. This retraining produces a corrected trained first control model. Advantageously, the corrected trained first control model utilizes the third input dataset, which is known to be beneficial for industrial processes because this dataset has already led to improved processes. Therefore, the first control model not only avoids degradation but can also be continuously improved.

[0030] In various embodiments, the method further includes the step of repeating the unfreezing, training, freezing, and comparison steps if the quality metric is outside a predefined metric. This can advantageously contribute to the targeted improvement of the first control model and thus to the improvement of industrial processes.

[0031] In various embodiments, training and / or retraining the first and / or second control models includes making predictions and / or using predictions. This may be possible because historical data may be used for training and / or retraining, making the responses of the industrial process known, at least in the near future. Making predictions and / or using predictions can advantageously help monitor or control further improvements to the industrial process, or further improve the industrial process itself, especially since critical, long-term, and / or unexpected responses to the process may be taken into consideration.

[0032] One aspect relates to a computer program product comprising instructions that, when executed by a computer and / or an artificial neural network (ANN), cause the computer and / or ANN to perform the methods described above and / or below.

[0033] One aspect relates to a computer-readable storage medium on which the aforementioned computer program or computer program product is stored.

[0034] One aspect involves a first control model and / or a second control model, configured to perform the methods described above and / or below.

[0035] One aspect relates to the use of the first control model for monitoring and / or controlling industrial processes. Attached Figure Description

[0036] The subject matter of the invention will be explained in more detail below with reference to preferred exemplary embodiments illustrated in the accompanying drawings, wherein:

[0037] Figure 1a A first control model according to an embodiment is schematically illustrated;

[0038] Figure 1b A second control model according to an embodiment is schematically illustrated;

[0039] Figure 2a A simplified diagram of a first control model according to an embodiment is shown schematically;

[0040] Figure 2b A simplified diagram of the second control model according to an embodiment is shown schematically;

[0041] Figure 3 A data flow according to an embodiment is shown;

[0042] Figure 4 Another data stream according to an embodiment is shown;

[0043] Figure 5 Another data stream according to an embodiment is shown.

[0044] The reference numerals used in the accompanying drawings and their meanings are listed in summary form in the reference numeral list. In principle, the same parts are provided with the same reference numerals in the accompanying drawings. Detailed Implementation

[0045] Figure 1aA first control model 10 according to an embodiment is schematically shown. The first control model may also be referred to as a "primary model". The first control model 10 has one or more inputs 11 and one or more outputs 19. The inputs 11 may come from user specifications, control specifications, and / or signals fed back from the industrial process 50. The outputs 19 can control or monitor the industrial process 50. Other components (not shown) may be helpful and / or necessary for monitoring or controlling the industrial process 50, such as networks, displays, databases, journals, and / or other components.

[0046] Figure 1b A second control model 20 according to an embodiment is schematically illustrated. The second control model may be referred to as a "surrogate model". The second control model 20 can operate in parallel with the first control model 10 (see [link]). Figure 1a The system can be operated by feeding a defined dataset into its input(s) 21. Output 29 can be configured to provide a quality metric qm and / or other data. The quality metric qm can be an indicator of the quality or performance of the first control model 10 and / or the industrial process 50.

[0047] Figure 2a A simplified diagram of a first control model 10 according to an embodiment is schematically shown. The first control model 10 includes an input layer 12, hidden layers 14, and an output layer 18. Each of the hidden layers 14 includes an array of neurons (depicted as an array of nodes) connected by weighted edges. In many control models, the number of hidden layers can be freely configured (possibly within a predefined range), and the weights of the weighted edges can be changed through the training process, which at least affects the weights. Furthermore, the first control model 10 has an input 11 and an output 19.

[0048] Figure 2b A simplified diagram of a second control model 20 according to an embodiment is schematically shown. The second control model 20 includes an input layer 22, a hidden layer 24, and an output layer 28. Furthermore, the second control model 20 has an input 21 and at least one output 29.

[0049] Using two control models 10 and 20 not only allows for the monitoring and / or control of industrial processes, but also provides a measure of the quality of the industrial process and / or its control model. For this purpose, both the first control model 10 and the second control model 20 are trained. Figure 2a and 2bThe diagram illustrates the relationship between the first control model or primary model 10 and the second control model or surrogate model 20. The two models are identical in terms of input, architecture (e.g., the number of hidden layers, nodes, and connections), and weights between the input layer and the first hidden layer, as well as between hidden layers. In this way, the performance of the two models 10 and 20 is closely correlated. If the features and relationships learned in the primary model become ineffective for new data and subsequent performance degrades, the same will happen to the surrogate model.

[0050] Figure 3 The data flow 300 according to an embodiment is illustrated, including a training process for a primary model 10 and a surrogate model 20. First, the primary model 10 is trained. Then, the primary model 10 is duplicated, and the output layer 28 is replaced. The output layer 28 can be selected in a manner where the data itself can be used for labeling (self-learning or self-supervised learning). As an example, a compression reconstruction task (e.g., an autoencoder) or a data regression task (e.g., predicting future values) can be implanted. Then, all layers 22, 24, except for the last layer 28, are frozen; that is, the training process is no longer allowed to change the weights between the input layer 22 and the hidden layer 24, or between the hidden layers 24. The surrogate model 20 is then retrained for the new task. Finally, the performance of the surrogate model 20 on the training data and the validation set is measured. The performance metric depends on the self-supervised learning task, such as the reconstruction error of the autoencoder or the mean squared error of the regression.

[0051] Figure 3Data flow 300 illustrates some details of the process of the initial training sequence of the first control model 10 and the second control model 20. In step 302, a first dataset 31 is provided. In step 304, the first control model 10 is trained using the first input dataset 31 as a first input 11. This produces a trained first control model or a trained primary model 10T in step 306. In step 308, the trained first control model 10T is copied to the second control model 20. Thus, the second input layer 22 is identical to the first input layer 12, and the plurality of second hidden layers 24 are identical to the plurality of first hidden layers 14. In step 310, the copied output layer 18 of the first control model 10 is replaced by the output layer 28 in the second control model 20. The values ​​of the output layer 28 are output to a second output 29, which is configured to output a quality metric qm or key performance indicator (KPI) of the first control model 10. The qm or KPI may include some predictions considering the behavior of the industrial process 50, for example, in the near future. In step 312, the hidden layer 24 of the second control model 20 is frozen. In step 314, the second control model 20 is trained using the first dataset 31. This produces the trained second control model or surrogate model 20T in step 316. In step 318, the quality metric 29 of the first control model 10 is provided by the trained second control model 20T, and it can be used in other steps.

[0052] Figure 4 A data flow 400 according to an embodiment is illustrated, including the application of agent model 20 during its productive use, such as for industrial process 50. Both primary model 10 and agent model 20 are fed the same real-time data 32 from the industrial process and / or a database. Primary model 10 can be used to generate predictions for monitoring or control tasks, which may result in actions being displayed or triggered on a human-machine interface. With the aid of agent model 20, performance or quality across a defined time window or a number of predictions is calculated. If performance falls outside a predefined threshold (e.g., below a predefined threshold), the user or administrator can be notified and / or retraining can be triggered.

[0053] Figure 4Data flow 400 illustrates some details of applying a second dataset (e.g., real-time data from industrial process 50) to the process of control models 10, 20. In step 402, a second dataset 32 ​​is provided. The second dataset 32 ​​may include real-time data from industrial process 50. In step 404, the second dataset 32 ​​is applied to the first control model 10, which may result in further training of the first control model 10 and modification of the first control model 10. In step 406, a first output 19 of the first control model 10 is provided to industrial process 50, for example, for controlling it. In step 408, the first output 19 of the first control model 10 is provided to a database, display, and / or another human-machine interface (HMI). Steps 406 and 408 may be executed in parallel, or one of the steps (e.g., 406) may be prioritized.

[0054] In step 410, the (unmodified) trained second control model 20T is run, and the quality metric qm is output at output 29. In step 412, the quality metric qm is compared with a predefined metric or measurement range, and it is checked whether the quality metric qm is outside the predefined metric (e.g., below the predefined metric). If the quality metric qm is within the predefined metric, then in step 414, a predefined action is executed; this may include doing nothing or sending a message. If in step 416 the quality metric qm is outside the predefined metric (e.g., below the predefined metric), then the predefined action is executed. The predefined action may include the action of data flow 500 (see...). Figure 5 ).

[0055] Figure 5 A data flow 500 according to an embodiment is shown, including how the surrogate model 20 is configured to find a suitable time window for retraining on historical data. This is based on the insight that not all historical data is likely to be beneficial for retraining models 10 and 20 due to changes in the underlying data. Based on this idea, different time windows (or a third dataset 33) are sampled from the entire historical dataset and used as training data for the self-learning surrogate model 20. Since the data itself is used for labeling, the performance of models 10 and 20 can be measured, for example, based on a test dataset with recent data, which may represent the current behavior of the industrial process 50. When the performance of the retrained surrogate model 20 shows a sufficiently good quality metric qm, the primary model 10 is retrained on the same third dataset 33 within that time window. Due to the high similarity between the two models and the fact that the selected historical data is capable of training a surrogate model 20 that can capture some recent behavior, the primary model 10 is likely to also perform well on that dataset 33.

[0056] Figure 5Data flow 500 illustrates some details of the process of retraining the first control model 10. In step 502, a third dataset 33 (a subset of historical data) is provided. In step 504, the hidden layer 24 of the second control model 20 is unfrozen. In step 506, the third dataset 33 is applied to the second control model 20, resulting in the retraining of the second control model 20. In step 508, the quality metric qm is output. In step 510, the hidden layer 24 is frozen. In step 512, the quality metric qm is compared with a predefined metric (or metric range), and some checks are performed to determine whether the quality metric qm is within the predefined metric. If the quality metric qm is not within the predefined metric, steps 504 to 510 are repeated, in many cases by selecting a different subset of historical data as the third dataset 33.

[0057] If the quality metric qm is within a predefined metric, then in step 514, the first control model 10 is retrained by using the selected third dataset 33 as input 11. In step 516, this retraining results in a corrected or updated first control model or a corrected primary model 10T2.

[0058] List of reference numerals

[0059] 10 First Control Model (Primary Model)

[0060] 10T trained first control model

[0061] 10T2 corrected or updated training first control model

[0062] 11 Inputs to the First Control Model

[0063] 12 Input Layer of the First Control Model

[0064] 14 Hidden Layers of the First Control Model

[0065] 18 Output layer of the first control model

[0066] 19 Output of the first control model

[0067] 20 Second Control Model (Proxy Model)

[0068] 20T trained second control model

[0069] 11 Inputs to the Second Control Model

[0070] 22 Input Layer of the Second Control Model

[0071] 24 Hidden Layer of the Second Control Model

[0072] 28 Output layer of the second control model

[0073] 29. Output (KPI) of the Second Control Model

[0074] 31 First Dataset

[0075] 32 Second dataset (real-time data from industrial processes)

[0076] 33 Third dataset (historical data)

[0077] 300 data stream

[0078] Steps for data flow 300 (302–318)

[0079] 400 data stream

[0080] Steps for data flow 402–416

[0081] 500 data streams

[0082] Steps for data flow 502–516

Claims

1. A method for determining a quality measure (qm) of a first control model (10) for monitoring or controlling an industrial process (50), wherein the first control model (10) is an artificial neural network, ANN, comprising a first input (11), a first input layer (12), a plurality of first hidden layers (14), a first output layer (18), and a first output (19), the method comprising the following steps: training the first control model (10) by means of a first input data set (31) as first input (11), resulting in a trained first control model (10T), wherein the first input data set (31) comprises historical data of one or more known monitoring or controlling situations; copying the trained first control model (10T) to a second control model (20), wherein the second control model (20) comprises a second input (21), a second input layer (22), a plurality of second hidden layers (24), a second output layer (28), and a second output (29), the second output (29) being configured to output the quality measure (qm) of the first control model (10), wherein, after copying, the second input layer (22) is identical to the first input layer (12), the plurality of second hidden layers (24) is identical to the plurality of first hidden layers (14), and the first output layer (18) is replaced by the second output layer (28); wherein a cost function and / or a reward of the second control model (20) comprises a quality related value, thereby changing the second output layer (22); freezing at least part of the plurality of second hidden layers (24); training the second control model (20) by means of the first input data set (31) as second input (21), resulting in a trained second control model (20T); and running the trained second control model (20T) by means of a second input data set (32) as second input (21), wherein the second output (29) outputs the quality measure (qm) of the first control model (10), wherein the second input data set (32) is real-time data from the industrial process.

2. The method according to claim 1, wherein the last step is repeated periodically and / or upon request.

3. The method according to claim 1, further comprising the following steps: comparing the quality measure (qm) to a predefined measure; and if the quality measure (qm) is outside the predefined measure, performing a predefined action.

4. The method according to claim 3, wherein the predefined action comprises at least one of: outputting an alarm, and / or re-training the first control model (10).

5. The method according to claim 4, wherein re-training the first control model (10) comprises the following steps: unfreezing the hidden layers (24) of the trained second control model (20T); training the second control model (20) by means of a third input data set (33) as second input (21), wherein the third input data set (33) is a historical data set selected from a plurality of second input data sets (32), wherein the second output (29) outputs the quality measure (qm); freezing the hidden layers (24) of the second control model (20); comparing the quality measure (qm) with a predefined measure; and if the quality measure (qm) is within the predefined measure, training the first control model (10) by means of the third input data set (33) as first input (11), resulting in a corrected trained first control model (10T2).

6. The method according to claim 5, further comprising the following steps: if the quality measure (qm) is outside the predefined measure, repeating the steps of thawing, training, freezing, and comparing.

7. The method according to claim 1, wherein training and / or retraining the first control model (10) and / or the second control model (20) comprises: making a prediction and / or using a prediction.

8. A computer program product comprising instructions which, when the program is executed by a computer and / or an artificial neural network, ANN, cause the computer and / or the ANN to carry out the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, wherein a computer program according to claim 8 is stored on the computer-readable storage medium.

10. A first control model (10) and / or a second control model (20) configured to carry out any one of claims 1 to 7.

11. Use of a first control model (10) according to any one of claims 1 to 7 or 10 for monitoring and / or controlling an industrial process (50).

Images