determining the appropriate sequence of actions to be taken depending on the operating state of the industrial plant

Abstract

A computer-implemented method (100) for determining an appropriate sequence of actions (6) to be taken during operation of an industrial plant (1) or a part of an industrial plant (1), comprising the steps of: • acquiring (110) values of a plurality of state variables (2) characterizing an operating state of the plant (1) or the part of the plant (1); • encoding (120), by at least one trained state encoder network (3), the plurality of state variables (2) into a representation (2a) of the operating state of the plant (1) or the part of the plant (1); • mapping (130), by a trained state-to-action network (4), the representation (2a) of the operating state into a representation (6a) of a sequence of actions (6) to be taken in response to the operating state; and • decoding (140), by a trained action decoder network (5), the representation (6a) of the sequence of actions (6) into the sought sequence of actions (6) to be taken.

Description

TECHNICAL FIELD

[0001] The present invention relates to monitoring an industrial plant and determining actions to be taken in response to certain operating states, in particular in response to abnormal operating states. BACKGROUND

[0002] The intended operation of many industrial plants is controlled by a distributed control system, DCS, which adjusts set points of low-level controllers in order to optimize one or more key performance indicators, KPIs. Any abnormal conditions of the plant, such as equipment failure or malfunction, are also continuously monitored. This monitoring can be at least partly integrated with the control by the DCS.

[0003] If an abnormal condition is detected, it is not always possible to automatically correct by means of the DCS. There are exceptional situations in every plant which require that a certain sequence of actions is carried out by an operator for correction. WO 2019 / 104 296 Al discloses an alarm management system which helps the operator to identify high-priority alarms. However, outputting an alarm to the operator is only the first step in correcting an abnormal condition. The operator must also carry out the correct sequence of actions.

[0004] US 2020 / 026 976 Al discloses a method for coordinating an industrial machine with a smart industrial assistant having a set of predefined commands. The state of the machine’s proprietary I / O interface is mapped to the predefined commands. Based on this mapping, a custom interface for the machine is generated.

[0005] US 2021 / 097 401 Al discloses a neural network system for processing input data values. The input data values are encoded by an encoder network and the encoding results are aggregated with an aggregator. The aggregated output and a combination of target input values are decoded by a decoder to produce a final output.

[0006] US 2014 / 237 487 Al discloses an event processing system and method of operation. Based on observations derived from measurements, the occurrence of an event is detected. Based on the event, a state transition to be performed and an action to be performed are determined.

[0007] Invention objectives

[0008] It is therefore an object of the present invention to help a plant operator to correct exceptional situations by calculating a sequence of actions to be taken from information obtained during monitoring of an industrial plant or a part of an industrial plant.

[0009] This object is achieved by a first method for determining a suitable sequence of events according to the first independent claim and by a second method for training a machine learning network for use in the first method according to the second independent claim. Further advantageous embodiments are detailed in the respective dependent claims. SUMMARY

[0010] The invention is defined by the appended claims. Embodiments and examples not covered by the claims are presented for the purpose of illustration and to facilitate the understanding of the claimed invention.

[0011] The invention provides a computer-implemented method for determining a suitable sequence of actions to take during operation of an industrial plant or a part of an industrial plant.

[0012] In the course of the method, values of a plurality of state variables characterizing an operational state of the plant or the part of the plant are acquired. For example, these values can be combined into a vector or a tensor. The specific set of state variables needed to characterize the state of the plant or the part of the plant is specific to the plant. Examples of such state variables include pressure, temperature, mass flow, voltage, current, fill level, and concentration of a substance in a mixture of substances.

[0013] The plurality of state variables, e.g., the vector or tensor comprising the values of the state variables, is encoded by at least one trained state encoder network into a representation of the operational state of the plant or the part of the plant. In particular, such a representation can have a much lower dimensionality than the original plurality of state variables. That is, the representation can depend on fewer variables than the original vector of state variable values.

[0014] Examples of state variables include, but are not limited to:

[0015] • at least variables indicative of process variables, such as pressure, mass flow, voltage, or current;

[0016] • events representing discrete changes in the plant, such as turning on or off a motor, a signal crossing an alarm limit, opening or closing a valve; and

[0017] • combinations of process variables and events.

[0018] If the state variables include both process variables and events, the state encoder network can be a combined state encoder network that takes both process variables and events as input. However, it is also possible to use a combination of two state encoder networks, where one state encoder network encodes the process variables and the other state encoder network encodes the events.

[0019] By means of the trained state-to-action network, the representation of the operating state is mapped to a representation of a sequence of actions to be taken in response to the operating state. In particular, just as a few variables can encode a complex operating state of a plant that depends on a very large number of state variables, a few variables can also encode a complex sequence of actions that comprises many different actions.

[0020] The representation of the sequence of actions is decoded by the trained action decoder network into the sought sequence of actions to be taken. The sequence of actions thus determined can then be used in any way possible in order to correct the abnormal situation. For example, the automatic execution of the actions in the sequence can be triggered, as far as the range of actions that can be executed automatically is concerned. For actions that can only be executed by an operator, the operator can be informed in any suitable way to execute the actions. Furthermore, any suitable action can be taken to help the operator to execute such actions. For example, if the operator needs to operate a certain control element in the DCS human-machine interface, the certain control element can be highlighted. If an action requires operating a physical control, such as a button, a switch or a knob, and the physical control is a control that is protected by a cover to prevent accidental or unintentional operation, the cover can be opened and / or turned on. Likewise, if an action requires a physical maintenance of certain equipment, a maintenance door to the location of the equipment can be opened, and / or the operator can be brought safely to the location of the equipment, such as by stopping the equipment that can harm the operator. Also hybrid solutions can be employed in the sense that the actions available in the HMI of the DCS are automatically executed without the need for human intervention or, optionally, at least under human supervision, i.e. when the sequence of actions proposed by the system is automatically carried out, human observation is carried out.

[0021] Examples of actions in the sequence of actions include:

[0022] • enabling or disabling equipment in the plant or a part of the plant;

[0023] • opening or closing a valve of the plant or a part of the plant; and

[0024] • changing a setpoint value of at least one low-level controller in the plant or a part of the plant.

[0025] The operating state of the plant is primarily evaluated when mapping the representation of the operating state to the representation of the sequence of actions. Since the representations typically have much lower dimensionality than the operating state (respectively the sequence of actions), this means that the evaluation only comprises a mapping between two low-dimensional spaces. This makes the state-to-action network easier to train than a network that directly takes a high-dimensional operating state as input and directly outputs a high-dimensional sequence of actions. In particular, the state-to-action network is easier to train in that the training requires a smaller number of training samples.

[0026] Moreover, the different tasks in the overall processing chain, i.e.

[0027] • from a high-dimensional input into a low-dimensional representation,

[0028] • processing this representation into another representation in another low-dimensional space, and

[0029] • from the second low-dimensional representation space into a second high-dimensional output space,

[0030] assigned to different networks, which can be specialized in their respective jobs. This improves the overall accuracy of the final determined output compared to using a single network that has to perform all tasks at the same time and possibly has to trade off between conflicting goals. This is somewhat analogous to the principle of Unix tools such as sed, awk or grep. Each tool is built to do only one simple job, and is good at it. For more complex jobs, the output of one tool is piped to another tool as input.

[0031] In a particularly advantageous embodiment, the state encoder network is chosen as the encoder part of an encoder-decoder arrangement that first maps the plurality of state variables to a representation of the operating state of the plant or of a part of the plant and then reconstructs the plurality of state variables from the representation. Such an encoder-decoder arrangement can be trained in a self-supervised manner. That is, a training sample can be input into the arrangement and the degree of agreement of the arrangement's result with the original input can be evaluated. For such training to work, the training samples do not need to be labeled with "ground truth values". In machine learning applications, obtaining "ground truth values" is the most expensive part of the training.

[0032] Moreover, training the reconstruction in this way can train the encoder-decoder arrangement to force the most significant parts of the input for the reconstruction through the "information bottleneck" of the low-dimensional representation. This eliminates less important parts of the input, such as noise. Thus, the encoder also gets some de-noising functionality by virtue of the training.

[0033] Alternatively or in combination therewith, in another particularly advantageous embodiment, the action decoder network is chosen as the decoder part of an encoder-decoder arrangement that first maps the sequence of actions and / or the processing result derived from the sequence of actions to a representation and then reconstructs the sequence of actions and / or the processing result derived from the sequence of actions from the representation. The advantages are the same.

[0034] Reconstructing the "processing result" means that, for example, the encoder-decoder arrangement can be trained to predict the next action in the sequence of actions based on a part of the sequence of actions.

[0035] Examples of networks that can be used as encoder networks and / or decoder networks of the operational state and / or sequence of actions include recurrent neural networks, RNNs, and transformer networks. In a recurrent neural network, the output is fed back as input, and the network is run for a predetermined number of iterations. A transformer neural network comprises a sequence of multiple encoding units, where each encoding unit comprises an attention head that computes a correlation between different parts of the input. Both of these architectures are particularly useful for processing sequences as input.

[0036] For example, the state-to-action network can comprise a convolutional neural network. In a particularly advantageous embodiment, the state-to-action network comprises a fully connected neural network. This architecture offers the most flexibility for training, but at the cost of many parameters per size of input and output. As discussed previously, the dimensionality of the representation of the operational state and the dimensionality of the representation of the sequence of actions are both rather low. Therefore, a fully connected neural network can be accommodated with a larger number of parameters.

[0037] Exemplary plants where the method is particularly advantageous, especially for correcting abnormal situations, include continuous plants or process plants that are configured to issue alarms and event data. For example, the method can be used to correct abnormal situations in:

[0038] • a waste incineration plant;

[0039] • a hydrocarbon separation plant;

[0040] • a reinjection system for injecting water into a hydrocarbon well;

[0041] • a hydrocarbon extraction facility; and / or

[0042] • a de-glycol regeneration plant.

[0043] A commonality of these plants is that abnormal situations often pose a safety problem. In the case of a high-priority alarm, this can be made worse if the wrong sequence of actions is deployed or if an error is made when implementing the correct sequence, such as omitting a step or swapping the order of two steps, which can lose the last chance to regain control of the plant. Furthermore, because fortunately safety-critical abnormal situations are very rare, there is little training data for these situations. Therefore, it is advantageous that the method can use less training data, because as discussed previously, the main reasoning is between spaces that are rather low in two dimensions.

[0044] The invention also provides a method for training a network arrangement for use in the method described above.

[0045] During the method, a pre-trained first encoder-decoder arrangement of an action encoder network and an action decoder network is acquired. Furthermore, a pre-trained second encoder-decoder arrangement of a state encoder network and a state decoder network is acquired.

[0046] A sample of training data is acquired. Each such sample comprises values of a plurality of state variables characterizing an operating state of a plant or a part of a plant. These state variables are input data for the arrangement to be trained. Each sample further comprises a sequence of actions taken in response to the operating state. The sequence is a "ground truth" label attached to the operating state of the sample.

[0047] The values of the state variables in each sample are encoded by the pre-trained state encoder network into a representation of the respective operating state. The resulting representation of the operating state is mapped to a representation of the sequence of actions. The sequence of actions encoded in this representation is the sequence of actions the network arrangement is making given the operating state characterized by the state variables.

[0048] The correspondence of this sequence of actions to the "ground truth" attached to the sample is measured by means of a cost function. This can be achieved in either of the following two ways or a combination thereof:

[0049] According to a first way, the loss function measures how well the representation of the sequence of actions agrees with a representation obtained by encoding the sequence of actions in the training sample by the pre-trained action encoder network;

[0050] According to a second way, the loss function measures how well a sequence of actions obtained by decoding the representation of the sequence of actions by the pre-trained action decoder network agrees with the sequence of actions in the training sample.

[0051] During the training process, the parameters characterizing the behavior of the state-to-action network to be trained are optimized such that the rating by the loss function when processing other training samples is likely to improve.

[0052] The state variables can be acquired during actual execution of the process on the plant, or after such execution from a plant history record, or from a simulation using a process simulator that produces the same state variables as the process on the plant. The use of a process simulator is particularly beneficial for newly commissioned plants for which there is little historical data, and when the model is first trained on simulation data to capture the general process dynamics, and then trained using limited data from actual process execution. Likewise, the sequence of actions can be monitored during execution of the process, or can be acquired after such execution from an action log, or for initial training can be acquired from simulated experiments or predetermined sequences of actions by actual plant operators. Since both the state variables in the plant history record and the actions in the action log are time-stamped, they can be correlated with each other. Thus, the training can be understood as "mining" the workflow of the plant operators in reacting to a particular situation, and teaching the network arrangement to propose that workflow when such situation or a generally similar situation occurs again. In this way, even knowledge that exists in the mind of the operators but is difficult to express in words or to communicate to another operator can be used.

[0053] For example, if the operators learn "if the flame turns blue, open the valve slowly", the perception of the moment when the flame turns blue can be slightly different for each operator. Furthermore, the concept of "slowly" opening the valve can also be different for different operators. The present training method allows to automatically capture the knowledge without leaving room for interpretation.

[0054] In a particularly advantageous embodiment, acquiring a pre-trained first encoder-decoder arrangement of an action encoder network and an action decoder network comprises:

[0055] • acquiring training samples of sequences of actions;

[0056] • providing the sequence of actions in each training sample and / or a processing result derived therefrom to the action encoder network to be trained, thereby acquiring a representation;

[0057] • providing the representation to the action decoder network to be trained, thereby acquiring a sequence of actions and / or a processing result derived therefrom;

[0058] • measuring, by means of a predetermined loss function, how well the sequence of actions and / or the processing result agrees with the sequence of actions and / or the processing result in the training sample; and

[0059] • optimizing parameters characterizing the behavior of the action encoder network to be trained and the action decoder network to be trained, such that the rating by the loss function is likely to improve when processing further training samples.

[0060] The training samples for this pre-training can have the same training samples as the main training discussed above, but it can also be performed on a set of training samples that is disjoint from the set of training samples used for the main training. For example, the pre-training can be performed once in a generic way for a particular type of plant. For each instance of a plant that is subsequently installed, the pre-trained encoder-decoder arrangement can be used to train the state-to-sequence network. Optionally, when moving from the generic training to the specific instance of the plant, the pre-training of the encoder-decoder arrangement can be refined using other training samples taken from this instance of the plant.

[0061] The pre-training can be done using a variety of tasks for which "ground truth labels" can be easily produced from the available process state data and action sequences. Examples of such tasks are reconstructing the input (either the plant state variables or the action sequence), predicting the next n elements of the sequence (either the plant state variables or the action sequence), identifying the correct next sequence segment among several presented sequence segments, identifying the correct previous sequence segment among several presented sequence segments, identifying whether a presented sequence is a contiguous sequence in the overall sequence, etc. Such tasks can also be combined in parallel or sequentially, with the benefit that this can further increase the amount of training data used for the pre-training, and also prevent overfitting the pre-trained model to a single task.

[0062] The same advantages apply in an analogous way to another particularly advantageous embodiment in which the pre-trained second encoder-decoder arrangement of a state encoder network and a state decoder network comprises:

[0063] • obtaining training samples comprising values of a plurality of state variables characterizing an operating state of a plant or a part of a plant;

[0064] • providing the values of the state variables in each training sample to the state encoder network to be trained, thereby obtaining a representation;

[0065] • providing the representation to the state decoder network to be trained, thereby obtaining values of the state variables;

[0066] • measuring, by means of a predetermined loss function, how well the values agree with the values in the training samples; and

[0067] • optimizing parameters characterizing the behavior of the state encoder network to be trained and the state decoder network to be trained such that the rating by the loss function is likely to improve when processing further training samples.

[0068] In another particularly advantageous embodiment, the to-be-trained action encoder network and the to-be-trained state encoder network are combined in one single network architecture. The parameters on which such a single network architecture relies can be less than the combination of two separate architectures, and thus training can be easier. Moreover, since the tasks performed by the two networks have commonalities, the two trainings can benefit from each other to some extent by "sharing" knowledge in one single network architecture.

[0069] In another particularly advantageous embodiment, obtaining training samples for training the first encoder-decoder arrangement and / or the second encoder-decoder arrangement and / or the state-to-action network comprises aggregating training samples obtained in a plurality of industrial plants. This can increase the overall variability of the set of training samples, and thus provide accuracy of the final network arrangement. As discussed earlier, abnormal situations with safety risks are rare in any given plant. It is often not practical to induce such situations just for the purpose of gathering training samples, since there is a safety risk. However, in a larger fleet of plants, instances of abnormal situations occur sufficiently often that a reasonable amount of training samples can be collected.

[0070] As discussed earlier, these methods are computer-implemented methods. Therefore, the present application also relates to one or more computer programs having machine-readable instructions which, when executed on one or more computers and / or computing instances, cause the one or more computers to perform the methods described above. In this context, also virtualized platforms, hardware controllers, network infrastructure devices such as switches, bridges, routers or wireless access points, and end devices in the network such as sensors, actuators or other industrial field devices which are capable of executing machine-readable instructions are considered to be computers.

[0071] Therefore, the present application also relates to a non-transitory storage medium and / or a download product having one or more computer programs. A download product is a product which can be sold in an online store and which can be immediately implemented by downloading. The present application also provides one or more computers and / or computing instances having one or more computer programs and / or one or more non-transitory machine-readable storage media and / or a download product. BRIEF DESCRIPTION OF DRAWINGS

[0072] In the following, the present application is explained using the drawings, which are not intended to limit the scope of the present application. The drawings show:

[0073] Figure 1 is an exemplary embodiment of a method 100 for determining a suitable sequence of actions to be taken during operation of an industrial plant;

[0074] Figure 2is an exemplary embodiment of a method 200 for training a network arrangement for use in the method 100;

[0075] Figure 3 is an illustration of two ways in which the state-to-action network 4 can be trained. DETAILED DESCRIPTION

[0076] Figure 1 is a schematic flow chart of an embodiment of a method 100 for determining an appropriate sequence of actions to be taken during operation of an industrial plant 1.

[0077] In step 110, values of a plurality of state variables 2 characterizing an operational state of the plant 1 or a part of the plant 1 are acquired.

[0078] In step 120, the plurality of state variables 2 is encoded by at least one trained state encoder network 3 into a representation 2a of the operational state of the plant 1 or a part of the plant 1.

[0079] According to block 121, the state encoder network 3 can be chosen as an encoder part of an encoder-decoder arrangement that first maps the plurality of state variables 2 into the representation 2a of the operational state of the plant 1 or a part of the plant 1 and then reconstructs the plurality of state variables 2 from the representation 2a.

[0080] In step 130, the representation 2a of the operational state is mapped to a representation 6a of a sequence of actions 6 to be taken by a trained state-to-action network 4 in response to the operational state.

[0081] In step 140, the representation 6a of the sequence of actions 6 is decoded into the sought sequence of actions 6 to be taken by a trained action decoder network 5.

[0082] According to block 141, the action decoder network 5 can be chosen as a decoder part of an encoder-decoder arrangement that first maps a sequence of actions 6 and / or a processing result derived from the sequence of actions 6 into the representation 6a and then reconstructs the sequence of actions 6 and / or the processing result derived from the sequence of actions 6 from the representation 6a.

[0083] Figure 2 is a schematic flow chart of an embodiment of a method 200 for training an arrangement of networks 3, 4, 5 for use in the above described method 100.

[0084] In step 210, a pre-trained first encoder-decoder arrangement of an action encoder network 5# and an action decoder network 5 is acquired. In Figure 2 In the example shown, the acquisition comprises the following additional steps:

[0085] • according to block 211, training samples of action sequences 6 are acquired;

[0086] • according to block 212, the action sequence 6 and / or the processing results derived from the action sequence 6 in each training sample are provided to the action encoder network 5# to be trained, thereby acquiring a representation 6a;

[0087] • according to block 213, the representation 6a is provided to the action decoder network 5 to be trained, thereby acquiring an action sequence 6' and / or processing results derived from the action sequence 6';

[0088] • according to block 214, the degree of agreement of the action sequence 6' and / or the processing results with the action sequence 6 and / or the processing results in the training sample is measured by means of a predetermined loss function 9; and

[0089] • according to block 215, the parameters 5a#, 5a representing the behavior of the action encoder network 5# to be trained and the action decoder network 5 to be trained are optimized so that the rating 9a by the loss function 9 is very likely to improve when processing further training samples. The final optimized state of the parameters 5a#, 5a is marked with reference 5a*#, 5a*.

[0090] In step 220, a pre-trained second encoder-decoder arrangement of the state encoder network 3 and the state decoder network 3# is acquired. In the example shown, this acquisition comprises the following additional steps: Figure 2

[0091] • according to block 221, training samples are acquired, which comprise values of a plurality of state variables 2 representing the operating state of the plant 1 or a part of the plant 1;

[0092] • according to block 222, the values of the state variables 2 in each training sample are provided to the state encoder network to be trained, thereby acquiring a representation 2a;

[0093] • according to block 223, the representation 2a is provided to the state decoder network 3# to be trained, thereby acquiring values 2' of the state variables;

[0094] • according to block 224, the degree of agreement of the values 2' with the values 2 in the training sample is measured by a predetermined loss function 10; and

[0095] • according to block 225, the parameters 3a, 3a# representing the behavior of the state encoder network 3 to be trained and the state decoder network 3# to be trained are optimized so that the rating 10a by the loss function 10 is very likely to improve when processing further training samples. The final optimized state of the parameters 3a, 3a# is marked with reference 3a*, 3a#*.

[0096] ​In step 230, samples 7 of training data are acquired. Each sample 7 comprises values of a plurality of state variables 2 characterizing an operating state of the plant 1 or a part of the plant 1 and a sequence of actions 6* taken in response to the operating state.

[0097] According to block 231, acquiring 230 the training samples 7 for training the first encoder-decoder arrangement and / or the second encoder-decoder arrangement and / or the state-to-action network 4 comprises aggregating training samples acquired in a plurality of industrial plants 1.

[0098] In step 240, the values of the state variables 2 in each sample 7 are encoded into a representation 2a of the respective operating state by means of a pre-trained state encoder network 3.

[0099] In step 250, the representation 2a of the operating state is mapped to a representation 6a of the sequence of actions 6.

[0100] In step 260, the degree of agreement of the representation 6a of the sequence of actions with a representation 6a* acquired by encoding the sequence of actions 6 in the training sample 7 by means of a pre-trained action encoder network 5# is measured by means of a predetermined loss function 8; and / or

[0101] • the degree of agreement of the representation 6a of the sequence of actions with a representation 6a* acquired by encoding the sequence of actions 6 in the training sample 7 by means of a pre-trained action encoder network 5#; and / or

[0102] • the degree of agreement of a sequence of actions 6* acquired by decoding the representation 6a of the sequence of actions 6 by means of a pre-trained action decoder network 5 with the sequence of actions 2* in the training sample 7;

[0103] In step 270, the parameters 4a characterizing the behavior of the state-to-action network 4 to be trained are optimized such that the rating 8a by the loss function 8 when processing further training samples 7 is very likely to improve. The final optimized state of the parameters 4a is labeled using reference sign 4a*.

[0104] Figure 3 Two ways in which the state-to-action network 4 can be trained are illustrated.

[0105] The values of one or more state variables 2 from the training sample 7 are encoded into a representation 2a of the operating state of the plant 1. By means of the state-to-action network 4, this representation 2a is mapped to a representation 6a of the sequence of actions 6. This representation 6a needs to be compared to the “ground truth” sequence of actions 6* in the training sample 7 in order to measure the degree of correctness of the representation 6a output by the state-to-action network 4.

[0106] In the first way, the action “ground truth” sequence 6* is encoded into a “ground truth” representation 6a* by means of a pre-trained action encoder network 5#. The loss function 8 measures the degree of agreement of the representation 6a output by the state-to-action network 4 with the “ground truth” representation 6a*.

[0107] In the second approach, the representation 6a output by the state-to-action network 4 is decoded by a pre-trained action decoder network 5 in order to obtain an action sequence 6'. Then, a loss function 8 measures how well this action sequence 6 agrees with an action "ground truth" sequence 6*.

[0108] List of reference signs

[0109] 1 industrial plant

[0110] 2 state variables characterizing the operating state of the plant 1

[0111] 2' state variables decoded during encoder-decoder training

[0112] 2a representation of the operating state of the plant 1

[0113] 3 state encoder network

[0114] 3a parameters characterizing the behavior of the network 3

[0115] 3a* final optimized state of the parameters 3a

[0116] 3# state decoder network

[0117] 3a# parameters characterizing the behavior of the network 3#

[0118] 3a#* final optimized state of the parameters 3a#

[0119] 4 state-to-action network

[0120] 4a parameters characterizing the behavior of the network 4

[0121] 4a* final optimized state of the parameters 4a

[0122] 5 action decoder network

[0123] 5a parameters characterizing the behavior of the network 5

[0124] 5a* final optimized state of the parameters 5a

[0125] 5# action encoder network

[0126] 5a# parameters characterizing the behavior of the network 5#

[0127] 5a#* final optimized state of the parameters 5a#

[0128] 6 action sequence

[0129] 6' sequence decoded during encoder-decoder training

[0130] 6* action sequence in the training sample 7

[0131] 6a representation of a sequence of actions 6

[0132] 6a* representation encoded from sequence 6*

[0133] 7 training samples for training the state-to-action network 4

[0134] 8 loss function for training the state-to-action network 4

[0135] 8a rating by the loss function 8

[0136] 9 loss function for the action encoder-decoder arrangement 5, 5

[0137] 9a rating by the loss function 9

[0138] 10 loss function for the state encoder-decoder arrangement 3, 3

[0139] 10a rating by the loss function 10

[0140] 100 method for determining a suitable sequence of actions 6

[0141] 110 obtaining state variables 2

[0142] 120 encoding the state variables into a representation 120

[0143] 121 selecting a state encoder 3 from an encoder-decoder arrangement

[0144] 130 mapping the representation 2a to a sequence representation 6a

[0145] 140 decoding the representation 6a into a sought sequence 6

[0146] 141 selecting an action decoder 5 from an encoder-decoder arrangement

[0147] 200 method for training an arrangement of networks 3, 4, 5

[0148] 210 obtaining a pre-trained first encoder-decoder arrangement 5, 5

[0149] 211 obtaining training samples of sequences 6

[0150] 212 providing the training sequences 6 to the action encoder network 5

[0151] 213 providing the representation 6a to the action decoder network 5

[0152] 214 rating the decoded sequence 6' with the loss function 9

[0153] 215 obtaining parameters 5a#, 5# of the networks 5a, 5

[0154] 220 obtaining a pre-trained second encoder-decoder arrangement 3, 3#

[0155] 221 obtaining training samples of state variables 2

[0156] 222 providing state variables 2 to a state encoder network 3

[0157] 223 providing representations 2a to a state decoder network 3#

[0158] 224 rating values 2' with a loss function 10#

[0159] 225 optimizing parameters 3a, 3a# of the networks 3, 3#

[0160] 230 obtaining samples 7 of training data

[0161] 231 aggregating training samples 7 across plants

[0162] 240 encoding training state variables 2 into representations 2a

[0163] 250 mapping state representations 2a to sequence representations 6a

[0164] 260 rating sequence representations 6a / sequences 6 with a loss function 8

[0165] 270 optimizing parameters 4a of the state-to-action network 4

Claims

1. A computer-implemented method (100) for determining an appropriate sequence (6) of actions to be taken during operation of an industrial plant (1) or a portion thereof, comprising the steps of: • Obtain the values ​​of multiple state variables (2) that characterize the operational state of the industrial plant (1) or a portion thereof; • The plurality of state variables (2) are encoded by at least one trained state encoder network (3) into a representation (2a) of the operating state of the industrial plant (1) or a part thereof, wherein the representation (2a) of the operating state depends on fewer variables than the operating state of the industrial plant (1). • The trained state-to-action network (4) maps the representation (2a) of the operation state to the representation (6a) of the action sequence (6) to be taken in response to the operation state, wherein the representation (6a) of the action sequence (6) depends on fewer variables than the action sequence (6). as well as • The representation (6a) of the action sequence (6) is decoded into the action sequence (6) to be taken by the trained action decoder network (5). The plurality of state variables (2) mentioned therein include one or more of the following: pressure, temperature, mass flow, voltage, current, fill level and / or concentration of substances in the mixture; The action sequence mentioned above includes one or more of the following: Enable or disable equipment in the industrial plant (1) or a part of the industrial plant (1); To open or close valves in the industrial plant (1) or a portion thereof; and Change the setpoint value of at least one low-level controller in the industrial plant (1) or a part of the industrial plant (1).

2. The method (100) according to claim 1, wherein the state encoder network (3) is selected as the encoder part of the encoder-decoder device, the encoder-decoder device first maps a plurality of state variables (2) to a representation (2a) of the operating state of the industrial plant (1) or a part of the industrial plant (1), and then reconstructs the plurality of state variables (2) from the representation (2a) of the operating state.

3. The method (100) according to claim 1 or 2, wherein the action decoder network (5) is selected as the decoder part of the encoder-decoder device, the encoder-decoder device first maps the action sequence (6) and / or the processing result derived from the action sequence (6) to a representation (6a) of the action sequence (6), and then reconstructs the action sequence (6) and / or the processing result derived from the action sequence (6) from the representation (6a) of the action sequence (6).

4. The method (100) according to claim 1 or 2, wherein the state encoder network (3) and / or the action decoder network (5) comprises a recurrent neural network (RNN) and / or a converter neural network.

5. The method (100) according to claim 1 or 2, wherein the state-to-action network (4) comprises a convolutional neural network and / or a fully connected neural network.

6. The method (100) according to claim 1 or 2, wherein the industrial plant (1) or a portion thereof is a continuous plant or process plant configured to issue alarms and event data.

7. The method (100) according to claim 1 or 2, wherein the industrial plant (1) or a portion thereof comprises one or more of the following: • Waste incineration plant; • Hydrocarbon separation plant; • An injection system used to inject water into hydrocarbon wells; • Hydrocarbon extraction facilities; and / or • Deglycolization and regeneration plant.

8. A computer-implemented method (200) for training a network (3, 4, 5) apparatus for use in the method (100) according to any one of claims 1 to 7, comprising the following steps: • Obtain a pre-trained first encoder-decoder device of the action encoder network (5#) and the action decoder network (5); • Obtain a pre-trained second encoder-decoder device for the state encoder network (3) and the state decoder network (3#); • Obtain training data samples (7), wherein each training data sample (7) includes the values ​​of multiple state variables (2) characterizing the operating state of the industrial plant (1) or a part of the industrial plant (1) and a sequence of actions taken in response to the operating state. ); • The values ​​of the plurality of state variables (2) in each training data sample (7) are encoded by the pre-trained state encoder network (3) into a representation (2a) of the corresponding operating state, wherein the representation (2a) of the operating state depends on fewer variables than the corresponding operating state. • The action network (4) from the state to be trained maps the representation (2a) of the operation state to the representation (6a) of the action sequence (6), wherein the representation (6a) of the action sequence (6) depends on fewer variables than the action sequence (6). •Measured using a first predetermined loss function (8) The representation (6a) of the action sequence and the action sequence in the training data sample (7) processed by the pre-trained action encoder network (5#) The representation obtained by encoding () The degree of consistency between ) and / or ○ The action sequence (6) obtained by decoding the representation (6a) of the action sequence (6) by the pre-trained action decoder network (5) and the action sequence in the training data sample (7) The degree of consistency between ) Thus, a rating (8a) was obtained; and • The parameters (4a) that characterize the behavior of the state to be trained to the action network (4) are optimized such that the rating (8a) by the first predetermined loss function (8) is likely to be improved when other training data samples (7) are processed. The plurality of state variables (2) mentioned therein include one or more of the following: pressure, temperature, mass flow, voltage, current, fill level and / or concentration of substances in the mixture; The action sequence mentioned above includes one or more of the following: Enable or disable equipment in the industrial plant (1) or a part of the industrial plant (1); To open or close valves in the industrial plant (1) or a portion thereof; and Change the setpoint value of at least one low-level controller in the industrial plant (1) or a part of the industrial plant (1).

9. The method (200) according to claim 8, wherein the pre-trained first encoder-decoder device for acquiring the action encoder network (5#) and the action decoder network (5) comprises: • Obtain training data samples of the action sequence (6); • Provide the action encoder network (5#) to be trained with the action sequence (6) in each training data sample and / or the processing result derived from the action sequence (6) to obtain the representation (6a) of the action sequence (6). • Provide the representation (6a) of the action sequence (6) to the action decoder network (5) to be trained, thereby obtaining the action sequence (6') and / or the processing result derived from the action sequence (6'). • The degree of consistency between the action sequence (6') and / or the processing result and the action sequence (6) and / or the processing result in the training data sample is measured by means of a second predetermined loss function (9); as well as • Optimize the parameters (5a#, 5a) that characterize the behavior of the action encoder network (5#) and the action decoder network (5) to be trained, such that the rating (9a) by the second predetermined loss function (9) is likely to be improved when other training data samples are processed.

10. The method (200) of claim 8, wherein the pre-trained second encoder-decoder device for obtaining the state encoder network (3) and the state decoder network (3#) comprises: • Obtain training data samples, which include the values ​​of multiple state variables (2) that characterize the operational state of the industrial plant (1) or a portion thereof; • Provide the state encoder network (3) to be trained with the values ​​of the plurality of state variables (2) in each training data sample to obtain a representation (2a) of the corresponding operating state. • Provide the representation (2a) of the operational state to the state decoder network (3#) to be trained, thereby obtaining the value (2') of the state variable; • The degree of consistency between the value (2') and the value in the training data sample is measured by means of a third predetermined loss function (10); as well as • Optimize the parameters (3a, 3a#) that characterize the behavior of the state encoder network (3) and the state decoder network (3#) to be trained, such that the rating (10a) by the third predetermined loss function (10) is likely to be improved when other training data samples are processed.

11. The method (200) according to claim 9 or 10, wherein the action encoder network (5#) to be trained and the state encoder network (3) to be trained are combined in a single network architecture.

12. The method (200) according to any one of claims 9 to 10, wherein obtaining training data samples (7) for training the first encoder-decoder device and / or the second encoder-decoder device and / or the state-to-action network (4) comprises: The training data samples were compiled from multiple industrial plants (1).

13. One or more computer programs comprising machine-readable instructions that, when executed on one or more computers, cause the one or more computers to perform the method (100, 200) according to any one of claims 1 to 12.

14. A non-transitory storage medium and / or download product having the computer program according to claim 13.

15. One or more computers having a computer program according to claim 13 and / or a non-transitory storage medium and / or a download product according to claim 14.

Images