Methods and systems for optimal operation of industrial processes

Abstract

A method for autonomous management of operation of an industrial process including a control layer to control equipment of the industrial process is provided. The method includes providing operational data; providing at least one process goal; assessing the operational data against the at least one process goal to provide a performance assessment (130) of the industrial process; providing the performance assessment to a supervisory layer including a decision-making engine having a memory device coupled with a processor; and autonomously selecting, by the decision-making engine, an action to minimize a difference between the performance assessment and the at least one process goal, to improve performance of the industrial process. The action includes at least one of (i) updating a model used in the industrial process and (ii) generating updated training data related to the industrial process for a model used in the industrial process.

Description

[0001] METHODS AND SYSTEMS FOR OPTIMAL OPERATION OF INDUSTRIAL PROCESSES

[0002] FIELD OF THE INVENTION

[0003] [1] The present disclosure is directed to methods and systems for autonomous optimization of industrial production processes, which may include but not be limited to the production of liquefied natural gas and / or hydrocarbons refining.

[0004] BACKGROUND OF THE INVENTION

[0005] [2] In complex industrial production processes, including the petrochemical processing such as LNG production and / or hydrocarbons refining, maintaining process performance, preferably under optimized conditions, is often desirable to maximize output while minimizing operating expenses (OpEx). To facilitate optimized production operation, such complex industrial processes typically employ some form of computerized control component in its operation to efficiently coordinate adjustments, particularly in response to changing conditions in an effort to maintain optimized production conditions. One of the examples for such computerized control components include a distributed control system (DCS), which provides a top-down cascade of adjustments using a network of computers. Instructions from the DCS are deployed throughout a plant and fed to individual controllers. Another example is the Supervisory Control and Data Acquisition (SCADA), which is a computer-based system for gathering and analyzing real-time data to monitor and control equipment that deals with critical and time-sensitive materials or events. Both DCS and SCADA, as well as other similar computerized plant control systems in the control layer, can be used to automate the control processes in complex industrial production processes. They, however, have their limits in facilitating optimized production because they are primarily programmed to respond to local conditions, such as increase or decrease in temperature, pressure, etc., rather than to changes that take place in other parts of the process.

[0006] [3] Generally speaking, to further enable optimized production, many industrial facilities additionally employ computerized control components (or process controllers) that are based on models, such as advanced process control (APC) and real-time optimizers (RTOs). Model-based process controllers such as APC and RTO typically use one or more predictive models to mathematically represent how one or more properties within an industrial process respond to changes made to the industrial process. Usually, theoretical “full-scale” first-principles models are used for offline simulation such as plant design and debottlenecking as well as for online applications like monitoring and optimization. These “full-scale” models may consist of thousands to millions of mathematical equations representing physical and chemical properties as well as mass and energy balances in a chemical process under consideration. In many cases, these full- scale models may not capture all of the physical phenomena as the mechanisms cannot be described mathematically or as simplifications are necessary for a tractable solution.

[0007] [4] These model-based control components may provide improved production optimization, an example of which is disclosed in US20230083389A1. Nevertheless, they also suffer from certain limitations. For instance, model-based controllers typically depend on having accurate models of a process's behavior in order to perform well and effectively control the process. As conditions change in the process, a controller's models typically need to be updated. Further, calibration and online execution of such a “full-scale” model can be challenging in terms of cost and sustainability, which has limited the applications in the industrial production processes.

[0008] [5] In particular, methods such as APCs and RTOs often require a dynamic model, and / or gain matrix updates if there is significant process change related dynamic behaviour, and this is despite such systems introducing a high degree of automation in the plant over fundamental base layer control. RTOs are designed to send set-points to the APC layer to optimize production or energy efficiency (which are often set as primary objectives / economic functions of the APC). However, for fast moving processes such as LNG, the time period during which RTOs can send updates is typically restricted to after the equipment or process is stable or in steady state. Not being able to send updates outside of the stable operation (such as during startup) can lead to missed optimization horizons, which can result in sub-optimal solutions or the solution getting rejected by the APC. Furthermore, these RTOs are mainly for steady state targets, and are unable to optimize for transient operating regimes of the plants. RTOs are typically based on rigorous physics based flow-sheeter models, which require significant & skilled resource to maintain these models for process unit optimization increasing OpEx of these technology solutions.

[0009] [6] Recently, with the emergence of artificial intelligence (Al), particularly machine learning, the process industry has an opportunity to develop and implement asset optimization with embedded Al. For example, US11853032 discloses computer-based process modeling and simulation methods and systems combine first principles models and machine learning models to benefit where either model is lacking; and US20200387818A1 discloses system and methods that provide a new paradigm for solving process system engineering (PSE) problems using embedded artificial intelligence (Al) techniques to facilitate process model building and deployment. Further, US20200166909A1 discloses machine learning-based methods and systems for automated object defect classification and adaptive, real-time control of manufacturing processes; and US11644816 discloses real-time intervention of an industrial process, which can include searching for a batch of candidate configurations for use by the industrial process, the batch of candidate configurations searched for by performing a batch Bayesian optimization (BBO). “Masksembles for Uncertainty estimation” discloses an alternative to the infinite number of functions captured by the Gaussian processes regression kernels. Durasov, N., Bagautdinov, T., Baque, P. and Fua, P., 2021. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (pp. 13539-13548).

[0010] [7] Although these references leverage machine learning for predictive model building and / or updating of the predictive models, they do not contemplate use of machine learning to enable autonomous operational decision-making or autonomous optimization of an industrial process. As such, there is still a need for improved methods and systems for optimal operation of complex industrial production processes, particularly beyond the capabilities of known RTOs.

[0011] SUMMARY OF THE INVENTION

[0012] [8] Accordingly, there is provided a computer-implemented method for autonomous management of operation of an industrial process having a control layer to control equipment of the industrial process. The method includes providing operational data related to the industrial process; providing at least one process goal for the industrial process; assessing the operational data against the at least one process goal to provide a performance assessment of the industrial process; and providing the performance assessment to a supervisory layer having a decisionmaking engine having a memory device coupled with a processor. The supervisory layer is capable of (including configured to) modifying one or more settings of the control layer and / or the decision-making engine. The supervisory layer is also capable of autonomously selecting, by the decision-making engine, an action to minimize a difference between the performance assessment and the at least one process goal, to improve performance of the industrial process. The action includes at least one of (i) updating a model used in the industrial process and (ii) generating updated training data related to the industrial process for a model used in the industrial process.

[0013] [9] Optionally, the step of assessing the operational data includes determining whether operational data includes new data. If it is determined operational data includes new data, autonomously selecting, by the decision-making engine, a model to be updated and autonomously selecting, by the decision-making engine, one or more updates to implement to update the selected model. Optionally, the step of determining whether operational data includes new data includes providing a criterium against which to determine whether the provided operational data includes new data, optionally, the supervisory layer autonomously selects the criterium; assessing the provided operational data against historical operational data with respect to the criterium to provide a geometrical distance between the provided operational data and the historical data; and assessing, by the supervisory layer, at least the geometrical distance with respect to the criterium to determine whether the provided operational data includes new data.

[0014]

[0010] Optionally, the method further includes determining whether the operational data indicates steady state operation for a selected period of time. If it is determined the operational data indicates steady state operation for the selected period of time, comparing the operational data during the selected period of time against historical steady state operational data.

[0015]

[0011] Optionally, the supervisory layer includes a machine learning algorithm (MLA), and the step of generating updated training data further includes generating updated training data related to the industrial process for the MLA. Optionally, the generating step includes exciting the industrial process at least by implementing at least one pre-determined change to one or more selected MVs and / or CVs; and tagging operational data generated from the excitation to provide tagged operational data. Optionally, the method further includes providing the tagged operational data as the updated training data. Optionally, the tagging includes providing data associated with operation of the control layer in smooth signal format; and providing data associated with operation of the industrial process in stepped signal format.

[0016]

[0012] Optionally, the method further includes observing an impact to the industrial process by the excitation; and correlating the observed impact to the at least one pre-determined change using at least the tagged operational data. Optionally, the method further includes selecting timing and duration of a period of time to perform the steps of exciting and tagging. Optionally, the supervisory layer autonomously selects the timing and duration of the period of time to perform the steps of exciting and tagging.

[0017]

[0013] According to another aspect, there is provided a computer-based system for autonomous management of operation of an industrial process having a control layer to control equipment of the industrial process. The system includes a monitoring engine to (a) receive operational data related to the industrial process; (b) receive at least one target goal for the industrial process; and (c) assess the operational data against the at least one target goal to provide a performance assessment of the industrial process. The system further includes a supervisory layer to receive the performance assessment. The supervisory layer includes a decision-making engine. The supervisory layer is capable of modifying one or more settings of the control layer and / or the decision-making engine, and the supervisory layer may autonomously select an action an action to minimize a difference between the performance assessment and the at least one process goal, to improve performance of the industrial process, the action includes at least one of (i) updating a model used in the industrial process and (ii) generating updated training data related to the industrial process for a model used in the industrial process. Optionally, the decision-making engine is capable of determining whether operational data includes new data. If it is determined operational data includes new data, autonomously selecting, a model to be updated and one or more updates to implement to update the selected model. Optionally, the supervisory layer includes a machine learning algorithm (MLA), and the generating updated training data further includes generating updated training data related to the industrial process for the MLA. Optionally, the decision-making engine may autonomously select timing and duration of period of time to perform: exciting the industrial process at least by implementing at least one predetermined change to one or more selected MVs and / or CVs; and tagging operational data generated from the excitation to provide tagged operational data.

[0018] BRIEF DESCRIPTION OF THE DRAWINGS

[0019]

[0014] The drawing figures depict one or more implementations in accordance with the present teachings, by way of example only, not by way of limitation. In the figures, like reference numerals refer to the same or similar elements.

[0020]

[0015] FIG. 1 is a depiction of a control hierarchy of the equipment at a facility, with a supervisory layer according to aspects disclosed herein located at the apex over a control layer.

[0021]

[0016] FIG. 2 is a flow diagram illustrating various components shown in FIG. 1 in operation according to aspects disclosed herein.

[0022]

[0017] FIG. 3 illustrates a schematic of a mixed refrigerant loop of an LNG plant that may benefit from optimal operation by a supervisory layer according to aspects disclosed herein.

[0023] DETAILED DESCRIPTION OF THE INVENTION

[0018] The present invention will now be described in detail with reference to embodiments thereof as illustrated in the accompanying drawings. References to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described. Other suitable modifications and adaptations of the variety of conditions and parameters normally encountered in the field, and which would be apparent to those skilled in the art, are within the spirit and scope of the invention.

[0024]

[0019] Although the description herein provides numerous specific details that are set forth for a thorough understanding of illustrative embodiments, it will be apparent to one skilled in the art that embodiments may be practiced without some or all of these specific details. In other instances, well known process steps and / or structures have not been described in detail in order to not unnecessarily obscure the present invention. The features and advantages of embodiments may be better understood with reference to the drawings and discussions that follow.

[0025]

[0020] In addition, as noted above, when like elements are used in one or more figures, identical reference characters will be used in each figure, and a detailed description of the element will be provided only at its first occurrence. Some features or components of the systems or processes described herein may be omitted in certain depicted configurations in the interest of clarity.

[0026]

[0021] In general, a machine learning model or algorithm receives input and generates output based on its received input and on values of model parameters. Also, machine learning provides a powerful mechanism to incorporate data into process models. Machine learning algorithms are typically easy to automate and can continuously improve as more data becomes available. In addition, these algorithms are good at handling multi-dimensional data and datasets containing different data types.

[0027]

[0022] The present disclosure provides systems and methods to operate and / or optimize an industrial manufacturing process (such as liquefaction of natural gas), where the settings for optimized operations are provided by a supervisory component employing a machine learning model or algorithm. The systems and methods disclosed herein are particular applications of a broad framework disclosed in Korkmaz, B.S., Zagorowska, M. and Mercangdz, M., 2023. Safe optimization of an industrial refrigeration process using an adaptive and explorative framework. IFAC-PapersOnLine, 56(2), pp.1400-1405.

[0028]

[0023] Non-limiting examples of facility 100 that can benefit from the embodiments disclosed herein include gas processing facilities, GTL, Chemicals and Refineries. An LNG plant to liquefy a natural gas feed stream is particularly suitable to benefit from the embodiments disclosed herein. The one or more processes at such facility include cooling the natural gas until it reaches the liquid state (or liquefaction process). The equipment supporting the liquefaction process typically includes one or more heat exchangers, including the last heat exchanger for cooling the natural gas to the liquid state, which is typically referred to as the main cryogenic heat exchanger (MCHE). While the present disclosure at times references the natural gas liquefaction process explicitly as an exemplary application of the described systems and methods, it is understood that the principles disclosed herein can be applied to other industrial manufacturing processes as known to one of ordinary skill.

[0029]

[0024] Accordingly, there is provided a system for autonomous management of operation of an industrial process having a control layer to control equipment associated therewith. The system includes a monitoring engine to receive operational data related to the industrial process, to receive at least one target goal for the industrial process; and to assess the operational data against the at least one target goal to provide a performance assessment of the industrial process. The system further includes a supervisory layer to receive the performance assessment, where the supervisory layer is capable of modifying one or more settings of the control layer and optionally includes the monitoring engine, and where the supervisory layer autonomously select an action to minimize a deviation between the performance assessment and the at least one target goal, thereby progressing performance of the industrial process. The action includes at least one of (i) a modification of one or more settings of at least one of: the control layer, and the decision-making engine, and (ii) a modification of one or more settings related to overall performance of a plant running the industrial process, and (ii) an update to at least one of: the predictive model and a model used by the decisionmaking engine in performing the autonomous selection. Reference to “minimizing a deviation” or “difference” is intended to be broadly interpreted where it can include reducing a numerical value, such as reducing a difference between a measured pressure and a target pressure, or it can refer broadly to maximizing or minimizing a function, such as selecting one or more actions or settings to maximize production or minimize the amount of power being used. Reference to “capable of’ an action can also include being configured to perform that action.

[0030]

[0025] FIG. 1 is a diagram illustrating a control hierarchy of the equipment (e.g., valves and flowrates, etc.) at facility 100, with supervisory layer 104 sitting at the apex over a control layer 102. FIG. 1 also includes certain optional components to facilitate understanding of the embodiments disclosed herein. The term “apex” and “on top” refers to a hierarchical arrangement where the supervisory layer 104 is capable of and preferably configured to adjust one or more settings of the control layer 102. The supervisory layer continually receives collected information about operation parameters in the facility and provides instructions to a lower layer (e.g., an optimization layer) based on the collected information, whereby the lower layer adjusts one or more operational parameters based on the instructions from the supervisory layer. Example settings include controller setpoints, objective functions used in optimizations (such as / in the equations set forth herein), including weights and parameters of objective functions, horizon lengths, sampling rates, controller tuning settings, constraints and constraint values (such as g and h, respectively in the equations set forth herein), constraint priorities and others. The interfacing components, such as a control layer 102, however, can provide data to the supervisory layer but they are not able to directly change the settings of the supervisory layer 104. In preferred circumstances, the supervisory layer 104 can change its own settings based on the received data and / or via its own input interface, which provides access for human interaction for example for system designers, process engineers, or operators.

[0031]

[0026] Non-limiting examples of suitable components in the control layer 102 include APC (Advanced Process Control), SCADA (Supervisory Control and Data Acquisition), and DCS (Distributed Control Systems). Advanced Process Control (APC) and Real Time Optimization (RTO) are standard tools applied to stabilize and optimize complex processes in real time (Seborg, D.E., Edgar, T.F., Mellichamp, D.A. and Doyle III, F.J., 2016. Process dynamics and control. John Wiley & Sons.). LNG plants typically use these tools as well in its operation. Typically, control layer 102 includes a base process, such as a distributed control system (DCS), which is below (controlled by) the APC or SCADA. An example of how a base process can be controlled can be found in, for instance, U.S. Pat. Nos. 7,266,975 and 6,272,882. Many process variables that can be adjusted (such as valve positions) are on an even lower level controlled by the base process, such as via communication channels 1 16 and 118. In short, components in the control layer 102, including the base process layer, control these process variables. Some of these components, including the APC and RTO tools are often model-based, which means they have adjustable hyperparameters or settings that can be strategically selected to further exploit the outputs of these tools, thereby leading to improved plant performance. As such, supervisory layer 104 can control the underlying process (e.g., NG liquefaction) by autonomously selecting values for the control layer.

[0032]

[0027] Control layer 102 can optionally and preferably comprise model-based optimization 120, which can be exploited by an APC and / or RTO as known by one of ordinary skill. The modelbased optimization 120 typically includes one or more optimization solvers exploring for solutions in predictive models to generate an output of optimal operational settings. These models may contain thousands to millions of mathematical equations representing physical and chemical properties as well as mass and energy balances in an industrial process under consideration. These are sometimes called predictive models because they can be used to “predict” a likely process performance outcome based on certain operational settings inputs from the optimization solver(s). The more accurate (or high fidelity) the model, the closer the prediction is to the actual outcome. These predictive model(s) and optimization solver(s) used by the control layer 102 have settings that are adjustable, and hence can be autonomously managed by the supervisory layer 104 in achieving an operational target goal, whether for steady state operation or for other more dynamic operation such as startup. As used herein, the term “autonomous” is inclusive and refers to both partial and complete autonomy or autonomous function (i.e., without direct human intervention) by the supervisory layer.

[0033]

[0028] Implementation may be real time, offline, or at periodic intervals. The interface between the supervisory layer and the control layer and / or other components of the facility can be implemented for both offline and online operation. The operational settings (solution vectors) can be implemented in the control layer, which in turn uses the solution vectors to adjust the adjustable process variables to achieve the selected objective.

[0034]

[0029] Control layer 102 may be communicatively coupled to suitable instruments that collect operational data at various sampling periods for the measurable process variables (such as, regularly and frequently: e.g., one sample per minute, intermittently: e.g., 20-30 minutes per sample, and / or prompted by an event: e.g., reaching a threshold). The instruments may communicate the collected data to an instrumentation computer, which may in turn communicate the collected data to a data server via suitable communications means.

[0035]

[0030] Although the data collected varies according to the type of facility and corresponding process(es), some typical measurable operational data that are collected may include a feed stream flow rate as measured by a flow meter, a feed stream temperature as measured by a temperature sensor, component feed concentrations as determined by an analyzer, and reflux stream temperature in a pipe as measured by a temperature sensor. The collected data may also include measurements for process output stream variables, such as the concentration of produced materials. The collected data may further include measurements for manipulated input variables, such as various flow rates of processing streams (refrigerants, products, etc.) as set by certain valves and determined by corresponding flow meters, and various pressures as controlled by corresponding valves. The collected data can reflect the operating conditions of the plant or facility 100 during a particular sampling period.

[0036]

[0031] The system includes monitoring engine 122 to receive data relevant to the operation of facility 100, such as operational data 108 related to the industrial process and at least one target goal 106 for the industrial process. Reference to 108 includes 108 A and 108B together or individually, as context dictates. Suitable process goals can include certain selected or set performance outputs for the one or more processes to be optimized, which may be referred to as the “target process” for convenience without intending to impose any restrictions. Examples of suitable process or target goals include key performance indicators (KPIs), such as production rates, throughput, continuous operational hours, frequency of intervention by the operator, efficiency, amount of time to reach steady state, amount of time during shutdown, amount of time to reach a target, or specific such as flow rate targets, pressure targets, temperature targets, profit rate (from operations), energy or fuel consumption, greenhouse gas (GHG) production, inventory, etc.

[0037]

[0032] The format of one or more process goals 106 can be pre-designed numerical values (for example corresponding to production rates, product quality specifications or other more complex performance descriptions such as not operating more than a temperature threshold for longer than a time threshold or a general definition of risk the supervisor is allowed to take for example based on confidence intervals to be used in the algorithms for satisfying safety constraints). Optionally, process goals 106 can be provided as natural language inputs for interpretation by supervisory layer 104. Alternatively, supervisory layer 104 can receive process targets 106 from other algorithmic and computational entities or agents representing other functions, such as enterprisewide operational optimization or from external interfaces, such as a maximum allowed power consumption limit communicated by a power grid operator in connection with the process plant or price levels or orders for products produced in the process plant.

[0038]

[0033] Process goals 106 can be provided at regular intervals, on-demand, or event-based such as a price change- depending on the requirements of the operation of facility 100. Optionally, certain process goals 106 can be provided directly to control layer 102 where these may not be set by the supervisory layer 104 but the parameters around which to achieve process goals 106 can still be selected and provided to the control layer 102 by the supervisory layer 104.

[0039]

[0034] Preferably, operational data 108 includes the collected operational data as described herein, such as collected by control layer 102. Operational data 108 can come from facility 100 and / or control layer 102 as operational data 108 A, which may be considered online or real-time operational data, meaning supervisory layer 104 is updated during the sampling period. Additionally or alternatively, operational data 108 may be provided offline or outside of the sampling period (indicated as 108B), such as from a data server of the control layer 102 or a digital twin of the one or more processes. In addition to the operational data that can be collected with various instruments as described herein and known to one of ordinary skill, operational data 108 can preferably further include control settings of control layer 102, such as predictive model parameters. The operational data 108 may be provided from control layer 102 and / or indirectly from a database.

[0040]

[0035] While FIG. 1 depicts monitoring engine 122 as part of supervisory layer 104, it is understood this is only for illustrative purpose as monitoring engine 122 can be located at any other suitable location or configuration as long as it provides the suitable outputs to supervisory layer 104. Monitoring engine 122 may receive operational data 108 and / or at least one target goal 106 directly or via another engine, such as interface engine 124.

[0041]

[0036] Interface engine 124 can provide communications between supervisory layer 104 and facility 100, as well as other external entities such as an operator or components external to the facility, such as a utility grid. Interface engine 124 can include a web portal or a web browser interface, a front-end user interface for an operator to use to communicate with supervisory layer 104 as desired according to aspects disclosed herein, such as to (i) provide the process goal 106, optimization objective f for the optimization problem, or constraint limits h, (ii) start an optimization session or to initiate startup of the MCHE, (iii) override parameters previously selected by the supervisory layer such as a confidence interval to be used by a control algorithm in the control layer, provide access to operational data. An operator can also receive responses from the supervisory layer 104 in certain inquiries or generate requested reports. Such interactions between an operator or external entities and interface engine 124 are depicted as 114. Communications of information as described herein can be done using suitable means known to one of ordinary skill, including wired and / or wireless data transfer, access to databases, etc.

[0042]

[0037] Monitoring engine 122 assesses the operational data 108 against the at least one target goal 106 to provide a performance assessment 130 of the industrial process. The performance assessment can be done in any suitable manner known to one of ordinary skill. For instance, monitoring engine 122 can evaluate operational data relevant to the performance of the control layer 102 to identify any discrepancies or failures in their performance. A variety of metrics to evaluate performance of the control layer 102, including evaluating against process targets 106, key performance indicators, including those provided as optimization objectives / like production rate or total profit prediction accuracy for the model -based optimization 120 used, computational performance including duration of calculations and memory -used, process performance including operational stability, satisfaction or violation of constraints, production related metrics such as production rates, another process specific metrics such as reaction yields or achieved purities. For example, monitoring engine 122 can compare the operational data 108 to see whether the operational data indicates a change above a threshold for certain variables (such as temperature, pressure, flow rate, output, etc.) in the process goals 106, or whether there an operator needed to override certain operational aspects by the control layer, or the target process needed to be operated below a certain power level. Additionally or alternatively, monitoring engine 122 can comprise other diagnostic systems, digital twins, that can be relied upon to provide information for performance assessment.

[0043]

[0038] The assessment can be done at any suitable timing and can be automatically programmed or triggered to start without an operator’s explicit command. For instance, the assessment can be done on a selected scheduled, which can be at regularly intervals such as every 15 minutes, or it can be according to plant operations, such as within startup and / or shutdown. The timing of the performance assessment can also optionally be triggered or activated by occurrence of an event. Selection of timing of assessment can be one of the actions that supervisory layer 104 autonomously selects, depending on the provided output from monitoring engine 122, including historical data and historical performance.

[0044]

[0039] Supervisory layer 104 can autonomously manage one or more settings of the control layer 102, particularly in scenarios where the control layer 102 predictive model(s) and optimization solver(s), by autonomously selecting an action to minimize a deviation between the performance assessment 130 and the at least one target goal 106, thereby progressing performance of the industrial process. Supervisory layer 104 includes a decision-making engine 128, which includes a memory device coupled with a hardware processor (both not shown). Supervisory layer 104 performs the autonomous selection at least by the decision-making engine using an optimization solver 144 to solve an optimization problem that includes (i) an objective of minimizing deviation between the performance assessment and the at least one target goal, and (ii) a solution having potential actions the supervisory layer can take to meet the objective while meeting any constraint associated with each action. Supervisory layer 104 can solve an optimization problem using any suitable techniques, such as using a surrogate model or a probability representation of the function to be optimized to approximate the true function based on available parameter values and associated observations. Examples of suitable techniques of building a probability representation include Gaussian Process, Bayesian Neural Network, Ensemble Modelling, etc. It is understood that the principles associated with descriptions related to optimization 120 in Example 1, such as consideration of uncertainty and / or balancing between exploration and exploitation with variable z, and corresponding descriptions beyond Example 1 around exploration and exploitation are also applicable here and are not repeated for the sake of simplicity.

[0045]

[0040] Alternatively or additionally, decision-making engine 128 includes a machine learning algorithm (MLA) 140 trained in a proxy model of the underlying process. An example is a machine learning algorithm (MLA) that is trained in a suitable manner to learn about a process having a plurality of manipulated variables that can be adjusted to effect a change in the output of the process. An example of a suitable way for the MLA to learn is to train the MLA in a proxy model of the one or more processes. The proxy model can sit in optional simulation engine 126, which while depicted as part of supervisory layer 104, it can suitably sit elsewhere (such as a cloud server) and provide a similar function as described herein. The proxy model can be built using techniques known by one of the ordinary skills. For instance, stored process data can be extracted to build plant surrogate models that can describe the plant dynamics with a certain level of fidelity, which can include selected manipulated variables (MVs), constraints, and / or control variables (CVs). The selected MVs are typically ones that can be adjusted within certain constraints to achieve a desired outcome, such as those according to the primary control objectives of the control layer (e.g., an APC). Such process data can be stored in data historian / data-lake.

[0046]

[0041] The proxy process model can be built using, for example, a simplified first principles model, a hybrid model, a surrogate model, or a regression model, and the process model can be trained as, for example, a clustering model, a classification model, a dimension-reduction model, or a deep-learning neural network model. These proxy models can be of any suitable form, such as purely data-driven models derived from multi-variable regression methods. The model may be generated using suitable means such as available process simulation software, including UniSim Design, Pro / II, AspenPlus, Python, Matlab, Simulink, etc. historical operational data of the same and / or other similar facilities and / or conditions, programmed physics principles, and any combination thereof.

[0047]

[0042] The MLA can be trained using suitable methods such as supervised learning, semisupervised learning, unsupervised learning, reinforcement learning, or any combination thereof. In general, training allows the MLA being trained to perceive and interpret its environment (such as explore the proxy process model), take actions and learn through trial and error. The extent of the exploration can depend on the type of training technique used. For instance, an example of supervised and / or semi-supervised learning can include predesigned rules, which can be more limiting on exploration than reinforcement learning in which a reward is given for a desired behavior or unsupervised or evolutionary learning.

[0048]

[0043] As used herein, the term “exploration,” and its variations, refers generally to trying different combinations of operating conditions or settings in solving the optimization equation and observing the corresponding results (such as meeting or failing to meet the objective). Exploration allows the MLA to learn or discover previously unknown relationship(s) between settings ’ and its impact on meeting objective thereby improving its understanding of the environment. The term “exploitation,” and its variations, refers generally to using conditions or settings X known to have certain impacts on the results in solving an optimization problem. Exploration tends to involve testing out conditions or settings that are new or the results of which have not been adequately confirmed, which introduces uncertainty. On the other hand, exploitation tends to use settings or conditions that have known impacts, thereby minimizing uncertainty. One example of when exploration may be preferred is when model accuracy degrades. Additionally or alternatively, random or scheduled exploration, preferably within feasible range, can be performed to test current configurations.

[0049]

[0044] Training can involve providing a history of performance metrics which can include the history of model prediction performance by the control layer 102, the history of constraint violations, and the history of the plant performance for example as plant profit or production rate. A reinforcement learning system can include a learning agent that autonomously learn by interacting with its environment. A reinforcement learning agent can assess one or more observations provided to it and autonomously select an action in response to the observation. It receives rewards for selecting correct or desired actions and costs for selecting incorrect or undesired actions. The agent can learn by maximizing its reward and minimizing its cost on the basis of a policy, a reward function, a value function, and, optionally, a model of the environment. A policy can define the learning agent's behavior by mapping states to actions. The policy can be a simple function, neural network or lookup table in some implementations, or may require a complex computation, such as a search process, in other situations. The reward function can represent the objective of the learning agent. On each time step, the environment can send to the learning agent a reward indicating whether the transition from one state to the next state as a result of an action is correct or incorrect for the agent.

[0050]

[0045] Optionally, the MLA 140 includes a reinforcement learning agent trained in the proxy model using performance assessments as observations, where in the training environment, the performance assessments are provided as part of the training data. The RL agent selects action(s) in response to the provided assessments to achieving one or more target goals, also provided as part of the training data set, to receive a reward. The training is designed to reward selection of actions that minimize a deviation between the provided performance assessment and at least one target goal. The actions available to the RL agent to take are the same actions available during operation of the underlying process. Optionally, supervisory layer 104 is capable of modifying one or more of its own settings, such as those of the MLA 140. In that scenario, the action autonomously selected by the supervisory layer 104 can include at least one of: (a) add or remove one or more elements of the reward function, and (b) adjust a weight assigned to the one or more elements of the reward function. Optionally, reinforcement learning (RL) can be used.

[0051] [461 One benefit of RL is that an RL agent can do non-linear optimization at very high execution frequency, including online non-linear optimization. One potential drawback is that it takes a lot of interactions with the environment (actual or simulated) to train an RL agent to have a high-quality control policy, meaning the agent has iteratively built a collection of known actions for a specific environment it can take to maximize the expected future reward. Training of an RL agent can be done via rewards that are based on relevant performance metrics, such as plant performance, model performance, and constraint satisfactions. Training of the RL agent can include monitoring information (such as operational data 108 described herein) available to the supervisory layer 104 and the actions available to the supervisor layer 104 (such as those described herein and are note repeated for sake of brevity). The reinforcement learning agent can be initially trained using a digital twin of the underlying process in a simulation environment using suitable methods (such as a proxy model) and can be deployed as part of the supervisory layer to continually learn and adapt during operation.

[0052]

[0047] Optionally, supervisory layer 104 may be provided with complex simulation software capable of generating high-fidelity simulations and synthetic data. The supervisory layer 104 can generate synthetic data using various experimentation means on these simulations including the training of reinforcement learning agents and testing of certain optimization equations.

[0053]

[0048] Additionally or alternatively, the autonomous selection of an action by supervisory layer 104 can be based on one or more rules prescribing an action for the performance assessment. In heuristic or rule-based algorithm, potential rules can be (i) to reduce or cease exploration after a constraint violation, (ii) to increase the confidence intervals used for enforcing constraints for a certain time, and / or (iii) to revert to a nominal rate of exploration and nominal value for confidence intervals. Another potential rule can be to respond to a loss of model prediction performance, such as after a consecutive predetermined number of bad predictions or a threshold of operator intervention. A potential response can include increasing the rate of exploration by a certain step first. If that does not lead to an improvement in prediction performance, a potential response can be to change predictive model types or model hyperparameters, retrain the new predictive models using historical data and deploy them if they provide good prediction performance.

[0049] Preferably, additionally, the operational data 108 and the assessment outcomes 130 are stored (such as in a suitable storage device, not shown) and decision-making engine 128 has access to the stored data for use as desired, such as ongoing training and learning where an MLA is employed. From the stored data, decision-making engine 128 has the option to further learn about its own performance, including the computations and configurations it had selected and associated outcomes (e.g., assessment outcomes 130 about performance of certain model-based optimization 120), and / or the history of the interactions between supervisory layer 104 with various connected entities). The more decision-making engine 128 can learn about the performance of the control layer with respect to autonomously selected actions, the better actions it can select going forward. For instance, an autonomously selected action may comprise decision-making engine 128 sending to simulation engine 126 instructions 134 including settings of a potential optimization equation to be sent to control layer 102 to be run. Simulation instructions 134 can also comprise potential plant testing schedules or other potential supervisory actions being considered by a supervisory layer. Simulation-engine 126 provides the simulation results 136 back to decision-making engine 128. Based on its analysis of simulation results 136, decision-making engine 128 can decide whether to send modify instructions 134 and / or take certain supervisory actions. Optionally, instructions 134 can additionally or alternatively include updates to the proxy model based on its analysis of the performance assessment 130. Other examples include decision-making engine 128 deciding when to conduct further training of its own MLA (such as 140), or another MLA in another supervisory layer above it and / or in control layer 102 (such as 142), or training of modelbased optimization 120 in control layer 102 or its own model-based optimization (such as 146), such as when new information is available, including when the proxy model has been updated or an outcome was observed that may not yield the expected result.

[0054]

[0050] It is understood that although not depicted, there can be more than one supervisory layer 104 to manage different aspects of operation. Additionally or alternatively, one supervisory layer can comprise a plurality (two or more) decision-making engines 128 to manage different aspects of operation. For instance, a first supervisory layer 104 (or decision-making engine 128) can manage the RTO component of the control layer 102, a second or more supervisory layer 104 (or decision-making engine 128) can manage the APC component of the control layer 102, yet third or fourth supervisory layers 104 (or decision-making engines 128) can manage other applications, such as DCS or SCAD A, in the base process layer. The various supervisory layers 1 104 (or decision-making engines 128) can each employ differing MLAs For example, some can use rule-based algorithms while others may use reinforcement learning. The number and arrangements can be adjusted as suitable to achieve the desired outcome, such as a set of supervisory layers 104 (or decision-making engines 128) to oversee various components in the control layer 102 and another set to oversee computational resources management. Also contemplated herein is atop supervisory layer 104 (or decision-making engine 128) autonomously selecting an action to modify at least one setting of one or more other supervisory layers 104 and / or decision-making engine 128; for instance, to configure the adjustable parameters of a bottom layer 104 and / or engine 128. In scenarios employing a plurality of supervisory layers 104 and / or decision-making engines 128, any number of layers 104 and / or engines 128 can share resources, such as monitoring engine 122, interface engine 124, simulation engine 126, and / or other resources that would be known to one of ordinary skill. Example 2 provides further descriptions that can be broadly applicable for all embodiments, including here. Such descriptions are not repeated for the sake of simplicity.

[0055]

[0051] FIG. 2 is a flow diagram illustrating the system described with respect to FIG. 1 in operation according to aspects disclosed herein. At step 202, at least one process goal 106 of the target or underlying process is provided, preferably to a monitoring engine 122. At step 204, operational data 108 of the target process is provided, preferably to the monitoring engine 122. It is understood that steps 202 and 204 can occur at the same time (partially or fully overlapping) or in any order. At step 216, the operational data 108 is assessed, preferably by the monitoring engine 122. At step 216, assessment of the operational data 108 can comprise assessing performance of the target process against the at least one process goal 106, where the assessed performance 130 is based at least on the provided operational data 108. Preferably, the monitoring engine 122 conducts the performance assessment.

[0056]

[0052] At step 208, the performance assessment 130 is provided to a decision-making engine 128 of a supervisory layer 104. At step 210, supervisory layer 104 autonomously selects an action to minimize a deviation between the performance assessment 130 and the at least one target goal 106, thereby improving performance of the underlying process. The action includes at least one of (i) a modification of one or more settings of at least one of: the control layer 102, and the decision-making engine 128, and (ii) a modification of one or more settings related to overall performance of a plant running the industrial process. Example 1 provides illustrative description of the types of actions that includes a modification to a setting of control layer 102.

[0057] [531 Some examples of actions that modify one or more settings of the decision-making engine 128 are shown below. That is, decision-making engine 128 (and thus supervisory layer 104) decides to modify its own settings by autonomously selecting one or more, including all, of the following:

[0058] • For scenarios where decision-making engine 128 includes model-based optimization solver 144, o autonomously select and deploy the model type (such as Gaussian Process or a Bayesian Neural Network), optionally based on (i) a statistical criterion for model selection such as Akaike information criterion (AIC), Bayesian information criterion (BIC), Bridge criterion (BC), Cross-validation, and (ii) validation performance of the models; optionally, additionally or alternatively, select the model(s) to be included in an ensemble if the ensemble approach is used; o autonomously select one or more hyperparameters for the selected model type (examples of hyperparameters include: selecting the number of hidden layers for a neural network model or the kernel function type for gaussian processes); o autonomously select when and how to build and / or update the database of available predictive models, including (i) train and validate new or updated predictive models, (ii) select training and validation datasets for building the models, (iii) provide estimated parameters of first principles models. o autonomously adjust weights and / or priority associated with certain equation variables in the optimization equations set forth herein, such as one or more objectives, and / or constraints, including add or remove certain constraints, o autonomously adjust horizon lengths for operation, optimization, or estimation tasks, o Adjust strategies related to exploration and exploitation, such as adjustment of the uncertainty multiplier s.

[0059] • Select the optimization solver used from an available set such as an interior point solver, a sequential quadratic programming solver, or a derivative-free solver, o Adjust settings on the optimization solver such as changing the maximum number of allowed iterations, maximum number of function evaluations or changing numerical tolerances or termination criteria.

[0060] • For scenarios where decision-making engine 128 includes MLA 140, o add or remove one or more elements of the objective function; and o adjust a weight assigned to the one or more elements of the objective function.

[0061]

[0054] Some examples of actions that modify one or more settings related to overall performance of a plant running the underlying process are shown below. These actions typically do not relate directly to the settings control layer 102 but still support the overall performance of facility 100, including production. Optionally, the supervisory layer 104 may send the instructions for these actions to the applicable components of facility 100 shown as 112 rather than through control layer 102. Examples of such actions can relate to plant testing, data collection, and computational resources management, such as one or more, including all, of the following:

[0062] • autonomously plan, schedule, and execute plant tests.

[0063] • autonomously adjust configuration of regulatory control loops during plant testing to prevent the control layer 102 interfering with the plant testing.

[0064] • autonomously generate a request to an operator to adjust the configuration of regulatory control loops.

[0065] • autonomously activate alternative regulatory control configurations to ensure the process conditions do not leave a safe set during testing. Alternatively utilize algorithms, which can simultaneously achieve plant testing and operational safety.

[0066] • autonomously select the amplitude and duration of such tests.

[0067] • autonomously select whether to repeat tests, optionally based on performance assessment of test results.

[0068] • autonomously plan and allocate computational tasks to computational resources, such as for training and re-training of predictive models, hyperparameter optimization, parameter estimation, training of MLAs, including reinforcement learning agents (whether the MLA is part of the decision-making engine 128 and / / or the control layer 102), automated testing of candidate models or algorithms on various simulation environments (proxy process models).

[0055] Decision-making engine 128 can have access to the outcome of the plant testing and / or plant data collection (such as a storage component storing these results and / or relevant elements extracted by monitoring engine 122), which decision-making engine 128 can use to autonomously select any of the actions contemplated herein, such as to update and / or create at least one, including all, of a model used by the control layer 102, a proxy model for the supervisory layer 104.

[0069]

[0056] Optionally, additionally, the decision-making engine 128 can autonomously decide whether and how to make updates to its own settings and / or settings of the control layer 102. For instance, a supervisory action is sending updates to the proxy model of simulation engine 126.

[0070]

[0057] At step 212, after the autonomous selection, the corresponding instructions, such as 110 and / or 112, can be sent to the appropriate receiving components for implementation.

[0071]

[0058] Optionally, at step 214, monitoring engine 122 can be configured to conduct the performance assessment throughout the operation of facility 100. The schedule of monitoring or performance assessment can be set as desired, such as at regular or irregular time intervals, upon occurrence of an event (such as, startup or shutdown), ad-hoc by an operator, and / or as determined by the decision-making engine 128.

[0072]

[0059] The decision-making engine 128 receiving a performance assessment output 130 or a set of performance outputs 130 can be one event that initiates steps 210 and 212, which result in supervisory actions being selected to address a particular aspect as intended by the supervisory layer 104, such as those actions related to model-based optimization 120 and the optimization equation(s) to be solved by control layer 102. Optionally, a supervisory layer can also decide to take certain supervisory actions after receiving a number of performance outputs 130 that indicates a certain pattern or trend, based on any applicable training objectives, such as plant testing or computational resources. The supervisory layer can be the same one overseeing the model-based optimization 120 and / or optimization equations and potentially taking supervisory actions more frequently to address real-time operational issues contained in the performance outputs 130. Additionally or alternatively, the supervisory layer can be one that is dedicated to the nonproduction aspects of facility 100.

[0073]

[0060] As noted, some of the actions that the supervisory layer 104 can autonomously select include making updates, particularly to various models used by the supervisory layer 104 and / or control layer 102. Updates can be one of the key aspects to optimal operation of the underlying process, i.e., minimize a deviation or difference between the performance assessment and the at least one target goal. That is because as conditions change over time, including degradation of equipment (such as fouling, catalyst aging or equipment malfunctioning) and inherent uncertainties, such as presence of unaccounted disturbance variables, the fidelity of the model or its predictive power may degrade to the point where the model no longer serves as a proxy to designated aspects of the underlying process.

[0074]

[0061] Updates can help incorporate changing conditions into the relevant model(s) to maintain or improve its predictive power. Some plants do not perform any updates over time to reflect changes (such as fouling, catalyst aging or equipment malfunctioning) that take place as part of plant operation, which leads to gradually higher degrees of mismatch between the model employed and the underlying process. Instead, to address the higher degree of mismatch, the control layer 102 is detuned to dampen the extent of its corrective programming in response to disturbances, which can lead to reduced control and sub-optimal performance. While updates are useful, it may be challenging in certain circumstances to determine when a mismatch between the model and process occurs and whether such mismatch warrants an update since updates can be consuming of resources and / or disrupt operations, leading to potential decrease in production. It is desirable to have a way to determine when operational data indicates a divergent from historical operation and implement an update autonomously.

[0075]

[0062] Accordingly, at step 218, assessment of the operational data 108 can additionally or alternatively comprise assessing whether the provided operational data contains data considered as new from previously received operational data (“new data”), optionally and preferably based on one or more criteria. If so, the outcome of such assessment can be provided to the supervisory layer at step 208. The supervisory layer may autonomously select to update in response to the new data as an action at step 222.

[0076]

[0063] Suitable manners to determine whether there is new data includes providing monitoring engine 122 with instructions to determine whether operational data 108 contains new data. The instructions can include one or more criteria against which the operational data 108 can be assessed. For instance, the incoming operational data 108 can be assessed against historical operational data to which monitoring engine 122 has access to determine the extent of geometrical distance (Euclidian distance), if any. Part of the criteria can be timing, that is, how far back to set the historical data for assessment (such as, minutes, hours, days, weeks, months, years, etc.). Such timing selection can be done by an operator and / or autonomously by the supervisory layer. Additionally or alternatively, the operational data can be further divided into different categories, such as steady state and dynamic operation, to facilitate identification of new data. For instance, the geometrical distance that may be considered new data in steady state operation may be smaller than for dynamic operation. Suitable manners of defining steady state and non-steady-state (or dynamic) operation include assessing certain operational data (selected as desired, such as based on the impact to the underlying process) over a selected period of time (such as 1, 5, 10, 15, 30, 45, 60, etc., minutes) to determine whether they remain relatively constant (such as, being within a selected % range). If the assessment indicates relatively constant operation, then the underlying process can be considered to be in steady state. If not and the operating conditions are greater than the selected % range, then the underlying process is in dynamic operation.

[0077]

[0064] The outcome of the assessment, such as the geometrical distance and optionally whether the data was collected during steady state or not, is provided to the supervisory layer, which then can autonomously decide the particular model to be updated, and whether and / or how to update the selected model as described herein and / or known to one of ordinary skill, such as any one or a combination of the predictive model (model -based optimization) 120 in control layer 102, predictive model (model-based optimization) 146, if any, in supervisory layer 128, such as decision-making engine 128, and simulation model(s) in simulation engine 126. In addition to autonomously selecting the relevant model(s) to be updated and the respective updates to be made, the supervisory layer 128 can also autonomously select to perform any of the other actions noted herein, according to its assessment. That is, it is understood that the model update is one of the various actions that the supervisory layer may autonomously select to perform, as described in the present disclosure. Some of the other actions that the supervisory layer may autonomously select can also be considered an update and equally apply here or vice versa, the principles described with respect to the model update can apply to other actions, as understood by one of ordinary skill.

[0078]

[0065] Model updating helps to minimize a deviation between the performance assessment and target goal. It is understood that the steps and acts related to model updating (such as 218, 208, 220, 210, 212, 222, etc.) may be performed independently of other steps and acts that are related to changing of operational settings. For instance, a separate decision-making engine 128 and / or supervisory layer 104 may be dedicated to processing operational data 108 to perform autonomous selection of actions related to model updates while one or more other engines 128 and / or supervisory layers 104 are tasked with managing the control layer or other aspects as described herein. While infeasibility is not specifically addressed herein, it is understood that suitable methods can be used to address any infeasibility that may arise.

[0079] [661 In embodiments that employ an MLA, it is further desirable to have training data that is as close to the underlying process as possible, preferably training data generated from the operation of the underlying process itself. Operational data generated during typical operation of the underlying process can result in low quality models because it is closed loop data, which means it reflects both the behavior of the underlying process and also that of the control. For high-fidelity models, it is desirable to have data related to the responses of the underlying process, which can be challenging to extract from typical operational data since it is obscured by the response of the control layer.

[0080]

[0067] One way to generate such training data is to intentionally excite or perturb the underlying process by implementing known changes to selected MVs and / or CVs and tag the operational data generated as a result of such excitation for correlation between the outcome of the selected changes. Although the word “excitation” is used here, it is intended to include all modifications, including relaxing of certain restraints, to MVs and / or CVs to perturb the underlying process for data collection. Suitable manners to perturb the process include modifying the usually smooth MV signals to appropriately designed stepped signals to allow isolated observation of the process response. Excitation can also involve bringing the operating conditions to atypical settings for data collection, which can be achieved through suitable means such as adjusting the MVs and / or CVs and / or temporarily imposing a different economic function. While excitation can be done at any time for any period of time during operation to collect operational data as step signals for model updating purposes, it is usually not preferred. This is because excitation can involve operating the underlying process at sub-optimal to provide buffer between the excited states and the limits (optimal configurations) of the underlying process to ensure safe operations.

[0081]

[0068] The negative economic impact or penalty of excitation can be balanced with the benefits from having relatively higher quality operational data as desired according to the following scenarios:

[0082] • Continuous Excitation: Always excite the underlying process and gather step-signal data and potentially incur the penalty during this session.

[0083] • Scheduled Excitation: Periodically execute experiments to allow regular model updates. • Ad-Hoc Excitation: Manually implement excitation conditions (selected MVs and / or CVs) when convenient to operations.

[0084] • Diagnostic Excitation: Implement excitation conditions when model degradation is detected.

[0085]

[0069] In addition to deciding amongst the four scenarios above, another decision point involves whether to implement an excitation session during steady state or dynamic operation, resulting in eight potential scenarios for excitation and step-data gathering. The supervisory layer, particularly one that employs an MLA, can be trained to assess the operational data as described herein to autonomously select from (i) any of the eight potential scenarios and / or (ii) which of the MVs and / or CVs to be adjusted and extent of adjustment. For instance, dynamic excitation may or may not result in negative economic impacts, depending on the MVs being adjusted, so an autonomous excitation session may be suitable during dynamic operation and less preferred in steady state operation, depending on the assessment of the operational data by supervisory layer.

[0086]

[0070] Although the present disclosure provides a supervisory layer that preferably selects certain actions autonomously as described herein, as shown in FIG. 2. It is understood, however, there can be an option to allow a user or operator to manually provide their respective inputs on any of the relevant parameters, including but not limited to constraint margins, exploration rates, amplitude, absolute input values, and / or override the selections made by the supervisory layer 104.

[0087]

[0071] The following examples illustrate application of various aspects disclosed herein and are not intended to limit the scope of the disclosure to just these examples.

[0088]

[0072] Example 1 - Autonomous configuration of optimization equations

[0089]

[0073] Example 1 provides an example of an application of the principles set forth herein for autonomous optimization of a facility, which can but need not to be facility 100, including a control layer that employs a predictive model to solve an optimization equation to generate intended optimal operational settings for a target process, such as production of a product stream, which process has many adjustable variables. One or more supervisory layers 104 and / or one or more decision-making engines 128 can be used to autonomously configure various aspects of the optimization equation, such as those set forth herein as exemplary supervisory actions related to model-based optimization 120 and optimization equations.

[0074] For instance, the supervisory layer 104 can provide to the control layer the following optimization equation, along with values for applicable variables, including the selection of which variables to include: s. t. gtXt) < h where “ / ’ represents the prescribed performance outcome including the at least one process goal; where “A” represents the new settings to achieve the prescribed outcome; where, optionally, “7” represents the time at which the problem is solved; where “g” represents one or more constraints imposed on the solution; where “ / ?” represents the prescribed constraint limit(s).

[0090]

[0075] Solving the optimization equation can generate one or more solutions X, for a particular time t and / or for multiple time t, which comprise optimal settings that achieve the prescribed outcome / while keeping the imposed constraint(s) g within the constraint limit h. The equation is solved repeatedly at time t and subsequent times, where / X, and g are vectors so they can include multiple MVs and CVs or their combinations. As another example, in addition to the decisions around the weights and priorities of the equation variables set forth, the supervisory layer 104 may select the numerical tolerances or termination criteria or the various time points for solving the equation. Although exemplary applications to an LNG plant will be described below, other general examples of prescribed outcome / include one or more KPIs noted elsewhere herein, including such as, production rate or a total profit from operations, constraint satisfactions or violations, model prediction errors). The imposed constraints g and constraint limits h are associated with the selected prescribed outcome.

[0091]

[0076] In general, there is a strong correlation between the usefulness of the output of a machine learning algorithm and the input or data upon which it has been trained. The more accurate the model is, the more useful the output. Models, however, typically have their limitations as they are often based on approximations, which give rise to model uncertainty - in both cases of the proxy process model and the surrogate models. The uncertainty in various models can significantly impact the predictions and decisions made based on these models.

[0092]

[0077] Many of the techniques noted above for building a probability representation provide their own respective uncertainty estimates, thereby addressing the uncertainty challenge. On the other hand, in instances relying on first principles models, such models typically do not include any uncertainty factor or estimate. Embodiments disclosed herein can provide an uncertainty factor (or estimates) heuristically or manually.

[0093]

[0078] Optionally, the decision-making engine 128 can decide to consider model uncertainty by providing the following optimization equation that includes model uncertainty with variable P. s. t. g(Xt,Pl t) < h where “ ” represents an uncertainty estimate of the uncertain parts of the predictive model in the model -based optimization being used (such as, 120).

[0094]

[0079] That is, decision-making engine 128 can use model uncertainty quantification P for setting up the optimization equation for the control layer to enforce optimization constraints.

[0095]

[0080] Optionally, decision-making engine 128 can choose to further or alternatively consider an additional amount of uncertainty associated with its various decisions in selecting the values in configuring the optimization equation (such as values g and A). This additional amount of uncertainty (e.g., decision uncertainty) can be an adjustable parameter that has different values for the different constraints g (e.g., different decision uncertainty elements where each element can have a different value) of the objective function f. As such, the total decision uncertainty U can be constructed in various suitable manners; where all the applicable uncertainty elements have equal values. If desired, the value of certain uncertainty elements can have different values that correspond to the weights or impact of that uncertainty element. In any case, the sum of the decision uncertainty elements provides the total decision uncertainty U.

[0096]

[0081] Decision-making engine 128 can autonomously select the value to assign to such decision uncertainty elements in setting up the optimization equation for the control layer 102 to solve, based at least on performance assessment 130 and any applicable training / 1 earning, prescribed rule, and / or objectives. Supervisory layer 104 can learn and adapt the model design parameters (whether in the proxy model, such as in simulation engine 126) to further improve solving the optimization problem, which translates to optimized operation.

[0097]

[0082] Optionally, the decision-making engine 128 can autonomously decide to further introduce a preference for exploration or exploitation of the selected model-based optimization 120 by introducing a factor z in setting up the optimization equation for the control layer. A selection by decision-making engine 128 of relatively more model exploration means it wants to 1 introduce relatively more uncertainty into the solution sets X generated by the control layer as the configurations it provides to the control layer contains instructions to explore the uncertainty parts of the model-based optimization 120 in solving the optimization equation. A selection of relatively more model exploitation means the decision-making engine 128 wants relatively less uncertainty in the solution sets X by providing instructions to avoid uncertain regions, the extent depends on the selected z value. Accordingly, decision-making engine 128 can autonomously decide to equation C below as the objective / for optimization equation A or B to the control layer as part of its supervisory actions. where “C” represents the prescribed outcome noted as / above in equation A, which can include a financial objective; where “t / ” represents total decision uncertainty, preferably based on the total uncertainty elements in P and where “z” represents a multiplier weighing U against C

[0098]

[0083] Examples of a financial objective C can be a cost function for a certain operation, with elements such as profit and / or performance criteria. U corresponds to the total decision uncertainty as noted herein. The multiplier z provides an option to adjust the overall uncertainty with respect to the prescribed outcome. Preferably, z is in a range from -1, which corresponds to an avoidance of uncertainty, and up to 1, which corresponds to 100% uncertainty, including the value of 0 which corresponds to 0% or no uncertainty. For instance, if the desire is to explore, then a value that is closer to 1 may be selected for z to allow for correspondingly greater uncertainties to be introduced into X as the value increases toward 1. If the desire is to exploit, then a value closer to 0 can be selected to reduce the importance of uncertainty in finding the solution(s). If it is desired to avoid uncertainty, then a value that is between 0 and -1 can be selected, which tends to lead to solution(s) that avoid uncertain regions of the decision space. Solutions generated with avoidance of uncertainty correspond to a more extreme form of exploitation. The value of z may be selected as desired, optionally by the supervisory layer 104 and / or by an operator. It is understood that values other than -1, 0, and 1 can be used in applying the principles described herein.

[0099]

[0084] Optionally, a local approximation of the solution to the optimization equation A, B or C may be provided to the control component (such as, 102, preferably an APC) using a Local Economics based APC real time optimizer (LEAR) integrated with the control component. An example of the LEAR is provided in US9268317B2.

[0100] [851 Some examples of actions that supervisory layer 104 can autonomously take may relate to model-based optimization 120, such as one or more, including all, of the following:

[0101] • autonomously select and deploy the model type of model -based optimization 120 (Gaussian Process or a Bayesian Neural Network), optionally based on (i) a statistical criterion for model selection such as Akaike information criterion (AIC), Bayesian information criterion (BIC), Bridge criterion (BC), Cross-validation, and (ii) validation performance of the models; optionally, additionally or alternatively, select the model(s) to be included in an ensemble if the ensemble approach is used;

[0102] • autonomously select whether and how to adjust the hyperparameters for the selected model type of model -based optimization 120 (examples of hyperparameters include: selecting the number of hidden layers for a neural network model or the kernel function type for gaussian processes);

[0103] • autonomously select when and how to build and update the database of available predictive models, including (i) train and validate new or updated predictive models, (ii) select training and validation datasets for building the models, (iii) provide estimate parameters of first principles models.

[0104]

[0086] Some examples of actions that supervisory layer 104 can autonomously take may relate to the optimization equation(s) to be solved by the control layer 102, such as one or more, including all, of the following:

[0105] • autonomously adjust weights and / or priority associated with certain equation variables in the optimization equations set forth herein, such as one or more objectives / like cost function, and / or constraints g and / or h, such as based on safety, including add or remove constraints g,

[0106] • autonomously adjust horizon lengths for operation, optimization, or estimation tasks,

[0107] • autonomously adjust strategies related to exploration and exploitation, such as adjustment of the uncertainty multiplier z,

[0108] • autonomously select the optimization solver used from an available set such as an interior point solver, a sequential quadratic programming solver, or a derivative-free solver, • autonomously adjust settings on the optimization solver such as changing the maximum number of allowed iterations, maximum number of function evaluations or changing numerical tolerances or termination criteria.

[0109]

[0087] Optionally, for scenarios where the control layer 102 includes a trained reinforcement learning agent (such as one embodiment described in Example 2) to solve the optimization problem, some examples of actions that the decision-making engine 128 can autonomously select includes at least one, including all, of the following:

[0110] • autonomously select the specific reward function, weight, and / or priority to use in training;

[0111] • autonomously select the type of RL agent (such as SAC, PPO);

[0112] • autonomously select when and how to a new RL agent using simulation engine 126;

[0113] • autonomously select when to deploy the trained RL algorithm in the control layer 102.

[0114]

[0088] Examples 2 - 4: LNG Plant

[0115]

[0089] Set forth below are some examples that apply the principles set forth herein to operate an LNG Plant. As noted elsewhere, an LNG production facility can benefit from the systems and methods disclosed herein. For instance, embodiments of the present disclosure can be used to operate and / or optimize the operation of different operating modes of the plant: start-up, ramp-up, steady state (or regular operations), and ramp-down. These examples are illustrative of the principles described herein and are not intended to limit the scope of such principles. It is intended that the descriptions with respect to the Examples also broadly apply to other embodiments disclosed herein beyond the Examples.

[0116]

[0090] Certain terms used herein are defined as follows:

[0117]

[0091] “NG” refers to natural gas. Natural gas is a naturally occurring hydrocarbon gas mixture primarily having methane, but commonly including varying amounts of other higher alkanes, and sometimes a small percentage of carbon dioxide, nitrogen, hydrogen sulfide, or helium.

[0118]

[0092] “LNG” refers to liquefied natural gas, which is typically cooled to at least a temperature whereat the gas can be in the liquid phase at about 1 bar pressure; for liquefied methane this temperature may be about -162° C.

[0119]

[0093] “Mixed refrigerant” or “MR” refers to a refrigerant comprised of two or more components. Depending on the stage of the heat exchanger (pre-cooler or main cryogenic heat exchanger), the refrigerant may include components such as methane, ethane, propane, and nitrogen.

[0120]

[0094] “HMR” and “LMR” refer to “heavy mixed refrigerant” and “light mixed refrigerant” respectively, indicating mixed refrigerant separated into light and heavy mixed refrigerant streams, where the terms “light” and “heavy” indicate average component weight of each stream relative to each other.

[0121]

[0095] ‘PMR” may refer to a pre-cool mixed refrigerant, i.e. a mixed refrigerant used in a precool circuit of a liquefaction system.

[0122]

[0096] “Bar” is a metric unit of pressure, defined as equal to 100 kPa. “Bar(a)” and “bara” are sometimes used to indicate absolute pressures and “bar(g)” and “barg” for gauge pressures. Herein, “2 barg” is similar to fuller descriptions such as “gauge pressure of 2 bar” or “2-bar gauge”.

[0123]

[0097] There are various techniques to liquefy natural gas, such as C3MR (propane and MR), SMR (single mixed refrigerant), DMR (double mixed refrigerant), or cascade-based liquefaction processes. Many of these techniques comprise a coil wound heat exchanger, typically the main cryogenic heat exchanger (MCHE), in which a substantial part of the cooling of the natural gas takes place.

[0124]

[0098] Example 2 - cooldown of heat exchangers, including a main cryogenic heat exchanger (MCHE)

[0125]

[0099] In a C3-MR liquefaction process, the refrigerant used for the pre-cooling heat exchanger is mainly propane and the refrigerant used for the main cryogenic heat exchanger is a mixed refrigerant. In a DMR process, typically, the refrigerant used for the pre-cooling heat exchanger is a first mixed refrigerant and the refrigerant used for the main cryogenic heat exchanger is a second mixed refrigerant. FIG. 3 illustrates an example of a liquefaction process that can benefit from the systems and methods describe herein for demonstration purpose. One of ordinary skill can apply the disclosed concepts to other processes in a similar manner.

[0126]

[0100] Referring to FIG. 3, there is shown a main cryogenic heat exchanger (MCHE) 322 and corresponding mixed refrigerant (MR) loop 324. Not shown is the optional pre-treatment section to pretreat feed gas, for instance to remove contaminants, such as one or more of acid gas, mercury, water, mercaptans, etc. The pre-treated gas may be provided to a pre-cool section, also not shown, for cooling to a predetermined pre-cool temperature to produce pre-cooled gas 302. As shown, pre-cooled gas 302 is provided to MCHE 322, which is part of refrigerant loop 324. In use, pre-cooled natural gas 302 enters through the tube side at the bottom of the MCHE 322, for instance at near ambient temperature, and exits at the top as Liquefied Natural Gas 350 at a rundown temperature of, for instance, about -150 to -145 degrees C, depending on operational settings. Liquefaction of the natural gas is achieved by flowing evaporating mixed refrigerant, which is a mixture of, for instance, liquid nitrogen (N2), methane (Ci), ethane (C2) and propane (C3), on the shell side of the heat exchanger 322. The mixed refrigerant is typically divided into two streams, Light Mixed Refrigerant (LMR) 396 and Heavy Mixed Refrigerant (HMR) 394. LMR provides cooling duty in a top section of the heat exchanger 322. HMR provides cooling duty in a middle and bottom section of the heat exchanger 322. Mixed refrigerant 340 exits the MCHE 322 in gaseous form and is routed to compressors 380, 382 for compression, which subsequently may be cooled to, for instance, about -33 degree C. Cooling the compressed refrigerant may be done by, for instance, air coolers, propane kettles (for a C3MR process) and / or an MR loop (for a DMR process), represented schematically by the coolers 388, 390.

[0127]

[0101] As part of the start-up of the LNG plant, the main equipment to be started up is the MCHE. The normal operating temperature at its cold end (such as near where LNG 350 exits) is typically between -100 degrees C and - 165 degrees C, depending on the refrigerant employed. Therefore, initial start-up of these types of exchangers involves cooling the cold end from ambient temperature (or pre-cooling temperature) to normal operating temperature and establishing proper spatial temperature profdes for subsequent production ramp-up and normal operations. An important consideration while starting up pre-cooling and liquefaction heat exchangers (such as MCHE) is that they must be cooled down in a gradual and controlled manner to prevent thermal stresses to the heat exchangers. Not doing so may cause thermal stresses to the heat exchangers that can impact mechanical integrity, and overall life of the heat exchangers that may eventually lead to undesirable plant shutdown, lower plant availability, and increased cost. At the same time, the cool-down rate preferably is as high as possible within these constraints to avoid unnecessary delays in reaching full capacity levels to achieve planned site production targets.

[0128]

[0102] The need to start-up the heat exchangers, including the MCHE, may also be present after the initial start-up of the plant, for instance during restart of the heat exchangers following a temporary plant shutdown or trip. In such a scenario, the heat exchanger may be warmed up from ambient temperature, hereafter referred to as "warm restart" or from an intermediate temperature between the normal operating temperature and ambient temperature, hereafter referred to as "cold restart." Both cold and warm restarts must also be performed in a gradual and controlled manner. The terms "cool-down" and "start-up" generally refer to heat exchanger cool-down during initial start-ups, cold restarts as well as warm restarts.

[0129]

[0103] The gradual and controlled start-up of the heat exchangers can be achieved with a set of sequential steps of different moves. The size of each move can be carried out by adjusting certain manipulated variables within prescribed constraints. One approach is to perform these steps manually where the refrigerant flow rates and composition are manually adjusted in a step- by-step manner to cool down the heat exchangers. This process requires heightened operator attention and skill, which may be challenging to achieve in new facilities and facilities with high operator turnover rate. Additionally, in the process, the rate of change of temperature is often manually calculated and may not be accurate. Further, manual start-up tends to be a step-by-step process and often involves corrective operations, and therefore is time consuming.

[0130]

[0104] Another approach is to automate the cool-down process with a programmable controller, such as the control component 102. Yet another approach is to manually cool down the heat exchangers to a certain production level then the control component takes over to further drive it These approaches, however, are prone to operator errors, and the performance of the system is dependent on the skill and the continuous attention of the operator potentially leading to underperformance. It has been observed from existing LNG operations that such manual systems are prone to excessive MCHE cooling and / or warming rates potentially leading to heat exchanger tube failures, leading to unplanned plant shutdowns, resulting in loss of production and increased maintenance cost.

[0131]

[0105] Moreover, the control schemes employed to optimize steady state operation after startup cannot be readily used for start-up purposes. During steady state the temperature profdes are already established and are to be maintained relatively stable and feed gas and refrigerant flow rate do not need to be increased from zero as in the case of start-up. This eliminates one critical variable in the control scheme. Additionally, during normal operation, refrigerant composition may require no or small adjustments, unlike during start-up where larger adjustments need to be made throughout the start-up process. In the case of mixed refrigerant processes, refrigerant component inventory be (partially) unavailable during start-up which further complicates the control process. Further, refrigerant compressors are often operating in recycle mode during start-up to prevent reaching the surge limit. These recycle valves may need to be gradually closed during the cool- down process, which is an additional variable to be adjusted. Furthermore, during start-up and heat exchanger cool down, the suction pressure needs to be monitored and refrigerant components (such as methane in the case of MR based process and N2 in N2 recycle process) need to be replenished in order to maintain a proper suction pressure. This also presents challenges during the start-up operation. Embodiments disclosed herein allow for the autonomous start-up of heat exchangers in a natural gas liquefaction facility, while minimizing operator intervention

[0132]

[0106] There is a number of variables that can be adjusted to achieve the desired cool-down of the heat exchangers. For instance, one or more of the refrigerant compressors 380, 382 may be provided with adjustable inlet guide vanes 320. The angle of the vanes 320 can be varied as one of the manipulated variables to achieve a broader range of performance characteristics. The inlet guide vanes 320 can for instance adjust and control the flow rate of gaseous refrigerant 340 flowing into a compressor (e g., 380 and / or 382). For a detailed description of inlet guide vanes, reference is made to, for instance, US-2010 / 0172745 or US20140311183.

[0133]

[0107] Optionally, if applicable, the MCHE equipment can comprise: the light mixed refrigerant (LMR) equipment and the heavy mixed refrigerant (HMR) equipment. Under such scenario, another manipulated variable may include one or more of, for instance, the flow rate of LMR (F-LMR), which can be controlled by adjusting the LMR valve (while not shown, it is understood by one of ordinary skill to be in the figure), and the flow rate of HMR (F-HMR), which can be controlled by adjusting the HMR valve 312.

[0134]

[0108] An optional manipulated variable can include a warm bundle bypass valve (not shown). The term ‘warm bundle’ as used herein, may relate to the bundle on the warm end of the heat exchanger, i.e. the bundle closest to the process stream inlet of the heat exchanger. In FIG. 3, warm bundle may refer to the entire bundle, or to a part of the bundle on the warm end of the heat exchanger. Temperature sensors are typically provided near at least some or all inlets and outlets of the main heat exchanger 322, allowing to calculate or estimate a temperature profile along the respective bundle between the respective inlet and outlet. The warm bundle bypass is a process configuration where part of the natural gas is fed into the bottom of a scrub column (not shown) and the remainder is routed via a warm bundle (not shown) in the MCHE 322 to the top of the scrub column (not shown).

[0135]

[0109] Optionally, a portion of the pre-cooled stream 302 may bypass the warm bundle. The bypass portion (not shown) from the warm bundle bypass subsequently may be recombined with the portion of the process stream that has passed through the warm bundle. The flow rate of the bypass portion, and consequently the ratio thereof with respect to the flow rate of the pre-cooled stream 302 in the warm bundle , can be controlled by a suitable flow actuator, such as the warm bundle bypass valve.

[0136] [HO] Still another optional manipulated variable can be a flare valve 304, which can be part of the equipment providing pre-cooled gas 302 to MCHE 322. During the time the MCHE has not reached sufficiently low temperatures, pre-cooled gas 302 may be (partially) flared rather than flowing through the MCHE 322. As the MCHE 322 approaches operational temperatures, the precooled gas 302 can be redirected through the MCHE 322 for liquefaction. The redirecting involves the manipulation (closing) of the position of flare valve 304.

[0137] [Hl] Still another example of a manipulated variable that can be controlled or adjusted in achieving the desired cool-down rate for the MCHE 322 can be LNG flow valve 306. Liquefied process stream 350 exiting MCHE 322 at the cold end may be provided to a flash vessel 310, which may also be referred to as end flash system and may include some form of pressure reduction system, such as an expander or Joule-Thompson valve (not shown).

[0138]

[0112] Yet another example of a manipulated variable can be compressor recycle valves 308. Refrigerant compressors typically have operating window that they need to stay inside of to guarantee integrity of the equipment. One such operating limit that may be of concern during MCHE 322 cooldown is the surge limit. If at any time there is insufficient flow through the compressor at given speed / head, the flow regime may become unstable, causing damage to the compressor and / or adjacent equipment. The compressor recycle valves 308 (also sometimes called spillback valves or anti-surge valves) ensure the required minimum flow may always be provided. When starting up MCHE 322, the initial cooling demand may be low, so recycle valves 308 are typically opened. As production ramps up and flow increases through MCHE 322, the process demand for refrigerant increases and the recycle valves 308 can be gradually closed as determined according to aspects disclosed herein.

[0139]

[0113] Accordingly, the list of potential MVs that can be manipulated or adjusted to achieve a desired cool-down rate of the MCHE includes one or more, including all, of the below:

[0140] • MV-1 : MR Compressor inlet guide vane position (e.g., position of 320)

[0141] • MV-2: LMR valve (e.g., LMR valve, which may be a JT valve)

[0142] • MV-3 : HMR valve (e.g., valve 312, which may be a JT valve) • MV-4: LNG flow valve (e.g., 306)

[0143] • MV-5: Compressor recycle valves (e.g., 308)

[0144]

[0114] And other manipulated variables that are mentioned in this disclosure, as well as others known to one of ordinary skill. In the context for this Example 2 of starting up an MCHE, the boundary conditions or constraints for this Example 2 relates to the maximum temperature rate of change (TROC) for the MCHE equipment. Preferably, the vendor of the respective equipment, such as LMR and / or HMR, provides such maximum rate of change, within which the facility itself can further select as the respective constraints, such as C-l and C-2.

[0145]

[0115] Optionally, to further maximize the rate of cooldown, thereby reducing the cooldown time, the optimization equation can further include a time constraint (C-3) that prescribes a timelimit (e.g., a selected number of hours) within which the process can achieve a temperature range at the cold end of the heat exchanger. It is understood that the given list of constraints C is for illustrative purposes and not intended to limit the present disclosure. Other constraints and their assigned value can be selected by one of ordinary skill.

[0146]

[0116] Accordingly, the optimization equation for cooldown of a heat exchanger, such as a MCHE, can be written as follows: s. t.g(Xt) < h being the optimally minimized rate of cooling of the target heat exchanger; where “A” being the settings for one or more, preferably all, of the following MVs, which have been selected to achieve f

[0147] MV-1 : MR Compressor inlet guide vane position (e.g., position of 320)

[0148] MV-2: LMR valve (e.g., LMR valve, which may be a JT valve)

[0149] MV-3: HMR valve (e.g., valve 312, which may be a IT valve)

[0150] MV-4: LNG flow valve (e.g., 306)

[0151] MV-5: Compressor recycle valves (e.g., 308); where optionally, being a time period during or point of time for which the X have been selected; where “g” and “A” being one or more, including all, of

[0152] C-l HMR TROC in a range from 15 degrees C / hour and up to 40 degrees C / hour C-2: LMR TROC in a range from 15 degrees C / hour and up to 40 degrees C / hour

[0153] C-3: time limit during which the selected X provide LMR and HMR with a temperature in a specified range (such as in a range from -155 degrees C and up to -135 degrees C)

[0154]

[0117] If it is desired to address uncertainty in the optimization equation itself, then either equation (B) or equation (C) may be used with the suitable variables for uncertainty inserted in a similar manner as shown for equation (D).

[0155]

[0118] Embodiments disclosed herein enable for optimization, particularly autonomous optimization, of a fast-moving process, such as cooldown of heat exchangers, which has been difficult to optimize with current methods, such as APC and RTO, due the challenges noted elsewhere. There is provided a MLA that is preferably a reinforcement learning agent, that is trained using a data set to solve an optimization problem including a proxy model simulating the heat exchangers environment. As a safety concern, an RL agent that still needs training to develop a high-quality policy preferably should not be deployed to be further trained or to take online (or real time) optimization actions in the actual environment, such as the control layer 102 (e.g., APC). Instead, the RL agent can be trained using training data set including a reinforcement learning infrastructure (e.g., a proxy model). Through a large number of interactions with the infrastructure or environment, the RL agent can iteratively build a control policy to maximize the expected future reward. Examples of suitable environments can include a digital twin, an interactive simulating model such as a UniSim model, an offline (or non-interactive) proxy model based on historical operational data of prior cooldowns at the same and / or different facilities, and / or a proxy model based on historical operational data that is further supplemented with physics-inspired machine learning (PIML) to improve its accuracy toward the high-fidelity provided by an interactive model, such as UniSim.

[0156]

[0119] The reinforcement learning infrastructure used may depend on resources and / or objectives. For instance, a benefit to interactive simulating models, like UniSim, is that they provide a process performance outcome for actions taken by the RL agent, which the RL agent can use to learn further. A draw back, however, is that training of an RL agent in such interactive models can be costly in terms of computing cost since the dynamic simulator takes significant time to converge for each iteration or set of action(s) taken by the RL agent.

[0157]

[0120] On the other hand, the offline proxy model based on historical operational data involves much less computing cost; however, a drawback is that the training is offline in that the RL agent does not have information on how the process would behave for a set of action(s) for which no applicable historical data exists in the offline proxy model. Such lack of information may give rise to prediction errors, including overpredicting value of operating in certain unseen regions. While special offline RL algorithms (such as, for example, conservative Q-learning (CQL)) can be used to generate a control policy from a historical dataset, the control policy generated is limited in its ability to extrapolate, which may lead to poor performance when deployed in an environment that may deviate from the historically observed data. Such limitation may be addressed by validation of the generated control policy against an interactive simulating model to ensure adequate performance before deployment in a real plant. The process of training an initial agent against historic data, deploying it against the simulator to validate its strategy and further improve its performance leverages the cost efficiency of training in an offline-environment and the interaction benefits provided by an interactive model with an agent with some training in a similar environment already, thereby reducing computing costs as compared to initially training an agent in the interactive environment. The agent trained offline can also be tested using a Physics Based Process Flow-Sheeter to evaluate its performance before deploying the strategy in a real plant.

[0158]

[0121] Optionally, a proxy model can be developed that approaches the accuracy of a Physics- Based Process Flow-Sheeter, such as UniSim simulator, thereby providing an interaction or outcome of a set of action(s) that have been taken but does not need as much computing power. Such model can be built using physics-inspired machine learning (PIML), which is a branch of machine learning that creates neural network models using knowledge of underlying physical principles to increase fidelity of the predictions.

[0159]

[0122] Regardless of the training environment being used, the reward and cost setup can be the same where the RL agent receives a cost or penalty score for taking actions that does not achieve the objective function f and / or violate the prescribed constraints g that are set forth in Example 2. The penalty score given can optionally be further divided into the three constraints of HMR TROC, LMR TROC, and time limit. This scorecard or metric allows for assessment of the relative performance of each cooldown run or iteration by the RL agent. A lower score indicates better performance, with a score of zero being the best possible score. The set of action(s) that the RL agent can take are the same ones set forth for the MVs in X in equation D.

[0160]

[0123] Optionally, a plurality of supervisory layers 104 and / or decision-making engines 128 can be employed with respect to Example 2. For instance, the parameters associated with training of the reinforcement agent MLA to cool down a heat exchanger as disclosed in Example 2 can be autonomously selected by a supervisory layer 104 and / or a decision-making engine 128. Illustrative actions that can be autonomously selected by a decision-making engine 128 are provided in Example 1 and are not repeated here for ease of readability. Additionally or alternatively, the trained MLA can be deployed as part of the control layer 102, such as depicted as MLA 142, at least in the aspect that the RL receives configurations selected autonomously by a supervisory layer 104 for the trained MLA to solve and generate a set of operational settings X. Additionally or alternatively, the trained MLA can be in a decision-making engine 128 that sits on top of a control layer, where the decision-making engine, with the help of the trained MLA, autonomously selects supervisory actions for implementation by the control layer.

[0161]

[0124] Example 3 - Steady State Operation

[0162]

[0125] Once the LNG plant has started up, various control schemes such as those described in U.S. Patent No. 5,791,160 or U.S. Patent No. 4,809,154 may be utilized to control parameters such as the LNG temperature, flow rate, heat exchanger temperature differences and so on. Optionally, the steady state (SS) operation may be conducted in two modes: (i) gas constrained or (ii) gas sufficient mode. While embodiments of the methods and systems described herein may be used in either mode, it is preferably used in the gas sufficient mode.

[0163]

[0126] Preferably the objectives in steady state operation are to maximize LNG production and maximize energy efficiency at the given production level. For reasons known to one of ordinary skill, such as those generally set forth herein, the levers or variables that can be adjusted to meet these objectives tend to vary from the MVs provided for Example 2 to cooldown a heat exchanger. The equation variables for steady state operation of a liquefaction process for optimization equation A can be written as follows: s. t.ff(Xt) < h where “f” represents at least one, including both, of Maximize LNG production and Maximize energy efficiency at the given production level where “X” being the settings for one or more, preferably all, of the following MVs, which have been selected to achieve f

[0164] MV-1 : MR Compressor inlet guide vane position (e.g., position of 320) MV-2: LMR valve (e.g., LMR valve, which may be a JT valve)

[0165] MV-3: HMR valve (e.g., valve 312, which may be a JT valve)

[0166] MV-4: LNG flow valve (e.g., 306)

[0167] MV-5: Compressor recycle valves (e.g., 308);

[0168] MV-6: Cooling kettle levels (not shown) where optionally ‘7” being a time period during or point of time for which the A’ have been selected; where “g” and “A” being one or more, including all, of

[0169] C-l LNG rundown temperature in a range from -165 degrees C and up to -155 degrees C

[0170] C-2 LMR flow in a range from 4000 tons / day and up to 5000 tons / day

[0171] C-3 PR discharge pressure in a range from 20 barg and up to 30 barg C-4 MR discharge pressure in a range from 40 barg and up to 55 barg C-5 C5+ in NG less than 0.5 %

[0172] C-6 HMR temperature in a range from -140 degrees C and up to -120 degrees C

[0173] C-7 MR return temperature in a range from -40 degrees C and up to -25 degrees C

[0174]

[0127] It is understood that different supervisory layer(s) 104 can be used to autonomously provide parameters to optimize the cooldown process and different supervisory layer(s) 104 for steady state operations. It is also within the scope of the disclosures to have the same set of one or more supervisory layer(s) 104 to manage both operations.

[0175]

[0128] Example 4 - Plant-wide Optimization

[0176]

[0129] Examples 2 and 3 demonstrate examples of application of the methods and systems disclosed herein to enable autonomous operation of individual aspects of a facility, such as cooldown of heat exchangers (which can be part of starting up an LNG plant) in Example 2, or individual units of a facility, such as autonomous operation or optimization of steady state operation of one or more LNG trains. This Example 4 provides an illustrative application of the invention to enable autonomous operation or optimization at a plant-wide level, such as across a plurality (two or more) LNG trains, which considers as one system the operational units (such as LNG trains), as well as the common systems to which they connect (such as upstream and downstream of the trains). One benefit to considering the whole plant is being able to leverage differences between individual trains to drive the common systems to their limits using the throughput of the individual trains, such as using the differences in the amount of end flash gas each train produces to drive a train rundown temperature against limits in the common downstream systems.

[0177] [1301 Example 4A- Upstream Optimization

[0178]

[0131] The common upstream system to which two or more LNG trains at an LNG plant connect is known to one of ordinary skill. Examples may include upstream gas production facilities, feed gas landing, processing and header systems and associated pipeline infrastructure, and upstream online gas production facility or offshore gas platform, if applicable.

[0179]

[0132] The equation variables for optimizing the upstream system for optimization equation A can be written as follows: s. t. g Xf) < h where “f ’ represents at least one, preferably both, of combined feed intake of the trains (to be maximized) and total fuel gas consumption (to be minimized); where “A” being the settings for one or more, preferably all, of the following MVs, which have been selected to achieve /

[0180] MV-1 : Train feed gas intake

[0181] MV-2: Pipeline flow balancing (optional)

[0182] MV-3: Feed gas header crossover valve (optional)

[0183] MV-4: Fixed pipeline feed (optional) where optionally being a time period during or point of time for which the A have been selected; where “g” and “A” being one or more, including all, of

[0184] C-l : Slug catcher overhead pressure in a range from 50 barg and up to 100 barg

[0185] C-2: Feed flow control valves position of greater than 80%

[0186] C-3: Feedgas header pressure control valves position of greater than 80%

[0187]

[0133] Example 4B - Downstream Optimization

[0188]

[0134] The common upstream system to which two or more LNG trains at an LNG plant connect is known to one of ordinary skill. Examples may include the end flash and associated pipeline infrastructure

[0189]

[0135] The equation variables for optimizing the upstream system for optimization equation

[0190] A can be written as follows: s. t.g^Xt') < h where “f ’ represents at least one, preferably both, of maximum combined LNG production across the selected trains and minimum energy consumption; where “A” being the settings for one or more, preferably all, of the following MVs, which have been selected to achieve f

[0191] MV-1: Train rundown temperature

[0192] MV-2: Boil-off-gas (BOG) to LNG trains where optionally being a time period during or point of time for which the Ahave been selected; where “g” and “A” being one or more, including all, of

[0193] C-l : BOG header pressure in a range from 20 mbarg and up to 50 mbarg

[0194] C-2: End-flash-gas (EFG) pressure in a range from 0.6 bara and up to 1.5 bara

[0195] C-3: EFG compressor power in a range from 10 MW and up to 18 MW

[0196]

[0136] Computer Components

[0197]

[0137] While example embodiments have been particularly shown and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the embodiments encompassed by the appended claims

[0198]

[0138] Examples of computer or processing systems that may implement the methods and systems described herein are well known and as such is not depicted herein. Also, the computer system is only one example of a suitable processing system and is not intended to suggest any limitation as to the scope of use or functionality of embodiments described herein. The processing system may be operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of suitable well-known computing systems, environments, and / or configurations may include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like.

[0199]

[0139] The computer system may be described in the general context of computer system executable instructions, such as program modules, being run by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. The computer system may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

[0200]

[0140] The components of computer system may include, but are not limited to, one or more processors or processing units, a system memory, and a bus that couples various system components including system memory to processor. The processor may include a module that performs the methods described herein. The module may be programmed into the integrated circuits of the processor, or loaded from memory, storage device, or network or combinations thereof.

[0201]

[0141] Computer system may include a variety of computer system readable media. Such media may be any available media that is accessible by computer system, and it may include both volatile and non-volatile media, removable and non-removable media.

[0202]

[0142] System memory can include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory or others. Computer system may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system can be provided for reading from and writing to a non-removable, non-volatile magnetic media (e.g., a “hard drive”). Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic disk (e.g., a “floppy disk”), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. In such instances, each can be connected to bus by one or more data media interfaces.

[0203]

[0143] The computer system can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter. Network adapter can communicate with the other components of computer system via bus. Other hardware and / or software components could be used in conjunction with computer system. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0144] It is understood that implementation of embodiments disclosed herein may be achieved with suitable computing environments, such as cloud computing, and / or any other type of computing environment now known or later developed.

[0204]

[0145] The computer system can include hardware and software components. Examples of hardware components include mainframes; RISC (Reduced Instruction Set Computer) architecture based servers; servers; blade servers; storage devices; and networks and networking components. Software components can include network application server software and database software.

[0205]

[0146] The present invention may be a system, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

[0206]

[0147] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0207]

[0148] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0208]

[0149] Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, Python, Fortran, JavaScript or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

[0209]

[0150] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0210]

[0151] These computer readable program instructions may be provided to a processor of a computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein includes an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart and / or block diagram block or blocks.

[0211]

[0152] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0212]

[0153] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be accomplished as one step, run concurrently, substantially concurrently, in a partially or wholly temporally overlapping manner, or the blocks may sometimes be run in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

Claims

CLAIMS1. A computer-implemented method for autonomous management of operation of an industrial process comprising a control layer (102) to control equipment of the industrial process: a. providing operational data (108) related to the industrial process; b. providing at least one process goal (106) for the industrial process; c. assessing the operational data against the at least one process goal to provide a performance assessment (130) of the industrial process; d. providing the performance assessment to a supervisory layer (104) comprising a decision-making engine (128) comprising a memory device coupled with a processor; wherein the supervisory layer being capable of modifying one or more settings of the control layer and / or the decision-making engine; and e. autonomously selecting, by the decision-making engine, an action to minimize a difference between the performance assessment and the at least one process goal, to improve performance of the industrial process, wherein the action comprises at least one of (i) updating a model used in the industrial process and (ii) generating updated training data related to the industrial process for a model used in the industrial process.

2. The computer-implemented method of claim 1, wherein the step of assessing the operational data comprises:- determining whether operational data (108) comprises new data; and- if it is determined operational data comprises new data, autonomously selecting, by the decision-making engine, a model to be updated and autonomously selecting, by the decision-making engine, one or more updates to implement to update the selected model.

3. The computer-implemented method of claim 2, wherein the step of determining whether operational data (108) comprises new data comprises:- providing a criterion against which to determine whether the provided operational data (108) comprises new data, optionally, wherein the supervisory layer autonomously selects the criterium;- assessing the provided operational data (108) against historical operational data with respect to the criterion to provide a geometrical distance between the provided operational data and the historical data; and- assessing, by the supervisory layer, at least the geometrical distance with respect to the criterium to determine whether the provided operational data comprises new data.

4. The computer-implemented method of claim 3, further comprising:- determining whether operational data (108) indicates steady state operation for a selected period of time; and- if it is determined operational data (108) indicates steady state operation for the selected period of time, comparing operational data (108) during the selected period of time against historical steady state operational data.

5. The computer-implemented method of prior claims, wherein the supervisory layer (104) comprises a machine learning algorithm (140) (MLA), the step of generating updated training data further comprising:- generating updated training data related to the industrial process for the MLA.

6. The computer-implemented method of claim 5, wherein the generating step comprises:- exciting the industrial process at least by implementing at least one pre-determined change to one or more selected MVs and / or CVs;- tagging operational data generated from the excitation to provide tagged operational data.

7. The computer-implemented method of claim 6, further comprising: providing the tagged operational data as the updated training data.

8. The computer-implemented method of claim 6 or claim 7, wherein the tagging comprises:- providing data associated with operation of the control layer in smooth signal format; and- providing data associated with operation of the industrial process in stepped signal format.

9. The computer-implemented method of any one of claims 6 - 8, further comprising: observing an impact to the industrial process by the excitation; and correlating the observed impact to the at least one pre-determined change using at least the tagged operational data.

10. The computer-implemented method of any one of claims 6 - 9, further comprising: selecting timing and duration of a period of time to perform the steps of exciting and tagging.

11. The computer-implemented method of claim 10, wherein the supervisory layer autonomously selects the timing and duration of the period of time to perform the steps of exciting and tagging.

12. A computer-based system for autonomous management of operation of an industrial process comprising a control layer (102) to control equipment of the industrial process, the system comprising:(i) a monitoring engine (128) to: a. receive operational data (108) related to the industrial process; b. receive at least one target goal (106) for the industrial process; and c. assess the operational data against the at least one target goal to provide a performance assessment (130) of the industrial process;(ii) a supervisory layer (104) to receive the performance assessment, wherein the supervisory layer comprises a decision-making engine; wherein the supervisory layer being capable of modifying one or more settings of the control layer and / or the decision-making engine; and wherein the supervisory layer being configured to autonomously select an action to minimize a difference between the performance assessment and the at least one process goal, to improve performance of the industrial process, wherein the actioncomprises at least one of (i) updating a model used in the industrial process and (ii) generating updated training data related to the industrial process for a model used in the industrial process.

13. The computer-based system of claim 12, wherein the decision-making engine being capable of determining whether operational data (108) comprises new data; and if it is determined operational data comprises new data, autonomously selecting, a model to be updated and one or more updates to update the selected model.

14. The computer-based system of claim 12, wherein the supervisory layer (104) comprises a machine learning algorithm (140), and wherein the generating updated training data further comprises:- generating updated training data related to the industrial process for the MLA.

15. The computer-implemented method of claiml4, wherein the decision-making engine being configured to autonomously selects timing and duration of period of time to perform:- exciting the industrial process at least by implementing at least one pre-determined change to one or more selected MVs and / or CVs; and- tagging operational data generated from the excitation to provide tagged operational data.

Images