Hydrocarbon system with autonomous optimization control

Abstract

A method executable by one or more processors includes: obtaining a measurement of a first variable at a current time step; using a first model to estimate a second variable at the current time step based on the measurement of the first variable; generating a control decision for a subsequent time step based on the measurement of the first variable and the estimate of the second variable using a reinforcement learning model; using a second model to predict a predicted value of the first variable for a subsequent time step based on the measurement of the first variable at the current time step; adjusting the control decision for the subsequent time steps based on constraints and future values ​​of the first variable; and controlling an actuator based on the control decision.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and benefit to U.S. Provisional Patent Application No. 63 / 472,132, filed June 9, 2023, the disclosure of which is incorporated herein by reference in its entirety. Background Technology

[0003] This disclosure relates to hydrocarbon sites. More specifically, this disclosure relates to networks or control systems for hydrocarbon sites, including but not limited to control systems using edge devices in industrial systems such as gas and oil extraction stations. Summary of the Invention

[0004] One implementation of this disclosure is a method executable by one or more processors. The method includes: obtaining a measurement of a first variable at a current time step; using a first model to estimate a second variable at the current time step based on the measurement of the first variable; generating a control decision for subsequent time steps based on the measurement of the first variable and the estimate of the second variable using a reinforcement learning model; using a second model to predict a predicted value of the first variable for subsequent time steps based on the measurement of the first variable at the current time step; adjusting the control decision for subsequent time steps based on constraints and future values ​​of the first variable; and controlling an actuator based on the control decision. Attached Figure Description

[0005] Figure 1 This is a perspective view of a hydrocarbon site equipped with a well device, according to some embodiments.

[0006] Figure 2 It is based on some implementation methods for Figure 1 A block diagram of the control system for hydrocarbon sites.

[0007] Figure 3 It is based on some implementation methods Figure 2 A block diagram of a part of the control system, showing a field controller that communicates with field devices, input devices, and output devices.

[0008] Figure 4 This is a diagram of a control system for a hydrocarbon system according to some embodiments.

[0009] Figure 5A This is a diagram of another control system for a hydrocarbon system according to some embodiments.

[0010] Figure 5B This is a diagram of another control system for a hydrocarbon system according to some embodiments.

[0011] Figure 5C This is an illustration of a simulator based on some implementation methods.

[0012] Figure 5D This is an illustration of another simulator for a hydrocarbon system according to some embodiments.

[0013] Figure 6 This is a block diagram of a control architecture for hydrocarbon systems according to some implementation methods.

[0014] Figure 7 This is a block diagram of a control architecture for hydrocarbon systems according to some implementation methods. Detailed Implementation

[0015] Before turning to the accompanying drawings, which illustrate certain exemplary embodiments in detail, it should be understood that this disclosure is not limited to the details or methods set forth in the specification or shown in the drawings. It should also be understood that the terminology used herein is for descriptive purposes only and should not be considered limiting.

[0016] Overview

[0017] Referring generally to the accompanying drawings, hydrocarbon sites can be operated, controlled, monitored, or serviced by a control system comprising various edge devices. This disclosure generally relates to providing autonomous, self-driven, and / or self-optimizing control systems that, for example, execute user-defined strategic task outlines by utilizing intelligent data packets deployed across distributed control systems in an efficient and scalable manner. The methods described herein can minimize non-value-adding reliance on human experts and maximize the potential of combined human systems. The systems and methods described herein can provide self-management of distributed computing resources and intelligent algorithms to adapt to unpredictable changes while hiding inherent complexities from operators and users. In some embodiments, the systems described herein include networks of sensors, controllers, devices, apparatuses, etc., configured to measure variables (e.g., process variables, environmental conditions, machine operating conditions, etc.), automatically think (e.g., analyze using trained domain expertise), automatically control processes (e.g., by controlling devices, actuators, apparatuses, etc.), and continuously improve performance via optimization techniques. The teachings described herein can provide progress toward zero-operator systems, for example, for applications in oil and gas equipment or other industrial equipment, such as for electric submersible pumps (ESPs), gas lift systems, chemical injection systems, etc.

[0018] Conventional process control techniques (including those involving some degree of predictive control or model-based operation) are well-suited to structured, relatively static environments, such as manufacturing line equipment. For other situations, such as equipment deployed in dynamic environments that change over time, and for various deployments, conventional process control techniques may lack the adaptability required for reliable operation over time and scalable deployment across diverse environments, systems, and usage scenarios, at least without significant human intervention to reprogram, reconfigure, retrain, build new models, etc., for each deployment and as conditions and dynamics change over time.

[0019] This disclosure relates to systems and methods advantageously configured for scalable, distributed deployment in environments such as oilfields with significant temporal and cross-deployment variability. The systems and methods described herein offer scalability (e.g., versatility), enabling easy deployment in multiple locations, across different systems, etc., without requiring extensive manual reprogramming or other intervention. The systems and methods described herein also provide for optimization over time, where continuous optimization and improvement deliver high value for deployments in environments, processes, systems, different hydrocarbon sites, different wells, etc., that are different from each other and dynamically change over time. These advantages provide for efficient initial deployment and temporal adaptation of systems and methods for control optimization, as described in further detail in the following paragraphs.

[0020] The teachings herein can be implemented using features disclosed in U.S. Patent Application Publication No. 2022-0018231, published January 20, 2022; U.S. Patent Application Publication No. 2022-0154889, published May 19, 2022; U.S. Patent Application Publication No. 2022-0180019, published June 9, 2022; and / or U.S. Patent Application Publication No. 2022-0170353, published June 2, 2022, the disclosures of which are incorporated herein by reference in their entirety.

[0021] System Overview

[0022] Hydrocarbon sites

[0023] Now refer to Figure 1Hydrocarbon site 100 can be an area from which hydrocarbons such as crude oil and natural gas can be extracted from the ground, processed, and / or stored. Therefore, hydrocarbon site 100 can include multiple wells and multiple well assemblies that can control the flow of hydrocarbons extracted from the wells. In one embodiment, the well assemblies at hydrocarbon site 100 can include any devices equipped to monitor and / or control the production of hydrocarbons at the well site. Therefore, well assemblies can include pumping units 32, submersible pumps 34, Christmas trees 36, and other devices for monitoring and controlling the flow of liquids or gases (e.g., oil, natural gas, and other substances). After hydrocarbons are extracted from the surface via the well assemblies, the extracted hydrocarbons can be distributed to other devices such as wellhead distribution manifolds 38, separators 40, storage tanks 42, and other devices for measuring, monitoring, separating, storing, and controlling the flow of liquids or gases (e.g., oil, natural gas, and other substances). At hydrocarbon site 100, pumping unit 32, submersible pump 34, Christmas tree 36, wellhead distribution manifold 38, separator 40, and storage tank 42 can be connected together via a network of pipes 44. Therefore, hydrocarbons extracted from the reservoir can be transported to various locations at hydrocarbon site 100 via the network of pipes 44.

[0024] When the bottomhole pressure is insufficient to extract hydrocarbons to the surface, pumping unit 32 can mechanically lift hydrocarbons (e.g., oil) out of the well. Submersible pump 34 can be a component that can be submerged in the hydrocarbon liquid to be pumped. Therefore, submersible pump 34 can include a hermetically sealed motor, preventing liquid from penetrating the seals and entering the motor. Furthermore, the hermetically sealed motor can push hydrocarbons from underground areas or reservoirs to the surface.

[0025] The Christmas tree 36, or production tree, can be an assembly of valves, valve cores, and fittings for flowing wells. Therefore, the Christmas tree 36 can be used in oil wells, gas wells, water injection wells, water treatment wells, gas injection wells, condensate wells, etc. The wellhead distribution manifold 38 can collect hydrocarbons that may have been extracted by the pumping unit 32, submersible pump 34, and Christmas tree 36, allowing the collected hydrocarbons to be guided along pathways to various hydrocarbon processing or storage areas within the hydrocarbon site 100.

[0026] The separator 40 may include a pressure vessel that can separate well fluids generated from oil and gas wells into separate gaseous and liquid components. For example, the separator 40 can separate hydrocarbons extracted by pumping unit 32, submersible pump 34, or Christmas tree 36 into petroleum, gaseous, and water components. After the hydrocarbons are separated, each separated component can be stored in a specific storage tank 42. The hydrocarbons stored in the storage tank 42 can be transported via pipeline 44 to transport vehicles, refineries, etc.

[0027] The well apparatus may also include a monitoring system, which may be placed at various locations within the hydrocarbon site 100 to monitor or provide information relating to certain aspects of the hydrocarbon site 100. Therefore, the monitoring system may be a controller, remote terminal unit (RTU), or any computing device that may include communication capabilities, processing capabilities, etc. For the purposes of discussion, the monitoring system is embodied as RTU 46 throughout this disclosure. However, it should be understood that RTU 46 may be any component capable of monitoring and / or controlling various parts at the hydrocarbon site 100. RTU 46 may include sensors, or may be coupled to various sensors capable of monitoring various properties associated with the parts at the hydrocarbon site 100.

[0028] RTU 46 can then analyze various properties associated with the component and control various operating parameters of the component. For example, RTU 46 can measure the pressure or differential pressure of a well or component (e.g., storage tank 42) in hydrocarbon site 100. RTU 46 can also measure the temperature of the contents stored inside the component in hydrocarbon site 100, the amount of hydrocarbons processed or extracted by the component in hydrocarbon site 100, etc. RTU 46 can also measure the level or amount of hydrocarbons stored in a component such as storage tank 42. In some embodiments, RTU 46 can be an iSens-GP pressure transmitter, iSens-DP differential pressure transmitter, iSens-MV multivariable transmitter, iSens-T2 temperature transmitter, iSens-L level transmitter, or Isens-1O flexible I / O transmitter manufactured by vMonitor® in Houston, Texas.

[0029] In one embodiment, RTU 46 may include sensors capable of measuring pressure, temperature, fill level, flow rate, etc. RTU 46 may also include a transmitter, such as a radio wave transmitter, capable of transmitting data acquired by the sensors via an antenna or the like. The sensors in RTU 46 may be wireless sensors capable of receiving and transmitting data signals between RTUs 26. To power the sensors and transmitter, RTU 46 may include a battery or be coupled to a continuous power source. Because RTU 46 may be installed in harsh outdoor and / or explosive environments, RTU 46 may be encapsulated in an explosion-proof container that meets certain standards established by the National Electrical Manufacturers Association (NEMA), such as NEMA 4X containers, NEMA 7X containers, etc.

[0030] The RTU 46 can transmit data acquired by sensors or processed by a processor to other monitoring systems, router devices, Supervisory Control and Data Acquisition (SCADA) devices, etc. Therefore, the RTU 46 allows users to monitor various properties of various components in the hydrocarbon site 100 without physically being located near the corresponding components. The RTU 46 can be configured to communicate with devices and mobile computing devices at the hydrocarbon site 100 via various networking protocols.

[0031] During operation, RTU 46 can receive real-time or near-real-time data associated with the well equipment. This data may include, for example, head pressure, head temperature, casing head pressure, streamline pressure, wellhead pressure, and wellhead temperature. In any case, RTU 46 can analyze real-time data relative to static data that can be stored in its memory. Static data may include well depth, tubing length, tubing size, nozzle size, reservoir pressure, bottom hole temperature, well test data, and fluid properties of extracted hydrocarbons. RTU 46 can also analyze real-time data relative to other data acquired by various types of instruments (e.g., cutter gauges, multiphase meters) to determine inflow dynamics (IPR) curves, the desired operating point of wellhead 30, key performance indicators (KPIs) associated with wellhead 30, and wellhead performance summary reports. Although RTU 46 can perform the analyses mentioned above, it may not be able to perform these analyses in a timely manner. Furthermore, by relying solely on the processor capabilities of the RTU 46, the RTU 46 is limited in the quantity and type of analyses it can perform. Additionally, its data storage capacity may also be limited due to its potentially size-limited nature.

[0032] In some implementations, RTU 46 can establish a communication link with the cloud-based computing system 12 described above. Therefore, the cloud-based computing system 12 can utilize its greater processing power to analyze data acquired by multiple RTUs 26. Furthermore, the cloud-based computing system 12 can access historical data associated with a corresponding RTU 46, data associated with well devices associated with a corresponding RTU 46, and data associated with hydrocarbon sites 100 associated with a corresponding RTU 46, etc., to further analyze the data acquired by the RTUs 46. The cloud-based computing system 12 communicates with the RTUs via one or more servers or networks (e.g., the Internet).

[0033] Field control system

[0034] Specific reference Figure 2The diagram illustrates a control system 200 (e.g., a network) for a hydrocarbon site 100 according to some embodiments. In some embodiments, the control system 200 includes or is configured to communicate with a cloud computing system 202 and is configured to control various operations of the site (e.g., hydrocarbon site 100) based on analyzing metadata from various devices within the control system 200. The cloud computing system 202 may include any processing circuitry, processors, memory, etc., or combinations thereof located remotely from the hydrocarbon site 100. In various embodiments, some or all of the processing circuitry, processors, memory, etc., or combinations thereof within the cloud computing system 202 may be executed by various devices disclosed within the control system 200. The control system 200 is also shown as including an edge device 204, a workstation 208, and a field controller 210.

[0035] Edge device 204 can be configured to run, execute, implement, store, etc., one or more of applications 206. Additionally, some or all of the processing circuitry, processors, memories, etc., in various devices included within the control system 200 (e.g., edge device 204, field controller 210, workstation 208, etc.) can be distributed across several other devices within the control system 200 or integrated into a single device. Edge device 204 can be configured to receive data from field controller 210 and provide data analysis to cloud computing system 202 based on the received data. The following will refer to... Figure 3 This will be described in more detail.

[0036] In some implementations, each edge device 204 includes a processing circuitry system with a processor and memory. The processor may be a general-purpose or special-purpose processor, an application-specific integrated circuit (ASIC), one or more field-programmable gate arrays (FPGAs), a set of processing units, or other suitable processing units. According to some implementations, the processor is configured to execute computer code or instructions stored in memory or received from other computer-readable media (e.g., CD-ROM, network storage device, remote server, etc.).

[0037] In some embodiments, the memory may include one or more means (e.g., memory cells, memory devices, storage devices, etc.) for storing data and / or computer code to implement and / or facilitate the various processes described herein. The memory may include random access memory (RAM), read-only memory (ROM), hard disk drive storage devices, temporary storage devices, non-volatile memory, flash memory, optical memory, or any other suitable memory for storing software objects and / or computer instructions. The memory may include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described herein. The memory may be communicatively connected to a processor via a processing circuitry system and may include computer code for performing (e.g., executed by the processor) one or more of the processes described herein.

[0038] The field controller 210 can be configured to control various operations at the well site and is communicatively coupled to the edge device 204. In some embodiments, the field controller 210 is configured to operate (e.g., provide control signals to field devices, provide setpoints to field devices, adjust setpoints or their operating parameters) the field devices (e.g., electric submersible pumps (ESPs), cranes, pumps, etc.) of the hydrocarbon site 100. The field controller 210 can be grouped into different sets based on which edge device 204 it communicates with. In some embodiments, the edge device 204 is configured to exchange any sensor data, measurement data, instrument data (e.g., flow meter data), control signals, stored data, maintenance data, setpoint adjustment results, operation adjustment results, diagnostic data, analytical data, metadata, etc., with the field controller 210. It should be understood that each edge device 204 can be associated with, correspond to, etc., multiple field controllers 210. In some implementations, metadata may include a description of the device or the name of the device, communication identifiers, port identifiers, unit value identifiers, range identifiers, signal type (e.g., analog or digital), data hierarchy, data redundancy for data or sensors providing data, or sensor identifiers, etc.

[0039] In some embodiments, one or more of the field controllers 210 may include a computing engine 212. The computing engine 212 may be configured to perform various control functions, diagnostic functions, analysis functions, reporting functions, metadata-related functions, etc. The computing engine 212 may be embedded in one or more of the field controllers 210, or it may be embedded in one or more of the edge devices 204. In some embodiments, any of the functions of the computing engine 212 are distributed across multiple edge devices 204 and / or multiple field controllers 210. In some embodiments, any of the functions of the computing engine 212 are performed by the cloud computing system 202.

[0040] Still refer to Figure 2 Workstation 208 can be configured to receive user instructions for controlling hydrocarbon site 100 and to provide control signals to various devices via control system 200. Workstation 208 may include any desktop computer, laptop computer, personal computer device, user interface, personal computer device, etc., or any general-purpose computing device thereof. In some embodiments, multiple workstations 208 (e.g., any...) A number of workstations 208 are associated with each edge device 204, while in other embodiments, one or more edge devices 204 are associated with a single workstation 208.

[0041] In some implementations, the field controller 210 may be configured to act as an edge device, performing additional processing (e.g., data analysis, mapping, etc.) before providing information to the cloud computing system 202. In some implementations, this reduces latency in information processing to the cloud computing system 202. In other implementations, the edge device 204 operates as a conventional edge device and performs significant storage and processing within the control system 200 (e.g., in the field, at / near hydrocarbon site 100, etc.) to mitigate latency caused by processing information in the cloud computing system 202.

[0042] Field controller

[0043] Now refer to Figure 3 This document illustrates a control system 200 for performing control of an output device 304 based on an input device 302, according to an exemplary embodiment. The control system 200 is shown as including an edge device 204 containing an application 206, a cloud computing system 202, a field controller 210, a field device 310, an input device 302, and an output device 304.

[0044] Input device 302 can be configured to provide various sensor data and / or field measurement results from hydrocarbon site 100 to field controller 210 for processing. For example, sensor 306 of input device 302 measures the pump speed of pump 34. Sensor 306 provides the pump speed of pump 34 to field controller 210 at regular intervals (e.g., continuously, every minute, every 5 minutes, etc.). Input device 302 can be wired or wirelessly connected to field controller 210 or any other device within system 200 (e.g., edge device 204). In some embodiments, input device 302 is coupled to various field devices (e.g., pumps, pumping units, cranes, etc.) and provides operational data of its respective field device to field controller 210.

[0045] Output device 304 can be configured to receive control signals from field controller 210 and adjust operation based on the received control signals. For example, field controller 210 determines that pump 34 is operating at a speed lower than considered optimal. Field controller 210 then sends a control signal to output device (e.g., actuator) 304 to increase the pump speed of pump 34. In some embodiments, output device 304 is configured to act as any device (e.g., actuator, etc.) capable of adjusting the operation of field equipment within hydrocarbon site 100. In some embodiments, various other field devices (e.g., field device 310) include some or all of the functions of input device 302 and output device 304 and provide sensor data and receive control signals from field controller 210.

[0046] In some implementations, control system 200 is configured to analyze various datasets (e.g., metadata) to determine the optimal control scheme for hydrocarbon site 100. The extensive processing required for this can be performed by an edge device (e.g., edge device 204) rather than processing all the metadata analysis in the cloud, as processing data at or near the site reduces latency compared to sending data to cloud computing system 202 for processing. For example, sensor 306 provides metadata to field controller 210. Field controller 210 processes the data to determine the type of data and / or the domain from which it receives data, and provides the data to edge device 204 for analysis. Applications within edge device 204 (e.g., application 206) can analyze the metadata to make decisions about the control scheme that would otherwise not be noticed by the processing within control system 200. For example, application 206 can infer that the received data was received by a flow meter sensor (e.g., sensor (1) 306) based on patterns found in the data and prior data already analyzed by edge device 204. Application 206 can make inferences, predictions, and calculations based on current and / or past data.

[0047] In some implementations, application 206 provides some or all of the data to cloud computing system 202 for further processing. Application 206 can be configured to infer from the received data, which improves the standardization of data analysis. For example, sensor (1) 306 and sensor (2) 306 may be flow sensors, but from different vendors. Therefore, sensor (1) 306 may provide data to field controller 210 in a different format than sensor (2) 306. However, application 206 of edge device 204 can still be able to standardize the data and determine that the two datasets come from flow sensors, even though the received data are in different formats (e.g., one dataset is provided under the Resource Description Framework (RDF) specification, and one dataset is provided as a data object, etc.). In various implementations, enabling edge device 204 to perform some or all of the metadata analysis can achieve improved data analysis and control schemes without significantly increasing processing latency.

[0048] Model-based control and AI agents

[0049] Now refer to Figure 4 A block diagram of a system 400 for providing optimal control of a physical system, according to some embodiments, is shown. The system can be provided via programming instructions stored on one or more non-transitory readable media and one or more processors operable to execute such programming instructions to perform the operations described herein and provide... Figure 4 The models, system identifiers, predictions, reinforcement learning, constraint filters, etc., shown and described herein. The system can be provided on edge hardware via remote computing resources (e.g., cloud resources and remote servers geographically distant from one or more actuators, sensors, and / or physical systems, etc.) (e.g., provided as part of, physically coupled to, and / or geographically close to, actuators, sensors, and / or physical systems), said remote computing resources being, for example, capable of communicating with one or more actuators, sensors, and / or physical systems via the Internet or other networks and / or distributed across any combination of such computing devices. For example, Figure 4 The components can be made from Figure 3 The field controller n 21, the field device n 310, the cloud computing system 202, and the edge device n 204, or a combination thereof, are provided in [the context of the provided text]. Figure 3 Provided on any one or a combination of the field controller n 210, field device n 310, cloud computing system 202, and edge device n 204, as... Figure 3 Provided as part of any one or a combination of the field controller n 210, field device n 310, cloud computing system 202 and edge device n 204.

[0050] Figure 4 The system is shown as including physical system 402. Physical system 402 is a dynamic environment including one or more actuators. The operation of one or more actuators affects one or more states of the system (e.g., measurable conditions, measured variables and / or unmeasured variables, such as temperature, pressure, speed, frequency, position, flow rate, resource consumption, power usage, etc.). One or more actuators may include one or more pumps, motors, valves (e.g., electric actuators operable to provide opening and closing of valves), power electronics, frequency converters, or other devices or means operable to physically affect the dynamics of physical system 402. Figure 4 The physical system 402 may include one or more sensors that measure one or more variables representing one or more states of the physical system 402. (Indicates time step) (Variable values). In some embodiments, physical system 402 is a hydrocarbon system. In various embodiments, physical system is hydrocarbon site 100 or any element or collection of elements of site 100.

[0051] Figure 4 A control system 404 is shown, including a digital twin element 406 and a safety-optimized controller (SOC) 408. The digital twin element 406 can be implemented using the teachings of U.S. Patent Application Publication No. 2022-0180019, filed December 7, 2021, the disclosure of which is incorporated herein by reference in its entirety.

[0052] Digital twin element 406 is shown as including one or more first models (model / ROM) 410 and one or more second models (data-driven system identifiers) 412. One or more first models 410 are configured to estimate the state or condition of the physical system, such as a state or condition of the physical system that is not directly measurable or cannot be directly measured (e.g., due to the absence of certain sensors, due to inherent unmeasurable characteristics of the state or condition, due to data flow or network limitations). The estimate may be a reward-related variable and may be indicated as representing the reward-related variable. In time value The estimated value can be obtained from one or more first models based on the measured variables. It is generated from at least a subset of the data. In various implementations, the estimated values ​​may be referred to as virtual points, dummy variables, synthetic variables, etc.

[0053] One or more first models 410 can be any type of model, such as a reduced-order model (ROM). In some embodiments, one or more first models 410 can be a digital twin of a physical system or include a digital twin of a physical system, such as the digital twin in U.S. Patent Application Publication No. 2022-0180019, filed December 7, 2021, the disclosure of which is incorporated herein by reference in its entirety. In some embodiments, one or more first models 410 include a physics-based first-principle model (equations, simulations, etc.), for example, suitable for calculating given measurement variables. The values ​​of unmeasured variables appear in the case of values. In some implementations, one or more first models 410 are inherently general / scalable, so that one or more first models do not need to be adapted, retrained, reconfigured, etc. for different deployments, but can be easily and efficiently deployed in different instances and for different physical systems.

[0054] One or more second models 412 are configured to predict the future values ​​of one or more variables of the physical system. For example... Figure 4 As shown, one or more second models 412 are based on The measured values ​​for ( At time step The value of is used to make a forward prediction for one time step. In other implementations, a multi-step forward prediction range is used. However, for one or more second models 412, it may be advantageous to focus on single-step forward prediction to ensure compliance with constraints with relatively low computational complexity, as discussed in further detail below.

[0055] One or more second models 412 are shown as being generated using data-driven system identifiers. The parameters, weights, etc., of one or more second models 412 can be fitted (trained, identified) using historical and / or simulated (synthetic) data related to the operation of the physical system to provide a data-driven system identifier for one or more second models 412. In some embodiments, one or more second models 412 include gray-box models having a structure based on the physical principles of the physical system and the parameters identified via the data-driven system identifier. In some embodiments, one or more second models 412 include neural networks (e.g., recurrent neural networks, long short-term memory), generative pre-trained converters (e.g., large language models), or other artificial intelligence models configured to process time-series data. As more data related to the dynamics of the physical system becomes available (e.g., (Based on further measurements), one or more second models 412 can be automatically retrained over time, for example, to automatically adapt one or more second models 412 to the dynamics of the physical system as it changes over time. In some implementations, one or more second models 412 comprise multiple models selected among them based on characteristics of the input data (e.g., based on the position of the measurement variables in the modeling space), wherein different models perform better on different input data.

[0056] like Figure 4 As shown, the system also includes a Safety Optimal Controller (SOC) 408. The SOC is shown as comprising a reinforcement learning model 414 and a constraint engine 416. The reinforcement learning model 414 is shown as receiving variables. and estimated variables The values ​​of (dummy variables, synthetic variables, etc.). The reinforcement learning model 414 may, for example, use a reward function to calculate the reward using the values ​​of the measured and / or estimated variables. For example, the reward function may output a numerical value that reflects the degree to which the measured and / or estimated variables reflect the extent to which the goal of the physical system is achieved. Subsequently, the reinforcement learning model may simultaneously perform the following: (1) generate optimistic executions for one or more actuators of the physical system that are expected to increase the reward (drive the system toward the goal). (2) and (3) perform self-retraining to improve its ability to make the system provide better (e.g., higher, lower) values ​​of the reward function. The reinforcement learning model 414 can be a neural network or other artificial intelligence model and can use proximal policy optimization or policy gradient-based learning methods to improve its output over time and adapt to changing system dynamics.

[0057] The output of reinforcement learning model 414 is shown as optimistic execution. The output is called optimistic because it represents the best possible execution (e.g., setup, control signals, commands, setpoints, target locations, on / off decisions, etc.) taken by one or more actuators of the physical system before considering the constraints imposed by constraint engine 416 and discussed below. The reinforcement learning model 414, as shown, is constructed, trained, operated, etc., without directly (explicitly, etc.) including constraints, which (in some embodiments) include a reward function used by the reinforcement learning model 414 (i.e., making the reinforcement learning model 414 independent of any constraints, etc.). Therefore, compared to methods where the optimization problem or model is formulated with constraints directly contained therein (e.g., as a large system of equations that may include intractable nonlinearities, etc.), the reinforcement learning model 414 can be structurally simpler and more efficient to construct (e.g., because the same structure can be used regardless of physical constraints), train (e.g., can be trained on less data), and execute (e.g., due to the relative simplicity of the model).

[0058] Optimistic execution from reinforcement learning model 414 The output is provided as input to constraint engine 416, which is shown to include actuator constraint filter 418 and response constraint filter 420. Actuator constraint filter 418 may impose constraints related to the physical limitations of actuator operation (e.g., maximum or minimum actuator capacity, frequency, speed, etc., actuator position limitations, etc.) to ensure that the execution provided by control system 404 can be satisfied by the actuator (e.g., actuator 308 which may be included in physical system 402) within the actuator's operational limitations.

[0059] Response constraint filter 420 can impose constraints (e.g., variables) on the physical response of physical system 402. or The constraints are related to the value limits. These limits can be desired boundaries (e.g., preferred operating ranges) or critical limits, outside which damage or other adverse consequences are expected to occur to the physical system 402. Therefore, continued compliance with these limits is essential for the proper operation of the system and can be achieved through… Figure 4 The implementation method shown is used to achieve this.

[0060] exist Figure 4 In the system, the actuator constraint filter 418 and the response constraint filter 420 adopt optimistic execution from the reinforcement learning model 414. As input, and output the constrained optimal execution. (Where "constrained" means ensuring compliance with the constraints imposed by the actuator constraint filter 418 and the response constraint filter 420). The actuator constraint filter and the response constraint filter can be implanted using control barrier functions and / or quadratic programming, for example, to find the constrained optimal execution. The constrained optimal execution tracks the optimistic execution as closely as possible. This ensures compliance with both actuator constraints and response constraints. Actuator constraint filter 418 and / or response constraint filter 420 can use data from one or more second models. Prediction is used to determine the constrained optimal execution. For example, used to determine a given predicted value Under the condition of constrained optimal execution Will provide time-step (constraints).

[0061] Advantageously, Figure 4 The method of the safety-optimal controller 408 in this paper (where "safe" or "safety" in this paper means essentially guaranteeing compliance with actuator constraints and / or response constraints) ensures compliance with constraints, while other methods (e.g., constraints are imposed as penalties in the reward function of a reinforcement learning model) may only penalize non-compliance and occasionally lead to non-compliance. Therefore, the teachings in this paper are well-suited to scenarios where compliance with constraints is critical for the operation of a physical system without damaging the device or causing other significant adverse consequences, while providing an easily trainable, self-improving reinforcement learning model to optimize operation.

[0062] Constrained optimal execution It is shown to be provided from constraint engine 416 to physical system 402. One or more actuators of physical system 402 (e.g., actuator 308) perform optimal execution according to constraints. At time step The operation will take place during this period. Physical system 402 will operate at time step... The process evolves dynamically, affecting states, variables, etc. (e.g., The value of ) at time step The process changes over time. Then it can be repeated at subsequent time steps. Figure 4 The control method enables the iterative execution of the above process over time to provide optimized operation of the physical system 402 in a manner that conforms to constraints.

[0063] As system 400 operates iteratively over time, reinforcement learning model 414 will be based on the constrained optimal execution. The resulting physical system is dynamically updated. Therefore, the reinforcement learning model 414 can implicitly consider the optimistic execution by the constraint engine 416. The adjustments made are adapted over time to minimize any suboptimal aspects that might arise from the operation of the constraint engine 416 while ensuring compliance with constraints. One or more first models 410 and one or more second models 412 can also be updated based on data generated through such iterations. This provides a complex control system comprising interrelated models and constraint filters that is modularly and holistically self-improving, providing optimized operation of the physical system 402 with computational efficiency while ensuring compliance with the constraints on the physical system 402.

[0064] Now refer to Figures 5A to 5D This illustrates some implementation methods. Figure 4 Various combinations of the components shown. Figures 5A to 5D Different combinations of one or more first models, one or more second models, reinforcement learning models, actuator constraint filters and / or response constraint filters (including omissions above) can be provided in various implementations.

[0065] Figure 5A The illustration includes a system 500, which comprises a physics simulator 502 providing a simulation of a physical system 402, a first model 410 for estimating reward-related variables based on measurement variables (e.g., measurement variables simulated by the physics simulator 52), a reinforcement learning model 414, and an actuator constraint filter 418. Figure 5A In the example shown, the following is omitted Figure 4 Other components. Such an implementation can be used to train the reinforcement learning model 414, for example, by initial training using the physics simulator 502 before deploying the reinforcement learning model 414 for online control. Thus, as Figures 5A to 5D The physical simulator 502 shown can be a real physical system, for example, including at least one actuator and / or at least one sensor (e.g., such as...). Figure 1 The hydrocarbon system (physical system 402) is replaced, for example, for the purpose of active online control and / or manipulation of real-world data as taught herein. The first model 410 can be provided as an automatic event detection model. As shown, the reinforcement learning model 414 can use the task and initial training state of the system 500 as input. For example, as shown, the reinforcement learning model 414 receives input indicating that the task of the system 400 is to remove the system from a low-flow condition (e.g., user input). The user input can be used to provide a reward relation that provides optimistic execution. To complete the task.

[0066] Figure 5BThe illustration includes a system 520 comprising a dynamic physics simulator 502 providing a simulation of a physical system 402 (or, in various embodiments, the physical system 402 itself), one or more second models 412 providing predictions (e.g., multi-step forward predictions) using current values ​​of variables from the dynamic physics simulator 502, and a model prediction controller 522 including actuator constraints 524 and response constraints 526 as components of an optimization performed by an optimizer 528 to directly output a constrained optimal execution based on predictions from one or more second models 412. The one or more second models 412 may include one or more forward system identification models. Thus, system 520 can provide model prediction control of the physical system 402 and / or its simulation, wherein the model prediction controller 522 operates to cause the physical system 402 (or simulator 502) to operate according to an optimal execution determined to minimize (e.g., maximize the optimization task overview) an objective subject to actuator constraints 524 and response constraints 526.

[0067] Figure 5C A diagram of a hypothetical simulator 540 is shown, wherein the hypothetical simulator includes a physical simulator 502 (providing a simulation of the physical system) and one or more second models 412 (e.g., forward system identification models) for predicting future values ​​of one or more variables based on current or historical values ​​provided by the physical simulator 502. According to various embodiments, such embodiments can deploy the above teachings for simulating, predicting future states, etc. The hypothetical simulator 540 can receive user input that varies on demand with initial states, physical system configurations, and / or system properties, and is configured to output predictions of one or more future values ​​of one or more variables. The output of the hypothetical simulator 540 can be displayed on a graphical user interface, otherwise communicated to the user, or used for further analysis and / or control operations.

[0068] Figure 5D It shows the relationship with Figure 4The illustration of a consistent system 550 is provided, but system 550 includes a dynamic physics simulator 502 instead of physical system 402 and further includes a user interface 552. Such an implementation can be used offline to generate data for training and validating various models and combinations thereof, such as initializing model training before sufficient historical data is available for a particular physical system. Such an implementation can also be used to generate simulation data that facilitates the design of physical systems, determine whether to invest in physical systems, etc. The user interface 552 may include a graphical user interface displayed on a personal computing device (e.g., desktop computer, laptop computer, tablet computer, smartphone, augmented reality headset, or virtual reality headset) and may display the results of such simulations of system 550 to the user, enabling the user to provide input related to: the configuration of the physical system (e.g., included equipment, included sensors, dimensions, physical layout, etc.), settings or conditions to be considered within the dynamic physics simulator 502 (e.g., well conditions, geological influences, etc.), and / or adjustments to aspects of the digital twin element 406 and / or the safety-optimized controller 408 (e.g., adjusting optimization objectives, modifying reward functions, adjusting actuator constraints or response constraints, etc.). Therefore, system 550 can provide users with the ability to set, adjust, and run simulations based on user input and selections. In some embodiments, user interface 552 includes an artificial intelligence agent configured to orchestrate the operation of dynamic physics simulator 502 and / or control system 404 based on user queries and requests.

[0069] Now refer to Figure 6 The diagram illustrates a system architecture 600 according to some embodiments. The system is shown as including an interconnected well network 602, which comprises any number of wells (shown as well A 604, well B 606, up to well n 607). The interconnected well network 602 can be fluidly connected, such as geological (e.g., underground) fluid connections and / or fluid connections in pipes, devices, etc., that receive the outputs of wells 602. Figure 6As shown, each well in the interconnected well network is served by artificial intelligence agents (shown as AI agent A 608, AI agent B 610, to AI agent n 612) and automated event detection (AED) tools (shown as automated event detector A 614, automated event detector B 616, to automated event detector n 618). A system-level artificial intelligence agent 620 coordinates and manages multiple well-connected artificial intelligence agents 608, 610, and 612. Automated event detection tools 614, 616, and 618 can be configured to automatically detect events occurring in their corresponding wells 604, 606, and 607, for example, using rule-based or model-based event detection techniques. Such event information can then be used by the corresponding AI agents 608, 610, and 612 to make operational decisions regarding the corresponding wells 604, 606, and 607, with the system AI agent 620 providing supervisory coordination for the AI ​​agents 608, 610, and 612. In some implementations, the system AI agent 620 can interoperate with AI agents 608, 610, and 612 to react and interact, thereby addressing (or minimizing) as many event severities as possible while optimizing the reward function. Therefore, Figure 6 This paper demonstrates how the various teachings presented in this paper can be implemented in a distributed manner within AI-enabled networks to provide scalable systems with multi-level intelligence, thereby enabling autonomous operation, for example, interconnected well networks.

[0070] Now refer to Figure 7 The diagram illustrates a system architecture 700 according to some embodiments. The system is shown as including an interconnected well network 702, which comprises any number of wells (shown as well A 704, well B 706, to well n 707). The interconnected well network 702 can be fluidly connected, such as geologically (e.g., underground) fluidly connected and / or fluidly connected via pipes, devices, etc., that receive the outputs of well 702. Figure 7 As shown, each well in the interconnected well network 702 is served by a control system (shown as control system A 714, control system B 716, to control system n 717), which can be implemented as follows: Figure 4Instances of control systems 404 include, for example, reinforcement learning models, constraint filters (e.g., actuator constraint filters and response constraint filters as described above), and digital twin elements such as one or more first and / or second models (e.g., system identification models) as described above. The control system receives measurements or other data from the corresponding wells and controls them (e.g., control system A 714 controls the equipment of well A 704, control system B 716 controls the equipment of well B, and control system n 718 controls the equipment of well n). Each well also has an associated AI agent, shown as AI agent A 708, AI agent B 710, to AI agent n 712, which can provide autonomous orchestration of the operations of the corresponding control system; thus, each AI agent can perform operations of the corresponding control system (e.g., by implementing the above-referenced...). Figure 4 (Detailed description of features) to control each corresponding well. A system-level AI agent 720 is also provided and shown as providing supervisory control of multiple AI agents 708, 710, 712 associated with different wells in the interconnected well network 702. In some embodiments, for example, the system-level AI agent 720 may provide adjusted constraint and / or reward functions to the multiple AI agents 708, 710, 712 for use by the control systems 714, 716, 718 to coordinate the operation of the interconnected well network 702, for example, such that changes affecting one well in the interconnected well network 702 can be proactively handled by the AI ​​agent system.

[0071] In some implementations, the system-level AI agent 720 orchestrates the operation of multiple AI agents 708, 710, 712 by making the reward function, constraints, and / or predictions for a first well (e.g., well A704) based on one or more variables associated with the first well (e.g., representing the condition, performance, etc. of well A704) and also based on one or more additional variables associated with one or more additional wells (e.g., variables representing the condition, performance, etc. of wells B706 to B707). For example, the reward function and / or constraints may be based on the sum, difference, product, or ratio of the variables of well A704 and the additional variables of well B706 (e.g., the sum of power consumption values, the sum of flow rates, the ratio of pump rates, the difference in temperature, etc.). As another example, the prediction of future values ​​for the first well A704 may be based on data from well B706, thus taking into account the physical influence of well B706 on well A704. Therefore, various such interrelationships between interconnected wells can be handled through various implementations of the teachings herein.

[0072] therefore, Figure 7 The scalability and adaptability of the systems and methods disclosed herein are demonstrated.

[0073] In some aspects, this disclosure relates to a non-transitory computer-readable medium containing one or more stored program instructions that, when executed by one or more processors, cause one or more processors to perform operations including providing an artificial intelligence agent. The artificial intelligence agent may include: a first model configured to estimate a second variable at a current time step based on a measurement of a first variable; a second model configured to predict a value of the first variable at a subsequent time step; a reinforcement learning model configured to output a control decision for a subsequent time step based on the measurement of the first variable and the estimate of the second variable; a constraint engine configured to adjust the control decision based on the predicted value of the first variable to ensure that the system conforms to constraints at a subsequent time step; and a control engine configured to operate the system using the adjusted control decision.

[0074] In some aspects, this disclosure relates to a non-transitory computer-readable medium containing one or more stored program instructions that, when executed by one or more processors, cause one or more processors to perform operations including providing an artificial intelligence agent. The artificial intelligence agent may include: a predictive model configured to predict a first variable at a subsequent time step; a reinforcement learning model configured to output control decisions for subsequent time steps based on measurements of the first variable; a constraint engine configured to adjust the control decisions based on the predicted values ​​of the first variable to ensure compliance with constraints at subsequent time steps; and a control engine configured to operate a physical system using the adjusted control decisions.

[0075] Configuration of exemplary implementation

[0076] As used herein, the terms “about,” “approximately,” “substantially,” and similar terms are intended to have a broad meaning consistent with common and accepted usage by one of ordinary skill in the art to which the subject matter of this disclosure pertains. Those skilled in the art who read this disclosure will understand that these terms are intended to allow for the description of certain features described and claimed, without limiting the scope of those features to the precise numerical ranges provided. Therefore, these terms should be interpreted as indicating that non-substantial or irrelevant modifications or alterations to the described and claimed subject matter are considered to be within the scope of this disclosure set forth in the appended claims.

[0077] It should be noted that the term “exemplary” and its variations, as used herein to describe various implementations, are intended to indicate that such implementations are possible examples, representations or illustrations of possible implementations (and such terms are not intended to imply that such implementations are necessarily unusual or the best examples).

[0078] As used herein, the term "coupled" and its variations mean two components directly or indirectly joined to each other. Such a connection can be static (e.g., permanent or fixed) or movable (e.g., removable or releasable). Such a connection can be achieved by directly coupling two components to each other, using a separate intervening component and any additional intermediate components coupled to each other, or using an intervening component integrally formed with one of the two components to become a single, integral body. If "coupled" or its variations are modified by an additional term (e.g., direct coupling), the general definition of "coupled" provided above is modified by the colloquial meaning of the additional term (e.g., "direct coupling" means two components joined without any separate intervening component), resulting in a narrower definition than the general definition of "coupled" provided above. Such a coupling can be mechanical, electrical, or fluid.

[0079] As used herein, the term "or" is used in its inclusive sense (rather than its exclusive sense), such that when used to connect a list of elements, the term "or" means one, some, or all of the elements in the list. Unless otherwise specified, connective language such as the phrase "at least one of X, Y, and Z" is understood to mean that the elements can be: any one of X, Y, and Z; X and Y; X and Z; Y and Z; or X, Y, and Z (i.e., any combination of X, Y, and Z). Therefore, unless otherwise indicated, such connective language is generally not intended to imply that certain implementations require at least one of X, at least one of Y, and at least one of Z to be present individually.

[0080] References to the position of elements (e.g., “top,” “bottom,” “above,” “below”) herein are used only to describe the orientation of the various elements in the accompanying drawings. It should be noted that the orientation of the various elements may differ according to other exemplary embodiments, and such variations are intended to be covered by this disclosure.

[0081] Hardware and data processing components for implementing the various processes, operations, illustrative logic, logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed by a general-purpose single-chip processor or multi-chip processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor or any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, a combination of one or more microprocessors with a DSP core, or any other such configuration. In some embodiments, specific processes and methods may be performed by a circuit system specific to a given function. Memory (e.g., memory, storage cell, storage device) may include one or more means (e.g., RAM, ROM, flash memory, hard disk storage device) for storing data and / or computer code to perform or facilitate the various processes, layers, and modules described in this disclosure. The memory may be or include volatile or non-volatile memory, and may include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described herein. According to an exemplary embodiment, the memory is communicatively connected to a processor via processing circuitry and includes computer code for performing (e.g., performed by the processing circuitry or the processor) one or more of the processes described herein.

[0082] This disclosure contemplates methods, systems, and program products on any machine-readable medium for performing various operations. Embodiments of this disclosure can be implemented using existing computer processors, or by a dedicated computer processor incorporated for a suitable system for this or another purpose, or by a hardwired system. Embodiments within the scope of this disclosure include program products comprising machine-readable media for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media can be any available medium accessible by a general-purpose computer, a special-purpose computer, or other machine having a processor. By way of example, such machine-readable media can include RAM, ROM, EPROM, EEPROM, or other optical disk storage devices, magnetic disk storage devices, or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of machine-executable instructions or data structures and accessible by a general-purpose computer, a special-purpose computer, or other machine having a processor. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions include, for example, instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing machine to perform a particular function or group of functions.

[0083] Although the accompanying drawings and descriptions may illustrate a specific order of method steps, such order may differ from the order depicted and described, unless otherwise specified above. Two or more steps may also be performed simultaneously or partially simultaneously, unless otherwise specified above. Such variations may depend, for example, on the chosen software and hardware system and the designer's choices. All such variations are within the scope of this disclosure. Similarly, the software implementation of the described method can be accomplished using standard programming techniques with rule-based logic and other logic to perform various connection steps, processing steps, comparison steps, and decision steps.

[0084] It is important to note that the construction and arrangement of the various systems and methods shown in the various exemplary embodiments are merely illustrative. Furthermore, any element disclosed in one embodiment may be combined with or used in any other embodiment disclosed herein. Although only one example of an element from one embodiment that may be combined with or used in another embodiment has been described above, it should be recognized that other elements of the various embodiments may be combined with or used in any of the other embodiments disclosed herein.

Claims

1. A method executable by one or more processors, comprising: By using a reinforcement learning model, control decisions for subsequent time steps are generated based on the measured or estimated values ​​of variables for the current time step. Using the model, the predicted value of the variable for the subsequent time steps is predicted based on the measured or estimated value of the variable for the current time step; The control decision for the subsequent time step is adjusted based on the constraints and the predicted values ​​of the variables. as well as Based on the control decision, the actuator is controlled.

2. The method according to claim 1, wherein, The model is a data-driven model generated using system identifiers.

3. The method of claim 1, further comprising using a digital twin to estimate the estimated value.

4. The method of claim 1, further comprising using a reduced-order model to estimate the estimated value.

5. The method of claim 1, further comprising training the reinforcement learning model to optimize a reward function including the variable and additional variables, wherein, The second variable is a function of the first variable; The generation of the control decision through the reinforcement learning model includes predicting, through the reinforcement learning model, that the control decision will produce the optimal value of the reward function.

6. The method according to claim 1, wherein, The constraints include limitations on the actuator, and adjusting the control decision includes making the control decision conform to the time steps of the actuator's limitations.

7. The method according to claim 1, wherein, The constraints include time steps that limit the physical conditions affected by the operation of the actuator.

8. The method according to claim 7, wherein, Violation of the aforementioned restrictions results in physical damage.

9. The method of claim 1, further comprising dynamically determining the constraints based on data from the artificial intelligence agent.

10. A system comprising: Multiple interconnected wells; Multiple artificial intelligence agents associated with the multiple interconnected wells; At least one of the plurality of AI agents is configured to control at least one actuator for at least one of the plurality of interconnected wells by means of: By using a reinforcement learning model, control decisions for subsequent time steps are generated based on the measured or estimated values ​​of variables for the current time step. Based on the measured or estimated value of the variable for the current time step, predict the predicted value of the variable for the subsequent time steps; The control decision for the subsequent time step is adjusted based on the constraints and the predicted values ​​of the variables. as well as The at least one actuator is controlled according to the control decision.

11. The system of claim 10, further comprising a supervisory AI agent configured to coordinate the operations of the plurality of AI agents by making the constraint a function of both the variable and the additional variable, wherein, The variable is associated with a first well among the plurality of interconnected wells, and the additional variable is associated with a second well among the plurality of interconnected wells.

12. The system according to claim 10, wherein, The predicted value includes using data from at least two of the plurality of interconnected wells, a data-driven model generated using system identifiers, and data from at least two of the plurality of interconnected wells.

13. The system according to claim 10, wherein, At least one of the plurality of AI agents is configured to estimate the estimated value using digital twins and data from at least two of the plurality of interconnected wells.

14. The system according to claim 10, wherein, At least one of the plurality of AI agents is configured to estimate the estimated value using a reduced-order model and data from at least two of the plurality of interconnect wells.

15. The system according to claim 10, wherein, At least one of the plurality of AI agents is configured to control at least one actuator for at least one of the plurality of interconnected wells by further training the reinforcement learning model to optimize a reward function including the variable and additional variables, wherein the variable is associated with the at least one well and the additional variables are associated with additional wells among the plurality of interconnected wells, wherein generating the control decision by the reinforcement learning model includes predicting by the reinforcement learning model that the control decision will produce an optimal value of the reward function.

16. The system according to claim 10, wherein, The constraints include limitations on the physical conditions affected by the operation of the at least one actuator.

17. The system according to claim 16, wherein, The control decision for adjusting the subsequent time steps is also based on additional constraints, wherein the additional constraints represent operational limitations of the at least one actuator.

18. The system according to claim 16, wherein, Violation of the aforementioned restrictions results in physical damage.

19. The system according to claim 10, wherein, At least one of the plurality of AI agents is configured to automatically determine the value of the constraint.

20. A non-transitory computer-readable medium storing one or more program instructions, said program instructions, when executed by one or more processors, causing said one or more processors to perform operations, said operations including providing an artificial intelligence agent, said artificial intelligence agent comprising: The first model is configured to estimate the value of the second variable at the current time step based on the measurement of the first variable; The second model is configured to predict the predicted values ​​of the first variable for subsequent time steps; A reinforcement learning model is configured to output control decisions for the subsequent time steps based on the measured value of the first variable and the estimated value of the second variable; A constraint engine is configured to adjust the control decision based on the predicted value of the first variable to ensure that the system conforms to the constraints at the subsequent time step; as well as The control engine is configured to operate the system using adjusted control decisions.

Images