Method and system for autonomous flow control in hydraulic stimulation operations

Abstract

A method can include determining time derivative pressure data based on pressure data for a pump system (121; 122; 123) performing a hydraulic stimulation operation in a geological region. The method can also include determining a moving average based on the time derivative pressure data and a predetermined time window. The method can also include determining a flow adjustment for the pump system (121; 122; 123) based on the moving average, an update interval for adjusting the flow, and a predetermined flow rule. A size of the predetermined time window can be different than the update interval. The method can also include determining the time derivative pressure data using smoothed pressure data. The method can also include determining the flow adjustment for the pump system (121; 122; 123) based on the time derivative pressure data derived from the smoothed pressure data.

Description

BACKGROUND

[0001] Natural fractures present in a subterranean formation are discontinuities that represent surfaces or regions of mechanical failure in the formation. In particular, fractures can form over geologic time as a result of movement and deformation within the subterranean rock and continue to form as a result of microseismic events. Natural fractures can be opened and altered to increase their permeability to hydrocarbon deposits in the subterranean formation by using various types of fluids. SUMMARY

[0002] This summary is provided to introduce a selection of concepts that are further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in limiting the scope of the claimed subject matter.

[0003] Generally, in one aspect, embodiments relate to a method that includes determining, by a computer processor, time derivative pressure data based on pressure data related to a pump system performing a hydraulic stimulation operation in a geologic region. The method also includes determining, by the computer processor, a moving average based on the time derivative pressure data and a predetermined time window. The method also includes determining, by the computer processor, a flow rate adjustment for the pump system based on the moving average, an update interval for adjusting the flow rate, and a predetermined flow rate rule. The predetermined time window has a different size than the update interval. The method also includes sending, by the computer processor and based on the flow rate adjustment, a command to change a flow rate within the hydraulic stimulation operation to the pump system.

[0004] Generally, in one aspect, embodiments relate to a method that includes determining, by a computer processor, smoothed pressure data based on pressure data related to a pump system performing a hydraulic stimulation operation in a geologic region. The method also includes determining, by the computer processor, time derivative pressure data based on the smoothed pressure data. The method also includes determining, by the computer processor, a flow rate adjustment for the pump system based on the smoothed pressure data, an update interval for adjusting the flow rate, and a predetermined flow rate rule. The method also includes sending, by the computer processor and based on the flow rate adjustment, a command to change a flow rate within the hydraulic stimulation operation to the pump system.

[0005] Generally, in one aspect, embodiments relate to a method comprising obtaining, by a computer processor, first observation data regarding a hydraulic stimulation environment. The first observation data includes pressure data and flow data regarding a hydraulic stimulation operation. The method further includes determining, by the computer processor and based on an agent policy and the first observation data, an action for a pump agent. The pump agent corresponds to a pump system, a fluid control system, and a proppant system connected to a wellbore. The action corresponds to a flow adjustment for the pump system. The method further includes obtaining, by the computer processor, second observation data regarding the hydraulic stimulation environment in response to the pump agent performing the action. The method further includes determining, by the computer processor, a reward value for the pump agent based on the second observation data and a reward function. The method further includes updating, by the computer processor, the agent policy based on the reward value to produce an updated policy. The updated policy determines an operation for the pump agent.

[0006] Generally, in one aspect, embodiments relate to a system comprising a pump system including a positive displacement pump. The system further includes various sensors coupled to the pump system. The sensors determine pressure data regarding a hydraulic stimulation operation. The system further includes a hydraulic stimulation manager including a computer processor. The hydraulic stimulation manager is coupled to the pump system. The hydraulic stimulation manager determines time derivative pressure data based on the pressure data regarding the pump system performing the hydraulic stimulation operation in a geological region. The hydraulic stimulation manager determines a moving average based on the time derivative pressure data and a predetermined time window. The hydraulic stimulation manager determines a flow adjustment for the pump system based on the moving average, an update interval for adjusting flow, and a predetermined flow rule. A size of the predetermined time window is different than the update interval. The hydraulic stimulation manager sends a command to the pump system to change flow within the hydraulic stimulation operation based on the flow adjustment.

[0007] Generally, in one aspect, embodiments relate to a system comprising a pump system including a positive displacement pump. The system further includes various sensors coupled to the pump system. The sensors determine pressure data regarding a hydraulic stimulation operation. The system further includes a hydraulic stimulation manager including a computer processor. The hydraulic stimulation manager is coupled to the pump system. The hydraulic stimulation manager determines, by the computer processor, smoothed pressure data based on the pressure data regarding the pump system. The hydraulic stimulation manager further determines time derivative pressure data based on the smoothed pressure data. The hydraulic stimulation manager determines a flow adjustment for the pump system based on the smoothed pressure data, an update interval for adjusting flow, and a predetermined flow rule. The hydraulic stimulation manager further sends a command to the pump system to change flow within the hydraulic stimulation operation based on the flow adjustment.

[0008] Generally, in one aspect, embodiments are directed to a system including various pump agents coupled to various positive displacement pumps. The system also includes various sensors coupled to the pump agents. The pump agents correspond to pump systems, fluid control systems, and / or proppant systems. The sensors determine pressure data regarding the hydraulic stimulation operation. The system also includes a hydraulic stimulation manager including a computer processor. The hydraulic stimulation manager is coupled to the pump agents. The hydraulic stimulation manager obtains first observation data regarding the hydraulic stimulation environment. The first observation data includes the pressure data and flow data regarding the hydraulic stimulation operation. The hydraulic stimulation manager also determines an action for at least one of the pump agents based on one or more agent policies of the pump agents and the first observation data. In some embodiments, the action corresponds to a flow adjustment for the pump system. The hydraulic stimulation manager also obtains second observation data regarding the hydraulic stimulation environment in response to performance of the one or more actions. The hydraulic stimulation manager determines a reward value for the respective pump agents based on the second observation data and a reward function. The hydraulic stimulation manager updates the one or more agent policies of the pump agents based on the reward values to produce one or more updated policies. The updated policies determine one or more actions for the one or more pump agents.

[0009] In some embodiments, the pressure data is obtained from various downhole sensors including a first downhole sensor coupled to a casing of a wellbore and a second downhole sensor coupled to a frac plug / ball within the wellbore, and the downhole sensors stream the pressure data in real-time through a well network to the hydraulic stimulation manager disposed at the surface of a wellsite.

[0010] In some embodiments, the one or more predetermined flow rules are determined by a reinforcement learning system including an action selector engine and a training system. The one or more predetermined flow rules can correspond to one or more agent policies determined by a reinforcement learning algorithm, and the one or more predetermined flow rules can define a size of a dynamic time window, a size of a flow adjustment increment, and various conditions for increasing, decreasing, and maintaining a flow of the pump system.

[0011] In some embodiments, the one or more predetermined flow rules include a first rule, a second rule, and a third rule. The first rule can correspond to no flow adjustment when a moving average is positive. The second rule can correspond to a positive flow increment when the moving average is negative or equal to zero. The third rule can correspond to a negative flow increment when a pressure value at a current time step is equal to or greater than a predetermined maximum pressure value.

[0012] In some embodiments, the flow data regarding the pump system is obtained from a flow sensor. The one or more flow rules can be based on the flow data from a previous time step.

[0013] In some embodiments, various recommended flow adjustments for the pump system are determined based on the moving average and one or more predetermined flow rules; the recommended flow adjustments are presented within a graphical user interface of the user device. In response to a selection of a recommended flow adjustment among the recommended flow adjustments, a command can be sent to the pump system to implement the recommended flow adjustment.

[0014] In some embodiments, a pressure curve is determined from the pressure data. A smoothing operation can be performed on the pressure curve using a discrete Fourier transform with a low pass filter, or alternatively using a median filter or any other smoothing method, to produce a smoothed pressure curve. The time derivative pressure data can be determined using the smoothed pressure curve.

[0015] In some embodiments, the pump system sends hydraulic fracturing fluid into the wellbore at a predetermined flow rate during a hydraulic stimulation operation. The hydraulic fracturing fluid can include at least one proppant and laterally creates a network of fractures from the wellbore.

[0016] In some embodiments, a training system obtains training data about a pump agent. The training system can determine a mismatch between the training data and observed data about one or more hydraulic stimulation operations based on a loss function. Various policy parameters of an updated policy can be adjusted based on the mismatch.

[0017] In some embodiments, a replay buffer obtains various pump agent trajectories about various pump agents. An updated reward function associated with at least one pump agent can be updated based on the pump agent trajectories. Various agent policies associated with the pump agents can be updated using the updated reward function.

[0018] In some embodiments, an agent policy is updated using training data corresponding to training pressure data and training flow data. In some embodiments, the observed data includes acoustic sensor data obtained from a distributed acoustic sensing (DAS) system disposed in the wellbore.

[0019] In some embodiments, a system determines various recommended flow adjustments for the pump system based on time derivative data or a moving average and one or more predetermined flow rules, the time derivative data based on smoothed pressure data. The system can present the recommended flow adjustments within a graphical user interface in a user device. The system can send a command to the pump system to implement a selection of a recommended flow adjustment among the recommended flow adjustments in response to the selection.

[0020] Other aspects and advantages of the claimed subject matter will be apparent from the following description and accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0021] Specific embodiments of the disclosed technology will now be described in detail with reference to the figures. Like

[0022] Figure 1 , Figure 2 and Figure 3 illustrate a system in accordance with one or more embodiments.

[0023] Figure 4 illustrate a flowchart in accordance with one or more embodiments.

[0024] Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 6F , Figure 6G , Figure 6H , Figure 61 , Figure 6J , Figure 6K , Figure 6L , Figure 6M , Figure 6N , Figure 60 , Figure 6P , Figure 6Q , Figure 6R , Figure 6S and Figure 6T illustrate examples in accordance with one or more embodiments.

[0025] Figure 7 illustrate a flowchart in accordance with one or more embodiments.

[0026] Figure 8A , Figure 8B , Figure 8C , Figure 8D and Figure 8E illustrate examples in accordance with one or more embodiments.

[0027] Figure 9 illustrate a system in accordance with one or more embodiments.

[0028] Figure 10 illustrate a flowchart in accordance with one or more embodiments.

[0029] Figure 11 illustrate a computer system in accordance with one or more embodiments. DETAILED DESCRIPTION

[0030] In the following detailed description of embodiments of the application, numerous specific details are set forth in order to provide a more thorough understanding of the application. However, it will be apparent to one of ordinary skill in the art that the present disclosure can be practiced without these specific details. In other instances, well-known features have not been described in detail to avoid unnecessarily complicating the description.

[0031] Throughout this application, ordinal numbers (e.g., first, second, third, etc.) can be used as adjectives to describe an element. The use of ordinal numbers does not imply or create any particular order for the elements, nor does it limit any element to only a single element, unless specifically disclosed, e.g., using the terms “before,” “after,” “single,” and other such terms. Rather, the use of ordinal numbers is to differentiate between elements. As an example, a first element is distinct from a second element, and a first element can include more than one element and be after (or before) a second element in an ordering of elements.

[0032] Generally, embodiments of the present disclosure include systems and methods for autonomous flow control to manage stage-to-stage execution of a hydraulic stimulation operation. In some embodiments, for example, one or more flow adjustments for a pump system can be determined based on predetermined flow rules and a moving average of time-derivative pressure data. More specifically, the flow rules can determine when to increase, decrease, or hold flow during the hydraulic stimulation operation using various stimulation parameter inputs (e.g., flow delta value, update interval, target flow, moving average, etc.). As such, the moving average can be determined by a time window, which can be specified by a user or automatically determined by a hydraulic stimulation manager, e.g., based on a machine learning algorithm. The size of the time window can control how the flow adjustments respond to changes in the time-derivative pressure data and transient noise within the pressure data.

[0033] Further, the flow rules can be determined through various reinforcement learning techniques. For example, a pump agent can operate on an agent policy corresponding to one or more flow rules. Through interaction with the hydraulic stimulation environment, different actions of the pump agent can be rewarded using a reward function. As such, the agent policy or flow rules can be optimized based on observations of the pump agent’s interaction with the hydraulic stimulation environment. In some embodiments, the agent policy or reward function can be optimized through a machine learning training process. For example, the hydraulic stimulation manager can include a reinforcement learning system for implementing a reinforcement learning algorithm and a training process for components within a well pattern (e.g., pump systems).

[0034] Therefore, some embodiments can use flow rules to increase the uniformity of fluid and proppant distribution across different perforation clusters within a specific hydraulic fracturing stage. Thus, autonomous pump flow control can provide one or more of the following: (i) more uniform distribution of proppant and fluid across individual perforation clusters; (ii) consistent performance between production enhancement stages within a hydraulic enhancement operation and between hydraulic enhancement operations (e.g., at different wells); (iii) lower mean treatment pressure for initiating and extending hydraulic fracturing; and (iv) reduced or eliminated screening and inter-well communication from hydraulic enhancement operations. Compared to manual flow control by operators, autonomous flow control prevents large inconsistencies in stage-to-stage execution that lead to significant uneven distribution of fluid and proppant across different perforation clusters, ultimately resulting in poor hydraulic fracturing performance.

[0035] Turn Figure 1 , Figure 1 A schematic diagram according to one or more embodiments is shown. Figure 1 As shown, a well pattern (e.g., well pattern Z 100) may include a completion assembly (e.g., completion assembly 190), one or more proppant systems (e.g., proppant system 192), one or more fluid mixing systems (e.g., fluid mixing system 194), and one or more pump systems (e.g., pump system A 121, pump system B 122, pump system N 123). For example, the completion assembly may include one or more expandable packers and a working string or casing string extending within the wellbore. The casing string may include steel casing or tubing, which may be divided into surface casing, intermediate casing, and / or production casing. The packer may include an expandable packer that seals an annular space defined between the completion equipment and the wellbore wall to divide the formation into multiple wellbore segments. During hydraulic enhancement operations, these wellbore segments may be enhanced individually or simultaneously. Similarly, the completion assembly may include perforating equipment (e.g., a perforating gun) for creating holes in different wellbore segments. For example, after receiving hydraulic enhancement fluid in a flow manifold (e.g., flow manifold Y 198), the hydraulic enhancement fluid can be transported down into the wellbore (e.g., to wellbore Z 199), for example, to a specific section for the hydraulic enhancement operation. During the hydraulic enhancement operation, the enhancement fluid can be transported to different wellbore sections, for example, in open-hole completion operations or casing completion operations (see below). Figure 3 (A more detailed description) Initiation and propagation of cracks.

[0036] With respect to the proppant system, the proppant system can include transfer devices, such as chutes and conveyors, for transferring proppant (also referred to simply as “proppant”) to the fluid mixing system. Likewise, the proppant system can include one or more proppant storage devices (e.g., proppant storage device A 193), such as silos and housings. In particular, silos can use a fill port to acquire proppant, which can then be transferred to the fluid mixing system using a discharge valve and / or outlet. The proppant system can then distribute the proppant to the fluid mixing system for use in generating a stimulation fluid. With respect to the fluid mixing system, the fluid mixing system can include hardware and / or software for combining proppant with one or more liquid additives and other fluids (e.g., water). More specifically, the fluid mixing system can include a blender and / or various liquid storage devices, which are added to one or more proppants to generate a particular stimulation fluid. Thus, the fluid mixing system can generate and / or distribute hydraulic stimulation fluids having various predetermined fluid properties (e.g., having a particular viscosity, density, or one or more rheological properties).

[0037] In some embodiments, for example, the hydraulic stimulation fluid can be a fracturing fluid that is injected into open fractures connected to a wellbore. Such injection can create and / or alter a fracture network in the subsurface. After a hydraulic fracturing operation has completed a desired fracture network, the liquid portion of the fracturing fluid can be removed from the wellbore. For more information on hydraulic stimulation fluids and proppants, see Figure 2 and the corresponding description below.

[0038] With respect to the pump system, the pump system can include hardware and software having functionality for supplying hydraulic stimulation fluid to a wellbore at one or more predetermined pressures and / or at one or more predetermined flow rates. For example, the pump system can include one or more positive displacement pumps that inject hydraulic stimulation fluid into a wellbore. Likewise, the pump system can include a pump controller that includes hardware and / or software for adjusting local flow rates and pump pressures, for example, in response to commands from a hydraulic stimulation manager. For example, the pump system can include one or more communication interfaces (e.g., communication interface A 141, communication interface B 142, communication interface N 143) and / or memories (e.g., memory A 151, memory B 152, memory N 153) for sending and / or obtaining data through a well network. The pump system can also obtain and / or store sensor data regarding one or more pump operations from one or more sensors (e.g., sensor A 161, sensor B 162, sensor N 163) coupled to a wellbore. For example, pressure data regarding a hydraulic stimulation operation can be acquired by the pump system from a wellhead sensor or a downhole sensor disposed in a wellbore. While the pump system can correspond to a single pump, in some embodiments, the pump system can correspond to multiple pumps.

[0039] Turning to Figure 2 ,Figure 2 A schematic diagram is shown in accordance with one or more embodiments. As Figure 2 shown, Figure 2 A hydraulic stimulation operation is shown that forms additional microfractures 212 within the formation 202. More specifically, a wellbore 204 can be located within the formation 202, with a casing string 206 positioned within the wellbore 204. Following a hydraulic fracturing process, for example, a large fracture 210 can exist within the formation 202 and extend outward from the wellbore 204. In particular, hydrocarbon reserves can be trapped within certain low-permeability formations, such as sand formations, carbonate formations, and / or shale formations. Thus, a stimulation treatment can increase the well production rate of one or more wells, with one type of stimulation treatment being hydraulic fracturing. In some embodiments, for example, hydraulic fracturing includes injecting a high-viscosity fluid into a wellbore at a sufficiently high injection rate such that sufficient pressure is generated in the wellbore to split the formation. Thus, a stimulation operation can be determined that achieves a desired height and / or length of one or more induced fractures.

[0040] Continuing with the description Figure 2 Various stimulation processes can be employed that use one or more techniques to ensure that the induced fractures become conductive after injection is stopped. For example, during acid fracturing of carbonate formations, an acid-based fluid can be injected into the formation to create etched fractures and conductive pathways. These conductive pathways can remain open when the induced fractures close. For sand or shale formations, proppants can be included in the hydraulic fracturing fluid such that the induced fractures remain propped open during or after the stimulation treatment. Likewise, in carbonate formations, the stimulation treatment can include both an acid fracturing fluid and proppants. Thus, heat generated within the formation, acid or solvent water sent into the formation, can all play a role in creating a reaction that results in one or more microfractures in the formation.

[0041] Continuing with the description of hydraulic fracturing, the hydraulic fracturing fluid can be pumped through the casing string 206 and into the target formation using various perforations in the casing string 206 (i.e., open holes). By injecting the hydraulic fracturing fluid at a sufficiently high pressure to break the rock within the target formation, the hydraulic fracturing operation can“break” the formation. As the high-pressure fluid injection continues, the fractures can continue to expand into a fracture network. This high pressure for injecting the hydraulic fracturing fluid can be referred to as“propagation pressure” or“extension pressure.” As the induced fractures continue to grow, proppants, such as sand, can be added to the fracturing fluid. Once the desired fracture network is formed, the fluid flow can be reversed, and the liquid portion of the fracturing fluid is removed. The proppants are intentionally left behind to prevent the fractures from closing themselves due to weight and stress within the formation. Thus, the proppants can“prop” or support the induced fractures by keeping a sufficient permeability to keep the induced fractures propped open for the flow of hydrocarbon fluids through the induced fractures. Thus, the proppants can form a granular packed bed within the formation with interstitial space connectivity. Thus, a higher permeability fracture can be created by the hydraulic fracturing operation.

[0042] In some embodiments, for example, a hydraulic fracturing fluid with an activator is injected into the formation 202, where the fluid migrates within the large fracture 210. Upon reaction induced by the activator, the injected fluid can generate one or more gases and heat, thereby causing the creation of microfractures 212 within the formation 202. Thus, the stimulation treatment can provide a pathway for migration and recovery of hydrocarbon deposits trapped within the formation 202 by a production well.

[0043] Further, fracture monitoring can be important for understanding and optimizing hydraulic fracturing treatments. For example, a hydraulic stimulation manager can perform a diagnostic that determines various stimulation effects, such as fracture geometry, proppant placement in one or more fractures, and / or fracture conductivity. Fracture monitoring can be performed using a distributed acoustic sensing (DAS) system implemented in the wellbore. In some embodiments, the DAS system includes various fiber optic sensors (e.g., distributed over a single mode optical fiber that is several kilometers long). Thus, backscattered light can be measured, and further analyzed using signal processing techniques to enable the DAS system to separate the optical fiber into an array of individual acoustic receivers. More specifically, various light pulses can be sent along the optical fiber, where characteristics of the backscattered light can change due to acoustic vibrations that interfere with the outer shell of the optical fiber. Through DAS processing, the locations of these perturbations can be identified.

[0044] Continuing the description of the DAS system, pumping operations can generate various acoustic signals along the wellbore and adjacent fractures, where acoustic sensing data depends on the geometry and physical properties of the extended fractures. Thus, quantitative DAS inversion can determine various fracture properties in hydraulic fracture monitoring. For example, the wellbore can be mapped in real-time by removing DAS pump noise data and matching the acquired data to a forward model regarding the propagation of pulses in the wellbore and adjacent fractures. Thus, the DAS inversion can identify various hydraulic stimulation features, such as tubing expansion, fluid-fluid interfaces, adjacent hydraulic fractures, the presence of a porous reservoir, and / or an annular compartment. During the initial stages of a hydraulic stimulation operation, the DAS inversion can determine location information for wireline logging equipment in the wellbore. For example, the DAS technology can verify whether perforating guns and packer setting devices are disposed at desired depths in the wellbore. In some embodiments, the DAS inversion is performed using additional data from a distributed temperature sensor (DTS) and / or microseismic monitoring techniques.

[0045] For example, in certain unconventional formations, a key factor in determining the economic viability of reservoir development is the presence of one or more sweet spots. Sweet spots are generally defined in this paper as areas within the reservoir representing optimal production or production potential. In specific geological regions, sweet spots can be identified based on a lack of ductility, disruption of internal cohesion, the rock's ability to deform and fracture with low-level inelastic behavior, and the rock's capacity for self-sustaining fracturing. In other words, hydraulic enhancement operations can be applied to fracturing formations to generate more microfractures, while exhibiting minimal plastic deformation under compression.

[0046] Turning Figure 3 , Figure 3 A system according to one or more embodiments is shown. Figure 3 As shown, the drilling system 300 may include a top drive rig 310 arranged around a drill bit logging tool 320. The top drive rig 310 may include a top drive 311, which is suspended in a derrick 312 via a traveling block 313. At the center of the top drive 311, a drive shaft 314 may be threaded to the top tube of the drill string 315, for example. The top drive 311 allows the drive shaft 314 to rotate, causing the drill string 315 and the drill bit logging tool 320 to cut rock at the bottom of the wellbore 316. A power cable 317 supplying power to the top drive 311 may be protected within one or more maintenance circuits 318 coupled to a control system 344. Drilling mud can then be pumped into the wellbore 316 via mud lines, the drive shaft 314, and / or the drill string 315.

[0047] The control system 344 may include one or more programmable logic controllers (PLCs) comprising hardware and / or software having the function of controlling one or more processes performed by the drilling system 300, hydraulic enhancement operations, and / or completion components. Specifically, the PLC may control valve states, fluid levels, pipeline pressures, warning alarms, and / or pressure releases across the entire well network (e.g., a network of wells connected to a drilling rig). In particular, the PLC may be a ruggedized computer system capable of withstanding, for example, vibrations around the wellbore, extreme temperatures, humid conditions, and / or dusty conditions. Without loss of generality, the term "control system" may refer to an operating control system for operating and controlling equipment, or a data acquisition and monitoring system for acquiring equipment data and monitoring the operations of drilling, hydraulic enhancement, or completion processes. The control system may also include interpretive software systems for analyzing and understanding drilling events, completion events, and hydraulic enhancement events and their respective processes.

[0048] When completing the well, a casing (e.g., casing string 206) can be inserted into the wellbore 316. The sides of the wellbore 316 can need support, so the casing can be used to support the sides of the wellbore 316. As such, the space between the casing and the untreated sides of the wellbore 316 can be cemented to hold the casing in place. Cement can be forced through the lower end of the casing and into the annular space between the casing and the wall of the wellbore 316. More specifically, a cementing plug can be used to push the cement out of the casing. For example, the cementing plug can be a rubber plug used to separate the cement slurry from other fluids, reducing contamination and maintaining predictable cement slurry performance. The well operation can include pumping a cement slurry into the wellbore 316 to displace the existing drilling fluid and fill the space between the casing and the untreated sides of the wellbore 316. The cement slurry can include a mixture of various additives and cement. After the cement slurry hardens, the cement can seal the wellbore 316 from non-hydrocarbons that attempt to enter the well stream. In some embodiments, the cement slurry is forced through the lower end of the casing and into the annular space between the casing and the wall of the wellbore 316. More specifically, a cementing plug can be used to push the cement slurry out of the casing. For example, the cementing plug can be a rubber plug used to separate the cement slurry from other fluids, reducing contamination and maintaining predictable cement slurry performance. A displacing fluid, such as water or appropriately weighted drilling fluid, can be pumped into the casing above the cementing plug. The displacing fluid can be a pressurized fluid that is used to push the cementing plug down through the casing to force the cement out of the casing outlet and back up into the annular space.

[0049] The completion operation can include a running of casing operation, a cementing operation, perforating the well, gravel packing, directional drilling, hydraulic stimulation of a reservoir zone, and / or installing a Christmas tree or wellhead assembly 316 at the wellbore. Also, the well operation can include open hole completion or cased hole completion. In open hole completion, an uncased wellbore can use one or more inflatable packers (or other packer types) to separate different fracturing stages or compartments. Within a given fracturing stage or compartment, the uncased wellbore can have multiple fracturing sleeves or ports where injected fluid can exit the wellbore and communicate with the formation. In cased hole completion (i.e., plug-and-perforation completion), a cased wellbore and plugs can be disposed to isolate previous fracturing stages, followed by forming multiple perforation clusters for a current fracturing stage (see, e.g., Figure 5 ) Also, open hole completion can refer to a well drilled to the top of a hydrocarbon reservoir. Thus, the well is cased at the top of the reservoir, while open at the bottom of the wellbore. Conversely, cased hole completion can include running a casing into the reservoir zone.

[0050] Some embodiments can include a perforation operation. More specifically, the perforation operation can include perforating the casing and cement at different locations in the wellbore 316 to enable hydrocarbons to enter the well stream from the created perforations. For example, some perforation operations include using a perforating gun at different production zones to create open segments of perforations through the casing, cement, and the sides of the wellbore 316. Hydrocarbons can then enter the well stream through these open segments. In some embodiments, the perforation operation is performed using a jet stream or a shaped charge to penetrate the casing around the wellbore 316.

[0051] As Figure 3 Further shown, the sensors 3321 can be included in a sensor assembly 323 that is positioned, for example, adjacent to the drill bit 324 in the wellbore 316 and coupled to the drill string 315. The sensors 321 can also be coupled to a processor assembly 323 that includes a processor, memory, and an analog-to-digital converter 322 for processing the sensor measurements. For example, the sensors 321 can include acoustic sensors such as accelerometers, measurement microphones, contact microphones, and hydrophones. Likewise, the sensors 321 can include other types of sensors such as transmitters and receivers that measure resistivity, gamma ray detectors, and the like. The sensors 321 can include hardware and / or software for generating different types of logging records such as acoustic logging records or density logging records that can provide well data about the wellbore including porosity, gas saturation, formation boundaries in the geological formation, fractures in the wellbore or completion cement, and many other information about the formation. If such logging data is acquired during a drilling operation (e.g., logging while drilling or measurement while drilling), the information can be used to adjust drilling parameters in real-time. Such adjustments can include rate of penetration (ROP), bottom hole circulating pressure, one or more drilling directions, changing mud weight, amount of weight on bit, and many other drilling parameters.

[0052] Returning Figure 1 In some embodiments, the well pattern includes a hydraulic fracturing manager (e.g., hydraulic fracturing manager X 180) that includes hardware and / or software for managing one or more hydraulic fracturing operations. For example, the hydraulic fracturing manager can be a controller that sends commands (e.g., command X 171) to one or more components (e.g., proppant system 192, fluid mixing system 194, pump system A 121, pump system B 122, pump system N 123) within the well pattern. In particular, the commands can be control signals that include functions to initiate and / or terminate one or more operations of the well pattern. Similarly, the commands can also adjust one or more parameters about the proppant system, the fluid mixing system, and / or the pump system, for example, with respect to the hydraulic fracturing fluid.

[0053] Additionally, the hydraulic stimulation manager can manage the production and / or distribution of hydraulic stimulation fluids (e.g., hydraulic stimulation fluid A 195, hydraulic stimulation fluid B 196, hydraulic stimulation fluid N 197) based on various stimulation parameters (e.g., stimulation parameters A 178). For example, the stimulation parameters can include different types and / or properties of hydraulic stimulation fluids as well as various flow parameters, such as a target flow rate, a flow rate increment for flow rate adjustments, a maximum allowable pressure, a start time for a particular stimulation stage, etc.

[0054] In some embodiments, the hydraulic stimulation manager includes functionality for using collected data to adjust one or more flow rates and / or one or more pressures within a hydraulic stimulation operation. For example, the hydraulic stimulation manager can acquire pressure data (e.g., acquired pressure data Y 172) from wellhead pressure sensors, downhole pressure sensors throughout the wellbore, and local sensors coupled to one or more pumps in the pump system. Likewise, the hydraulic stimulation manager can acquire flow rate data (e.g., acquired flow rate data Z 173) from high pressure flow meters based on Coriolis mass flow technology, volumetric sensors, turbines, ultrasonic sensors, and / or volumetric measurements. During a hydraulic stimulation operation, for example, the pumped flow rate of an injection fluid can be ramped from zero to a predetermined target flow rate. Other types of data can also be collected, such as completion data (e.g., completion 191), logging data, seismic data, and / or geological data that can be used to determine stimulation parameters for the hydraulic stimulation operation.

[0055] In some embodiments, the hydraulic stimulation manager implements automatic flow rate control using one or more flow rate rules (e.g., flow rate rules 179), for example. For instance, a particular flow rate rule can determine a flow rate adjustment during a hydraulic stimulation operation. To illustrate, a flow rate rule can specify a particular increment to increase or decrease a flow rate based on various parameter values (e.g., pressure values, time derivative pressure values, moving average values, target flow rates, current flow rate values, etc.). In particular, the flow rate rules can be user-defined rules, such as specified by user selections within a graphical user interface provided by the hydraulic stimulation manager. In some embodiments, the hydraulic stimulation manager, proppant system, and / or fluid control system include one or more components of a computer system similar to the computer system 1102 described in the Figure 11 and the corresponding description.

[0056] In some embodiments, one or more machine learning models (e.g., machine learning model N 176) and one or more machine learning algorithms (e.g., machine learning algorithm O 177) are used to determine the flow rules. For example, the hydraulic stimulation manager can store and / or analyze pressure data (e.g., acquired pressure data Y 172, historical pressure data M 175), flow data (e.g., acquired flow data Z 173), geologic data (e.g., geologic model A 185), well logging data, seismic data, and other sensed data (e.g., DAS data) to generate and / or update one or more machine learning models. Thus, different types of machine learning models can be trained, such as convolutional neural networks, deep neural networks, recurrent neural networks, support vector machines, decision trees, inductive learning models, deductive learning models, supervised learning models, unsupervised learning models, reinforcement learning models, and the like. In some embodiments, two or more different types of machine learning models are integrated into a single machine learning architecture, e.g., a machine learning model can include a reinforcement learning model and a neural network. In some embodiments, the hydraulic stimulation manager can generate augmented or synthetic data to produce large amounts of interpreted data for training a particular model.

[0057] With respect to neural networks, for example, a neural network can include one or more hidden layers, where a hidden layer includes one or more neurons. A neuron can be a modeled node or object that generally mimics a neuron of the human brain. In particular, a neuron can combine data inputs with a set of weights, i.e., a set of network weights and biases that are used to adjust the data inputs. These network weights and biases can amplify or reduce the value of a particular data input, thereby assigning an amount of importance to various data inputs for the task being modeled. Through machine learning, a neural network can determine which data inputs should receive a higher priority in determining one or more specified outputs of the neural network. Likewise, these weighted data inputs can be summed such that the sum is passed through an activation function of the neuron to other hidden layers within the neural network. Thus, the activation function can determine whether and to what extent the output of a neuron progresses to other neurons, where the output can again be weighted for use as input to the next hidden layer.

[0058] Turning to reinforcement learning, the hydraulic stimulation manager can use a reinforcement learning system (e.g., reinforcement learning system Y 110) to execute one or more reinforcement learning algorithms. In particular, a reinforcement learning algorithm can be a method of autonomously learning an agent policy through multiple iterations of trial and error based on observed data. The goal of a reinforcement learning algorithm can be to learn an agent policy p that maps states of an environment to actions so as to maximize an expected reward J(p). A value reward can describe one or more qualities of a particular state, agent action, and / or trajectory at a particular time within an operation such as a hydraulic stimulation operation. Accordingly, a reinforcement learning system can include hardware and / or software having functionality to implement one or more reinforcement learning algorithms. For example, a reinforcement learning system can include an action selector engine (e.g., action selector engine B 112) to determine commands and / or pump actions based on policy data (e.g., policy data A 111) and one or more reward functions (e.g., reward function C 113). More specifically, a reinforcement learning algorithm can train a policy to make a series of decisions based on observed states of an environment so as to maximize a cumulative reward determined by a reward function. For example, a reinforcement learning algorithm can employ a trial-and-error process to determine one or more agent policies based on interactions of various agents with a complex environment. Accordingly, a reinforcement learning algorithm can include a reward function that teaches a particular action selection engine to follow certain rules (e.g., flow rules 179) while still allowing the reinforcement learning model to retain information learned from pressure data, flow data, acoustic sensing data, and / or geologic data.

[0059] In some embodiments, one or more components in a reinforcement learning system are trained using a training system (e.g., training system D 114). For example, an agent policy and / or a reward function can be updated through a training process performed by a machine learning algorithm. In some embodiments, historical data (e.g., historical stimulation data M 174), augmented data, and / or synthetic data can provide supervised signals for training an action selector engine, such as through an imitation learning algorithm. In another embodiment, an interactive expert can provide data for tuning an agent policy and / or a reward function. For more information on reinforcement learning models and algorithms, see Figure 9 and Figure 10 and the corresponding descriptions below.

[0060] While Figure 1 , Figure 2 and Figure 3 illustrate various configurations of components, other configurations can be used without departing from the scope of the disclosure. For example, Figure 1 , Figure 2 and Figure 3Various components in can be combined to form a single component. As another example, the functions performed by a single component can be performed by two or more components.

[0061] Turning to Figure 4 , Figure 4 A flow diagram in accordance with one or more embodiments is shown. Specifically, Figure 4 A general method of using flow rules based on pressure data to regulate pump flow is described. Figure 4 One or more blocks in can be performed by one or more components (e.g., hydraulic stimulation manager X 180) as described in Figure 1 , Figure 2 and Figure 3 Although various blocks in are presented and described sequentially, one of ordinary skill in the art will appreciate that some or all of these blocks can be executed in a different order, can be combined or omitted, and some or all of these blocks can be executed in parallel. Further, these blocks can be executed actively or passively. Figure 4

[0062] In block 400, pressure data is obtained regarding hydraulic stimulation operations in a geological region of interest, in accordance with one or more embodiments. The geological region of interest can be a portion of a geological region or volume that includes one or more formations of interest that are desired or selected for further stimulation. For example, in hydraulic fracturing, the geological region can correspond to one or more wellbore intervals following a perforation operation. Further, the pressure data can be based on surface treatment pressure measurements that are streamed in real-time over a well network. Alternatively, the pressure data can be based on downhole treatment pressure measurements that are streamed in real-time over a well network from downhole sensors disposed in wellbores, such as attached to downhole casing strings and / or frac plugs, for example, as described below in Figure 5 Further, the pressure data can be based on a combination of types of pressure sensors and / or surface and downhole treatment pressure measurements. In some embodiments, historical pressure data from previous hydraulic stimulation operations is used.

[0063] Turning to Figure 5 , Figure 5 Various downhole sensors in accordance with one or more embodiments are shown. In Figure 5 ​In this system, well system 500 includes casing 506 having a fracturing plug 504 disposed at the end of a section of the wellbore. As shown, well system 500 includes multiple perforation clusters (e.g., perforation cluster A 501, perforation cluster B 502, perforation cluster N 503), through which fluid (e.g., fluid flow Q510) flows. Specifically, various plug-and-perf operations can be performed at well system 500. For example, a plug-and-perf operation may include setting a fracturing plug at a predetermined depth, followed by perforation of a region within the wellbore. In open-hole completions, a ball can be pumped into the wellbore to move a ball-actuated sleeve or port that isolates a previous fracturing stage and opens multiple fracturing sleeves or ports, at which fracturing production fluid can exit the casing and communicate with the formation. Subsequently, the fracturing plug / ball can be dissolved or drilled out to allow fluid to flow out of the wellbore. After one stage is completed, another fracturing plug is installed, forming a new set of perforations in a plug-perforation completion. Similarly, after one stage is completed in an open-hole completion, another ball is pumped into the wellbore. The fracturing fluid process is repeated, i.e., moving upwards along the wellbore from the end (e.g., the toe in a horizontal well) to the beginning (e.g., the heel). The fracturing plug / ball can operate as a check valve to provide wellbore formation isolation, for example, between formations in a multi-stage production enhancement process. The fracturing plug / ball can isolate lower formations during production enhancement but allows fluid to flow from below once the production enhancement operation is complete. Examples of fracturing plugs / balls include dissolvable fracturing plugs / balls and composite fracturing plugs / balls.

[0064] During production enhancement operations, various downhole sensors (e.g., downhole sensor A 507, downhole sensor B 505) can be used to monitor fluid flow. Specifically, one downhole sensor (e.g., downhole sensor A 507) can be coupled to the casing 506, while another downhole sensor (e.g., downhole sensor B 505) can be coupled to the fracturing plug 504. However, other configurations of downhole sensors are also conceivable within the wellbore or well system.

[0065] return Figure 4 In block 410, one or more preprocessing operations are performed on the pressure data according to one or more embodiments. Specifically, the preprocessing operations may include a smoothing operation performed on the pressure curve of the acquired pressure data. For example, the smoothing operation may use a discrete Fourier transform with a low-pass filter, or alternatively, a median filter or any other smoothing method, to remove pressure transients. After preprocessing, according to Figure 4The smoothed pressure curve can be used by the hydraulic fracturing manager to determine time derivative pressure data. In some embodiments, the flow rate adjustment is based on the time derivative pressure data using the smoothed pressure curve without utilizing a moving average of the time derivative pressure data (e.g., from Figure 4 In other embodiments, the smoothing operation can be applied to the time derivative pressure data determined from the raw pressure data. In such embodiments, the flow rate adjustment is based on the smoothed time derivative pressure data using the raw pressure curve.

[0066] In some embodiments, one or more pre-processing operations are applied to other types of data inputs to determine the flow rate of the pump system. For example, the flow rate rules can use surface acoustic sensing data to determine the flow rate adjustment. The surface acoustic sensing data can be acquired using an active acoustic device attached to the wellhead. The active acoustic device can continuously emit and record reflected acoustic signals to give an indication about the fracture propagation intensity (e.g., a higher signal can be associated with a faster fracture initiation or propagation, while a lower signal is associated with a slower fracture initiation or propagation). In some embodiments, downhole sensor data (e.g., distributed acoustic sensing (DAS) data) can be pre-processed for use by the hydraulic fracturing manager to monitor the hydraulic fracturing operation.

[0067] In block 420, the time derivative pressure data is determined using the pressure data from the hydraulic fracturing operation, according to one or more embodiments. For example, the hydraulic fracturing manager can use the available pressure data or pre-processed pressure data to determine the time derivative pressure data (i.e., dp / dt). As such, the time derivative pressure data can correspond to the pressure curve that is further analyzed by the hydraulic fracturing manager.

[0068] In block 430, a moving average of the time derivative pressure data is determined using a time window that is separate from the update interval, according to one or more embodiments. The moving average can correspond to an average of the time derivative pressure data over a sampling period defined by, for example, the size of the time window. While the moving average can be obtained for the hydraulic fracturing operation, the moving average can only be analyzed at different times specified by the update interval. For example, the update interval can specify a minimum time for determining the flow rate adjustment for one or more pump systems. By using the moving average, the flow rate processing can reduce or eliminate the noise level inherent in the acquired pressure data and corresponding time derivative pressure data. In other words, the moving average can continuously reflect the rate of change in the pressure curve while being less disturbed by local noise in the pressure sensor measurements.

[0069] Furthermore, the time window can be separated from the update interval to provide a flexible architecture for flow manipulation. Instead of evaluating pressure data in one analysis phase and adjusting the flow rate in a subsequent analysis phase, the hydraulic enhancement manager can determine the optimal time window and update interval for adjusting the flow rate. Such flow manipulation techniques can address different levels of noise that may be present in different hydraulic enhancement environments. Thus, one set of time window parameters can be used in noisy operations, while another set of time window parameters works better in different less noisy environments. Similarly, different types of desired fracture networks can also lead to the use of different update intervals and time windows. In some embodiments, the time window size and update interval can be changed during a particular hydraulic enhancement operation, for example, as noise increases or decreases. For illustrative purposes, Figure 6M to Figure 6T The corresponding description below provides an example of time window separation.

[0070] In some embodiments, the hydraulic production enhancement manager may receive user selection of one or more parameters related to a time window, such as specifying a particular window size, a specific starting time step for updating flow based on a moving average, etc. In some embodiments, the optimal window size may be determined automatically, for example, through a reinforcement learning algorithm or another machine learning algorithm.

[0071] In block 440, according to one or more embodiments, one or more traffic conditioning methods are determined based on one or more moving averages and one or more predetermined traffic rules. For example, the traffic rules may include one or more of the following input parameters: initial traffic (R0), traffic increment (R... in ), target traffic (R) f ), maximum permissible ground handling pressure (P) max ), window size (W) s ) and update interval (T) u Some input parameters can remain constant during (or between) hydraulic enhancement operations, while parameters such as window size and flow increment (i.e., W) are also important. s and R in Other input parameters can be changed. For example, the goal of a hydraulic production enhancement operation might be to increase the flow rate from the initial flow rate (e.g., R0 = 5 bbl / min) (R... f =~795l / min) increased to the target flow rate (e.g., R f =30bbl / min)(R f = ~4770 l / min), while keeping the maximum permissible ground treatment pressure below a specific value (e.g., P). max =4000psi)(P max =275.8 bar).

[0072] In some embodiments, the flow regulation is based on smoothed pressure data without using a moving average. For example, the time derivative pressure data can be based on pressure data processed using a smoothing operation. In this way, no time window can be applied to the underlying pressure data and block 440 can be modified accordingly.

[0073] In some embodiments, the one or more flow rules include one or more thresholds for one or more hydraulic stimulation parameters. For example, multiple thresholds (i.e., dp / dt avg，t ) of a moving average can be used to adjust the flow accordingly. Thus, when dp / dt avg，t ≤ threshold 1, then R t := R t-1 + R in . Assuming threshold 2 < threshold 1, if dp / dt avg，t ≤ threshold 2, then R t := R t-1 + x*R in , where x is a specified number (e.g., x = 2).

[0074] In block 450, one or more commands are sent to one or more pump systems based on the one or more flow regulations, according to one or more embodiments. Based on the particular flow regulation, the commands can be formed to correspond to particular stimulation parameter values, e.g., flow, pressure values, update interval values, etc. Thus, the commands can be control signals generated, for example, by a control system, or network messages transmitted through a well network to regulate one or more stimulation parameters. For example, the commands can be transmitted from a hydraulic stimulation manager or control system on the surface of a well site to one or more pump controllers in one or more pump systems. The pump systems can be similar to the pump systems (121, 122, 123) described above in Figure 1 and the corresponding description. In some embodiments, the one or more commands can regulate stimulation parameters of one or more hydraulic stimulation operations, e.g., stimulation parameters for one or more pump systems. For more information on hydraulic stimulation operations, see Figure 1 and Figure 2 and the corresponding description above.

[0075] In block 460, one or more hydraulic stimulation operations are performed based on the one or more commands, according to one or more embodiments. As Figure 4As shown, the process for increasing pumping flow can be adjusted for hydraulic stimulation operations, e.g., a rapid flow ramp can be desired instead of a slow flow ramp. For manual flow adjustments, this can result in large inconsistencies in stage-to-stage execution, which potentially leads to significant uneven distribution of fluid and proppant between different perforation clusters within a given stimulation stage. For hydraulic fracturing, this uneven distribution can result in poor performance. Accordingly, the hydraulic stimulation manager can use flow rules and moving averages to implement autonomous flow control for one or more hydraulic stimulation operations.

[0076] In some embodiments, while Figure 4 the process shows a sequential series of operations, the process can also include one or more iterative operations, such as an update loop similar to the update loops described below in Figure 7 and corresponding descriptions. For example, the time derivative pressure data can be iteratively obtained and analyzed by the hydraulic stimulation manager within an update loop in order to determine whether to make one or more flow adjustments.

[0077] Turning to Figure 6A to Figure 6T , Figure 6A to Figure 6T Examples of flow adjustments in accordance with one or more embodiments are provided. The following embodiments are for illustrative purposes only and are not intended to limit the scope of the disclosed technology. In Figure 6A , Figure 6B , Figure 6C and Figure 6D , a time window with a window size of five time steps (i.e., W s = 5) and a flow increment (R in ) of 5 bbl / min was used to determine the flow. Specifically, Figure 6A shows the obtained pressure data, Figure 6B shows the time derivative pressure data, Figure 6C shows the moving average based on the selected time window, and Figure 6D shows the corresponding flow adjustments for the corresponding hydraulic stimulation operation. Accordingly, Figure 6A to Figure 6D the flow rules in in = 5 bbl / min (795 l / min), R f = 30 bbl / min (4770 l / min), P max = 4000 psi (275.8 bar), and W s = T u = 5.

[0078] In Figure 6E , Figure 6F , Figure 6G and Figure 6HIn this case, the window size is 2 time steps (i.e., W). s =2) The flow rate is determined within a time window with a flow rate increment of 5 bbl / min (795 l / min). Similar to... Figure 6A , Figure 6B , Figure 6C and Figure 6D , Figure 6E , Figure 6F , Figure 6G and Figure 6H These correspond to pressure data, time derivative pressure data, moving average, and flow regulation for hydraulic production enhancement operations, respectively. Therefore, Figure 6E to Figure 6H The flow rules in the code use the following inputs: R0 = 5 bbl / min, R in = 5bbl / min (795l / min), R f =30bb1 / min (4770l / min), P max = 4000 psi (275.8 bar), and W s =T u =2. From Figure 6D and Figure 6H It can be seen that when the window size decreases, the flow rate regulation becomes more sensitive to fluctuations in processing pressure.

[0079] exist Figure 61 , Figure 6J , Figure 6K and Figure 6L In this case, the window size is 2 time steps (i.e., W). s =2) The flow rate is determined within a time window with a flow rate increment of 2.5 bbl / min. Similar to... Figure 6A , Figure 6B , Figure 6C and Figure 6D , Figure 61 , Figure 6J , Figure 6K and Figure 6L These correspond to pressure data, time derivative pressure data, moving average, and flow regulation for hydraulic production enhancement operations, respectively. Therefore, Figure 61 to Figure 6L The flow rules in the code use the following inputs: R0 = 5 bbl / min (7951 / min), R in = 5bbl / min (795l / min), R f =30bbl / min (4770l / min), P max = 4000 psi (275.8 bar), and W s =T u =2. From Figure 6H and Figure 6L It can be seen that using the same window size and reducing the traffic increment results in a smoother traffic increase.

[0080] In Figure 6M , Figure 6N , Figure 60 , Figure 6P , Figure 6Q , Figure 6R , Figure 6S and Figure 6T , Figure 6M to 6T shows an example where the window size and update interval are separated. In Figure 6M , Figure 6N , Figure 60 and Figure 6P , a time window with a window size of four time steps (i.e., W s = 4) is used to determine the flow rate, which is the same size as an update interval of four time steps (i.e., T u = 4). In contrast, Figure 6Q , Figure 6R , Figure 6S and Figure 6T show a different case where a time window with a window size of four time steps (i.e., W s = 4) is used to determine the flow rate, which is different from an update interval of 2 time steps (i.e., T u = 2). Thus, Figure 6P and Figure 6T show different flow rate adjustments, although the same pressure data is processed.

[0081] Turning to Figure 7 , Figure 7 shows a flow diagram in accordance with one or more embodiments. In particular, Figure 7 describes a particular method of using flow rate rules to determine recommended flow rate adjustments for a user to select. Figure 7 One or more of the blocks in Figure 1 , Figure 2 and Figure 3 may be performed by one or more components (e.g., hydraulic stimulation manager X 180) as described in Figure 7 Although the various blocks in may be presented and described sequentially, one having ordinary skill in the art will appreciate that some or all of these blocks can be executed in a different order, can be executed at the same time, can be omitted or combined, and some or all of these blocks can be executed in parallel. Furthermore, these blocks can be actively or passively executed.

[0082] In block 700, a hydraulic stimulation operation is initiated in accordance with one or more embodiments. In particular, the hydraulic stimulation operation can be automatically initiated by a hydraulic stimulation manager. In some embodiments, an operator provides one or more user inputs to begin the hydraulic stimulation operation, for example, at a fluid control system, a pump system, and / or a proppant system.

[0083] In block 705, according to one or more embodiments, a starting time step is selected to determine one or more flow adjustments. For example, the starting time step can be a current time step for analysis of possible flow adjustments or a start of pump system operation. Likewise, the starting time step can be a time step zero at a starting flow (R0) under a pumping flow (R t ).

[0084] In block 710, according to one or more embodiments, flow data is obtained for the selected time step for one or more pump systems. For example, flow data can be obtained from one or more flow meters for use in determining flow adjustments based on one or more flow rules. In some embodiments, flow data is not collected for determining flow adjustments.

[0085] In block 720, according to one or more embodiments, pressure data is obtained for the selected time step for one or more pump systems. For example, block 720 can be similar to block 400 described above in Figure 4 and corresponding description.

[0086] In block 730, according to one or more embodiments, time derivative pressure data is determined based on the pressure data. For example, block 730 can be similar to block 420 described above in Figure 4 and corresponding description.

[0087] In block 740, according to one or more embodiments, a time window is determined for analysis of the time derivative pressure data. In some embodiments, for example, the time window can be a dynamic time window having a window size that is adjusted during the hydraulic stimulation operation. The window size adjustment can be performed in response to detecting an increase and / or decrease in noise within the hydraulic stimulation operation.

[0088] In some embodiments, a moving average and / or smoothed value is determined for non-pressure data, such as density data or acoustic sensing data (e.g., using a discrete Fourier transform with a low pass filter or any other smoothing method). In cases where sensor data has significant interference or noise during the hydraulic stimulation operation, a corresponding moving average and / or smoothed value of the non-pressure data can be determined for use in flow adjustments.

[0089] In block 750, according to one or more embodiments, a moving average is determined using the time window and the time derivative pressure data. For example, block 750 can be similar to block 430 described above in Figure 4 and corresponding description. In some embodiments, smoothed pressure data is used to determine the time derivative pressure data for recommended flow adjustments (i.e., the moving average of the time derivative pressure data is not used for recommended flow adjustments).

[0090] At block 760, one or more recommended flow adjustments are determined based on one or more predetermined flow rules, flow data, an update interval, and a moving average, according to one or more embodiments. At each time step, the hydraulic stimulation manager can determine a recommended flow adjustment based on the moving average of the time derivative pressure data. The recommended flow adjustment can be determined according to the update interval. As such, two or more recommended flow adjustments can be determined and provided to the user device. For example, the moving average can be used to determine a flow adjustment table that can be displayed to the user within the graphical user interface. When additional time steps are made during the hydraulic stimulation operation, the one or more flow adjustment tables can be modified accordingly.

[0091] In some embodiments, the hydraulic stimulation manager executes a flow algorithm. Initially, the hydraulic stimulation manager can verify whether four criteria are met: 1) whether the current time step (t) is divisible by the update interval (T u ); 2) whether the flow at the previous time step (R t-1 ) is less than the target flow (R f ); 3) whether the treatment pressure (P t ) for the current time step is less than the maximum allowed treatment pressure (P max ); and 4) whether the moving average of the pressure derivative (dp / dt avg，t ) for the current time step is less than or equal to zero. If all four of the above criteria are met, the recommended flow adjustment is determined to be the adjusted flow that includes the previous flow at the previous time step (R t-1 ) and an additional flow increment (R in ). Thereafter, the algorithm can move to the next time step (t+1) and subsequent time steps to determine whether another flow adjustment is needed.

[0092] Continuing with the description of the flow algorithm, if one or more of the four criteria described above are not met, the flow algorithm can verify whether two additional criteria are met: 1) whether the current time step (t) is divisible by the update interval (T u ); and 2) whether the treatment pressure (P t ) for the current time step is greater than or equal to the maximum allowed treatment pressure (P max ). If both additional criteria are met, the recommended flow adjustment can be the flow at the previous time step (R t-1 ) minus a single flow increment (R in ). The flow algorithm can proceed to the next time step (t+1) to determine an additional flow adjustment. However, if one or both of the two additional criteria described above are violated, the current flow, i.e., the flow at the previous time step (R t-1 ), can be recommended.

[0093] In some embodiments, the flow algorithm is based on flow data that is based on measured pump flow. For example, the recommended flow adjustment can be based on measured flow. In particular, measured pressure and measured flow data can be continuously acquired. Thus, the hydraulic stimulation manager can use an additional rule to determine whether the difference between the recommended flow at the current time step and the measured flow at the previous time step is greater than some threshold (e.g., greater than one flow increment). If this condition is met, the recommended flow can be made the same as the recommended flow in the previous time step.

[0094] In block 765, one or more recommended flow adjustments are presented within the graphical user interface, in accordance with one or more embodiments.

[0095] Turning to Figure 8A , turn 8B, turn 8C, and turn 8D, Figure 8A to Figure 8D Examples of determining recommended flow adjustments are provided in accordance with one or more embodiments. In Figure 8A , the pressure is processed the same as the pressure data shown in Figure 6A , Figure 6E , Figure 61 , Figure 6M and Figure 6Q . Thus, the time derivative pressure data in Figure 8B is the same as the time derivative data in Figure 6B , Figure 6F , Figure 6J , Figure 6N and Figure 6R . Thus, the moving average in Figure 8C is based on a time window with window size W s = 4 seconds. In Figure 8D , the recommended flow is determined based on an update interval of T u = 2 seconds and measured flow. In the case of flow rules that are independent of measured flow data (except at the time of reaching the target flow), as described in Figure 6A to Figure 6T , the flow recommendation algorithm here checks whether the difference between the recommended flow at the current time step and the measured flow at the previous time step is greater than some threshold (e.g., greater than one flow increment). If this condition is met, the recommended flow will be the same as the recommended flow in the previous time step.

[0096] In addition, by making Figure 8A to Figure 8DThe flow rules in the determination of the recommended flow adjustments depend on the measured flow data from the previous time step, and the flow processing can handle the lag time in the flow regulation in real time. This can allow the pump system to catch up with the recommended flow regulation. Thus, various flow rules can implement a consultative system for flow control, where the hydraulic stimulation manager continuously obtains the acquired pressure data and flow data, while also recommending flow adjustments based on a predefined update interval. The recommended flow can be displayed for an operator to manually adjust the flow (rather than using a fully autonomous system). A consultative system can be used in large-scale applications, where the edge device displays the recommended flow regulation while continuously reading the pressure data and flow data.

[0097] Returning to Figure 7 , in block 770, a selection of one or more recommended flow adjustments is obtained using a graphical user interface according to one or more embodiments. For example, the hydraulic stimulation manager can communicate with one or more user devices, such as a human-machine interface located at a wellsite. The user can then select which recommended flow to use for the hydraulic stimulation operation.

[0098] In block 775, one or more commands are sent to adjust one or more flows with respect to one or more pump systems based on the selection of the one or more recommended flow adjustments, according to one or more embodiments.

[0099] In block 780, a determination is made as to whether the hydraulic stimulation operation is complete, according to one or more embodiments. For example, the hydraulic stimulation job can be designated for a particular time range for completion, such as when the pump systems connected to the flow manifold are designated for termination of the operation. Likewise, once a target flow is reached, Figure 7 The process described in the determination of the recommended flow adjustments can terminate when the hydraulic stimulation operation is complete. The process can end when the hydraulic stimulation operation is complete. In the event that the hydraulic stimulation operation is not complete, the process can proceed to block 790.

[0100] In block 790, another time step is selected for analysis of the hydraulic stimulation operation based on the update interval, according to one or more embodiments. For example, the time step can be incremented according to a particular time step interval, such as 1 second. Likewise, the next time step can be selected based on the update interval. In some embodiments, the increment between time steps changes, such as due to increased noise being detected during the hydraulic stimulation operation.

[0101] While the above Figure 4 and Figure 7Various embodiments for autonomous flow control in hydraulic stimulation operations (e.g., hydraulic fracturing treatments) are described, but other embodiments including various flow control processes (or other control processes) that need to be adjusted in response to variables in the surrounding environment can be contemplated. For example, similar techniques can be used to determine flow adjustments in any case where a master controller manages one or more pump systems.

[0102] Turning to Figure 8E , Figure 8E Examples of determining flow adjustments in accordance with one or more embodiments are provided. The following embodiments are for illustrative purposes only and are not intended to limit the scope of the disclosed technology. In Figure 8E , Figure 8E A well X 810 with corresponding sensor data (i.e., pressure data X 811 and flow data Y 812) is shown. For the pressure data X 811, a time derivative function 841 is applied first to produce time derivative pressure data (not shown). Then, a moving average function 842 is applied to the time derivative pressure data to produce a moving average of the time derivative pressure data 830. By using the moving average 830 and the flow data Y 812, a flow rule function 850 determines a flow adjustment Z 860 for the pump system.

[0103] In some embodiments, the pressure data involved in Figure 4 , Figure 5 , Figure 6A to Figure 6T , Figure 7 and Figure 8A to Figure 8D may be based on: 1) surface treatment pressure measurements transmitted in real-time from wellhead and / or pump ground sensors through the well network; 2) downhole treatment pressure measurements streamed in real-time from downhole sensors arranged in the wellbore (e.g., sensors attached to downhole casing strings and / or frac plugs and / or any other downhole completion equipment) through the well network; or 3) both surface and downhole treatment pressure measurements and / or any given combination. An advantage of using downhole pressure measurements compared to surface treatment pressure measurements can be that the formation response to the hydraulic stimulation operation can be captured faster, where the pressure can be sensed at a long distance from the target formation. Moreover, the downhole pressure measurements replace the usual estimation of frictional pressure due to the pumping of fluid down the casing string with actual downhole measurements, thus a more accurate representation of the formation response can be captured during the fracturing operation.

[0104] Turning to Figure 9 , Figure 9 A reinforcement learning system in accordance with one or more embodiments is shown. As Figure 9As shown, the reinforcement learning system (e.g., reinforcement learning system 915) can include one or more pump agents (e.g., pump agent X 951, pump agent Y 952, pump agent Z 953) that include hardware and / or software for performing one or more actions (e.g., action X 971, action Y 972, action Z 973). In some embodiments, for example, a pump agent can correspond to a pump system, such as Figure 1 and the corresponding description of the pump system (121, 122, 123). In another embodiment, a pump agent can be a pump controller in a particular pump system. Also, in some embodiments, a pump agent corresponds only to the control of hydraulic stimulation operations by the hydraulic stimulation manager that are distributed across multiple pump systems. Thus, the actions (971, 972, 973) can correspond to commands of the hydraulic stimulation manager or particular pump operations implemented by a pump controller.

[0105] Continuing the description Figure 9 , a reinforcement learning algorithm can determine actions based on the interaction of an agent with a particular environment. In other words, the environment can define one or more states resulting from these actions, and it can be described as a Markov Decision Process (MDP). Thus, a particular environment can have a set of states S, a set of actions A, where a transition model P(s'|s,a) can describe the probability that an action a in state s leads to state s', and a reward function R(s,a). An agent can perform different actions based on a policy π of the agent for this environment. With respect to the environment of hydraulic stimulation operations, various actions performed by a pump agent can result in a change of one or more states of the hydraulic stimulation environment. For example, a hydraulic stimulation environment (e.g., hydraulic stimulation environment X 965) can be a real-world well environment in which multiple mechanical agents interact with this real-world environment.

[0106] Based on the actions of a particular agent, for example, the hydraulic stimulation environment can experience various physical effects that are embodied in changes to pressure values (e.g., pressure values 963), acoustic sensing values (e.g., acoustic sensing values 962 that can be obtained from a DAS system), and / or flow values (e.g., flow values 961). Also, the hydraulic stimulation environment can be a particular geology output (e.g., represented by geology values 964). For example, shale reservoirs and carbonate formations can have different hydraulic stimulation environments based on their geological differences. In some embodiments, the hydraulic stimulation environment is a simulated environment in which a pump agent can be implemented as one or more computers that interact with the simulated environment. For example, a hydraulic stimulation operation can be simulated as a video game in which a pump agent can be simulated as a user playing the video game.

[0107] In some embodiments, the reinforcement learning system includes an action selector engine (e.g., action selector engine A 920). In particular, the action selector engine includes hardware and / or software having functionality to determine one or more actions based on one or more agent policies (e.g., agent policy 921) characterizing one or more current states of the hydraulic stimulation environment and observation data (e.g., observation data A 980). Some examples of the action selector engine can include machine learning models, such as artificial neural networks or random forests, that determine actions based on various input features within the observation data. Depending on changes to the observation data, a particular action can be instructed to be performed by the pump agent, e.g., through a command or control signal. In another embodiment, the action selector engine determines actual commands for components in the well network (e.g., the action can correspond to a command X 171 to adjust pump operation of the pump system A 121 in the well network). In another embodiment, the action selector engine determines action scores for different actions to adjust pump operation by the pump agent, where different action scores correspond to different flow adjustments. Figure 1

[0108] Turning to the observation data, the observation data can include pressure data, flow data, geologic data (such as obtained from well logging data or seismic data), and acoustic sensing data (e.g., from a DAS system). The observation data at a particular time step can include data from previous time steps that can be beneficial to characterize the hydraulic stimulation environment. In some embodiments, the reinforcement learning system can include a replay buffer (e.g., replay buffer 990) that stores observation data associated with different pump agent trajectories (e.g., pump agent X trajectory 991, pump agent Y trajectory 992, pump agent Z trajectory 993). For example, a trajectory can specify a sequence of observations characterizing respective states of the environment. In some embodiments, the trajectories can correspond to vectors of different states of the hydraulic stimulation environment, different actions performed by various pump agents, and / or various reward values obtained in response to different actions.

[0109] In some embodiments, the reinforcement learning system determines the agent policies of one or more agents using one or more pure reinforcement learning algorithms (e.g., without expert demonstrations). The pure reinforcement learning algorithms can start from a random policy that is continually improved through trial and error based on various rewards received using a reward function. For example, the policies can be trained using a software simulator for the hydraulic fracturing environment. The resulting simulated trained policies can provide a starting point in actual stimulation operations to further improve the policies.

[0110] ​In some embodiments, the reinforcement learning system includes a training system (e.g., training system 935). The training system can be coupled to the action selector engine and include hardware and / or software having functionality for updating one or more policy parameters (e.g., policy parameters 922) in the respective agent policies and / or one or more reward parameters (e.g., reward parameters 924) in the respective reward functions. In particular, the training system can use training data (e.g., training data 945) to iteratively update the agent policies and / or reward functions using one or more machine learning algorithms (e.g., machine learning algorithm C 933). Here, the training data can include demonstrations or trajectories of experts, which can be obtained by enabling a human to control the actions of the pump agent and recording the resulting expert observations of the hydraulic stimulation environment. The expert demonstrations can also correspond to historical data (e.g., historical observation data A 946), augmented data (e.g., augmented observation data A 947), and / or synthetic data (e.g., synthetic observation data C 948), which provide optimized actions with respect to particular states of the hydraulic stimulation environment. Accordingly, the training system can use a loss function (e.g., loss function D 934) to determine a difference between the demonstrations of the experts and selected actions based on the one or more agent policies. The training system can use the difference to update the parameters within the reinforcement learning system through one or more imitation learning processes.

[0111] For the imitation learning processes, the reinforcement learning system can use the imitation learning processes to learn an optimal agent policy or an optimal reward function from the demonstrations or supervisory signals of the experts. Such techniques can be distinguished from pure reinforcement learning algorithms that learn from sparse rewards or by manually specifying a reward function. In other words, by having a teacher demonstrate desired behavior rather than artificially designing such behavior, the reinforcement learning system can have an easier path to learning an optimal agent policy or actual reward function. Accordingly, the demonstrations of the experts can form a trajectory τ = (s0, a0, s1, a1,...), where the actions of the experts are determined by a policy of the experts, which can correspond to an optimal policy. In some embodiments, the experts can be queried during a training process to obtain training data for a particular hydraulic stimulation environment.

[0112] In one embodiment, the imitation learning processes are behavior cloning processes. For example, the reinforcement learning system can use behavior cloning to directly map one or more states to one or more actions, thereby forming state-action pairs. Based on the demonstrations of the experts, the state-action pairs can be determined using supervised learning and a loss function. Hydraulic stimulation environments that have a set of identifiable states can be well suited for behavior cloning processes.

[0113] In another embodiment, the imitation learning process is a direct policy learning (DPL) process or a direct policy search process that uses the training data to determine a policy of an agent that maximizes expected rewards and / or reduces expected losses. Specifically, the DPL process can iteratively access the interactive expert during the training process. With sufficient training data, the pump agent can remember past mistakes and train the policy of the agent so as to converge to an optimal policy. For example, the DPL process can implement a data aggregation algorithm or a policy aggregation algorithm. In the data aggregation algorithm, the data aggregation algorithm can train the policy of the agent using the entire training dataset. In the policy aggregation algorithm, the policy aggregation algorithm trains the policy of the agent based on the training data obtained in previous iterations and subsequently combines the current policy with the previous policies using geometric blending.

[0114] In another embodiment, the imitation learning process is an inverse reinforcement learning (IRL) process. For example, the IRL process can determine a reward function of the hydraulic stimulation environment based on the demonstrations of the expert. After determining the actual reward function, the IRL process can use reinforcement learning to determine an optimal policy that maximizes the identified reward function. In particular, the reward function can be parameterized (e.g., reward parameters 924) and the reward parameters can be iteratively updated. After identifying the reward function and the optimal policy, the optimal policy can be compared to the policy of the identified expert.

[0115] Continuing the description of the IRL process, the IRL process method can be a model-based algorithm or a model-free algorithm. In the model-based algorithm, the reinforcement learning system can determine a linear reward function or a forward model of the hydraulic stimulation environment for learning a policy of the agent. In the model-free algorithm, the reward function can be complex and thus a machine learning model (e.g., a neural network) can be used to approximate the complex reward function.

[0116] While Figure 9 Various configurations of the components are shown, but other configurations can be used without departing from the scope of the disclosure. For example, Figure 9 Various components in can be combined to form a single component. As another example, the functionality of a single component can be performed by two or more components.

[0117] Turning to Figure 10 , Figure 10 Flowcharts in accordance with one or more embodiments are shown. Specifically, Figure 10 A particular method of determining traffic rules using a reinforcement learning algorithm is described. Figure 10 One or more blocks in can be performed by as Figure 1 , Figure 2 , Figure 3 and Figure 9One or more components (e.g., hydraulic stimulation manager X 180 or reinforcement learning system 110) described in the middle are performed. Although the various blocks in Figure 10

[0118] In block 1000, a hydraulic stimulation operation having a hydraulic stimulation environment is initiated, according to one or more embodiments.

[0119] In block 1010, observation data regarding the hydraulic stimulation environment is obtained, according to one or more embodiments.

[0120] In block 1020, an action for a pump agent is determined based on the observation data, an agent policy, and a reinforcement learning algorithm, according to one or more embodiments. For example, reinforcement learning can be used to control flow rates during the hydraulic stimulation operation.

[0121] In some embodiments, the agent policy can be trained using historical fracturing data, where flow rates were previously controlled using a rules-based algorithm. One technique for training the agent policy can include assigning a positive reward when a relatively low pressure value is obtained, assigning a positive reward when a smoother overall pressure behavior is obtained (e.g., avoiding sudden pressure spikes), assigning a negative reward when a sudden increase in pressure is obtained, and so on, using the previous fracturing data. Thus, the agent policy can be trained to recommend various flow rate adjustments that result in a gradual pressure ramp down and a desired pressure behavior. By increasing the number of hydraulic stimulation operations, the agent can learn and improve.

[0122] In block 1030, observation data regarding the hydraulic stimulation environment is obtained in response to performance of the action by the pump agent, according to one or more embodiments.

[0123] In block 1040, a reward value is determined using the observation data and a reward function associated with the pump agent, according to one or more embodiments. In a reinforcement learning algorithm, the pump agent can perform an action in the hydraulic stimulation environment and then receive a positive or negative reward. For example, the goal of the pump agent can be to maximize its cumulative (positive) reward over the long term.

[0124] ​In block 1050, the agent policy is updated based on the reward values, in accordance with one or more embodiments. For example, the reinforcement learning system can adjust the policy of the agent based on various rewards obtained through previous actions. In some embodiments, the training system is used to update the agent policy. In particular, one or more imitation learning techniques can be used to learn and / or adjust the agent policy or reward function based on the training data. For example, a loss function can be used to obtain a mismatch between the observation data and the training data. Accordingly, the policy parameters or reward parameters can be adjusted based on the error data associated with the mismatch.

[0125] In block 1060, it is determined whether the hydraulic stimulation operation is complete, in accordance with one or more embodiments. When the hydraulic stimulation operation is complete, the process can end. In the event that the hydraulic stimulation operation is not complete, the process can proceed to block 1020 to determine another action for the pump agent in accordance with the updated agent policy.

[0126] Embodiments can be implemented on a computer system. Figure 11 is a block diagram of a computer system 1102 to provide computational functionality associated with the described algorithms, methods, functions, processes, flows, and procedures as described in this disclosure, in accordance with an implementation. The computer 1102 shown is intended to encompass any computing device such as a high-performance computing (HPC), server, desktop computer, laptop / notebook computer, wireless data port, smart phone, personal data assistant (PDA), tablet computing device, one or more processors within these devices, or any other suitable processing devices including physical or virtual instances of the computing devices (or both), including instances of the computing devices in a cloud computing environment. Additionally, the computer 1102 can include a computer that includes an input device, such as a keypad, keyboard, touch screen, or other device that can accept user information, and an output device that conveys information associated with the operation of the computer 1102, including digital data, visual or audio information (or a combination of information), or a GUI.

[0127] The computer 1102 can serve in the capacity of a client, network component, server, database, or other persistent memory device, or other component (or combination of roles) of a computer system used in executing the subject matter described in this disclosure. The computer 1102 shown is communicably coupled with a network 1130 or cloud. In some implementations, one or more components of the computer 1102 can be configured to operate within an environment that includes a cloud-computing-based, local, global, or other environment (or combination of environments).

[0128] At a high level, computer 1102 is an electronic computing device operable to receive, transmit, process, store, or manage data and information associated with the described subject matter. According to some implementations, computer 1102 can also include or be communicably coupled to an application server, an email server, a web server, a caching server, a streaming data server, a business intelligence (BI) server, or other server (or combination of servers).

[0129] Computer 1102 can receive requests from client applications (for example, client applications executing on another computer 1102) over network 1130 or cloud and respond to the received requests by processing them in appropriate software applications. Further, requests can also be sent to computer 1102 from internal users (for example, from a command console or through other appropriate access methods), external or third-parties, other automated applications, and any other appropriate entities, individuals, systems, or computers.

[0130] Each of the components of the computer 1102 can communicate using the system bus 1103. In some implementations, any or all of the components of the computer 1102, whether hardware, software, or a combination of hardware and software, can be connected or communicatively coupled to, or with, the interface 1104 (or a combination of the two) using the system bus 1103, an application programming interface (API) 1112, or a service layer 1113 (or a combination of the API 1112 and the service layer 1113). The API 1112 can include specifications for routines, data structures, and object classes. The API 1112 can be computer language-independent or dependent, and refer to complete interfaces, individual functions, or even a set of APIs. The service layer 1113 provides software services to other components of the computer 1102 or other components (whether illustrated or not) that are

[0131] The computer 1102 includes an interface 1104. Although shown as a single interface 1104 in Figure 11 FIG. 10, two or more interfaces 1104 can be used according to the particular needs, desires, or particular implementations of the computer 1102. The interface 1104 is used by the computer 1102 for communicating with some other system that is connected to the network 1130. Generally, the interface 1104 includes logic encoded in software or hardware (or a combination of software and hardware) and is operable to communicate with the network 1130. More specifically, the interface 1104 can include software supporting one or more communication protocols associated with communications such that the network 1130 or hardware of the interface 1104 are used for

[0132] The computer 1102 includes at least one computer processor 1105. Although shown as a single computer processor 1105 in Figure 11 FIG. 10, two or more processors can be used according to the particular needs, desires, or particular implementations of the computer 1102. Generally, the computer processor 1105 executes instructions and manipulates data to operate the computer 1102 and to implement algorithms, methods, functions, processes, flows, and procedures as described in this disclosure.

[0133] The computer 1102 also includes memory 1106 that retains data used by the computer 1102 or other components (or a combination thereof) connected to the network 1130. For example, the memory 1106 can be a database that stores data consistent with this disclosure. Although shown as a single memory 1106 in Figure 11 FIG. 10, two or more memories can be used according to the particular needs, desires, or particular implementations of the computer 1102 and the described functionality. While the memory 1106 is illustrated as an integral component of the computer 1102, in alternative implementations, the memory 1106 can be external to the computer 1102.

[0134] The application 1107 is an algorithmic software engine that provides functionality according to the particular needs, desires, or particular implementations of the computer 1102, and particularly with respect to the functionality described in this disclosure. For example, the application 1107 can be used as one or more components, modules, applications, etc. Further, although shown as a single application 1107, the application 1107 can be implemented as multiple applications 1107 on the computer 1102. Also, although shown as integrated with the computer 1102, in alternative implementations, the application 1107 can be external to the computer 1102.

[0135] There can be any number of computers 1102 associated with or external to the computer system that hosts the computer 1102, each computer 1102 communicating over network 1130. Additionally, the term "client," "user," and other appropriate terminology can be used interchangeably as appropriate, without departing from the scope of the present disclosure. Moreover, the present disclosure contemplates that many users might use one computer 1102, or that one user might use multiple computers 1102.

[0136] In some embodiments, the computer 1102 is implemented as part of a cloud computing system. For example, the cloud computing system can include one or more remote servers and various other cloud components, such as cloud storage units and edge servers. In particular, the cloud computing system can perform one or more computing operations without the direct active management of a user device or local computer system. As such, the cloud computing system can have different functionality distributed across multiple locations from a central server, which can be performed using one or more Internet connections. More particularly, the cloud computing system can operate according to one or more service models, such as Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), Mobile "Backend" as a Service (MBaaS), serverless computing, and / or Function as a Service (FaaS).

[0137] While only a few embodiments have been described above, it is understood by those skilled in the art that many modifications can be made of the illustrated embodiments, without departing from the scope of the disclosure. Therefore, all such modifications are intended to be included within the scope of the disclosure as defined by the appended claims. In the claims, any functional limits are intended to cover structures described herein that perform the described function and equivalents of those structures. Similarly, any functional limit steps in the claims are intended to cover acts described herein that perform the described function and equivalents of those acts.

Claims

1. A method for autonomous flow control in hydraulic production enhancement operations, comprising: The time derivative pressure data is determined by a computer processor based on pressure data of a pump system performing hydraulic enhancement operations in a geological area; The computer processor determines the moving average based on the time derivative stress data and a predetermined time window; The computer processor determines the flow regulation of the pump system based on the moving average, the update interval for adjusting the flow rate, and one or more predetermined flow rate rules, wherein the size of the predetermined time window is different from the update interval; as well as The computer processor, based on the flow regulation, sends a command to the pump system to change the flow rate within the hydraulic production enhancement operation.

2. The method according to claim 1, in, The pressure data is obtained from multiple downhole sensors, including a first downhole sensor coupled to the casing of the wellbore and a second downhole sensor coupled to the fracturing plug inside the wellbore. The multiple downhole sensors transmit the pressure data in real time to the hydraulic production enhancement manager located on the surface of the well site via the well network.

3. The method according to claim 1, in, The one or more predetermined traffic rules are determined by a reinforcement learning system that includes an action selector engine and a training system. Wherein, the one or more predetermined traffic rules correspond to one or more agent policies determined by a reinforcement learning algorithm, and The one or more predetermined flow rules define the size of the dynamic time window, the size of the flow regulation increment, and multiple conditions for increasing, decreasing, and maintaining the flow of the pump system.

4. The method according to claim 1, in, The one or more predetermined traffic rules include a first rule, a second rule, and a third rule. The first rule corresponds to not performing flow regulation when the moving average of the time derivative pressure data is positive. Wherein, the second rule corresponds to the positive flow increment when the moving average of the time derivative pressure data is negative or equal to zero, and The third rule corresponds to the negative flow increment when the pressure value at the current time step is equal to or greater than the predetermined maximum pressure value.

5. The method according to claim 1, further comprising: Flow data about the pump system is obtained from the flow sensor. The one or more traffic rules are based on traffic data from previous time steps.

6. The method according to claim 1, further comprising: Multiple recommended flow adjustments for the pump system are determined based on the moving average and one or more predetermined flow rules; The multiple recommended traffic adjustments are presented within the graphical user interface of the user device; In response to selecting a recommended flow regulation from the plurality of recommended flow regulation options, a command is sent to the pump system to implement the recommended flow regulation.

7. The method according to claim 1, further comprising: Determine the pressure curve based on the pressure data; as well as A smoothing operation is performed on the pressure curve using a discrete Fourier transform with a low-pass filter or using a median filter to produce a smooth pressure curve. The smoothed pressure curve is used to determine the time derivative pressure data.

8. The method according to any one of claims 1 to 7, in, The pump system delivers hydraulic fracturing fluid to the wellbore at a predetermined flow rate during hydraulic production enhancement operations. The hydraulic fracturing fluid contains at least one proppant, and The hydraulic fracturing fluid generates a network of fractures laterally from the wellbore.

9. A system for autonomous flow control in hydraulic production enhancement operations, comprising: Pump system, the pump system including positive displacement pump; Multiple sensors coupled to the pump system, wherein the multiple sensors determine pressure data regarding hydraulic production enhancement operations; and A hydraulic production enhancement manager including a computer processor, wherein the hydraulic production enhancement manager is coupled to the pump system, and the hydraulic production enhancement manager includes the following functions: Determine the time derivative pressure data based on the pressure data of the pump system; A moving average is determined based on the time derivative pressure data and a predetermined time window; Flow regulation for the pump system is determined based on the moving average, an update interval for adjusting flow, and one or more predetermined flow rules, wherein the size of the predetermined time window differs from the update interval; and Based on the flow regulation, a command to change the flow rate within the hydraulic production enhancement operation is sent to the pump system.

10. The system according to claim 9, in, The one or more predetermined traffic rules are determined by a reinforcement learning system that includes an action selector engine and a training system. Wherein, the one or more predetermined traffic rules correspond to one or more agent policies determined by a reinforcement learning algorithm, and The one or more predetermined flow rules define the size of the dynamic time window, the size of the flow regulation increment, and multiple conditions for increasing, decreasing, and maintaining the flow of the pump system.

11. The system according to claim 9, in, The one or more predetermined traffic rules include a first rule, a second rule, and a third rule. The first rule corresponds to not performing flow regulation when the moving average of the time derivative pressure data is positive. Wherein, the second rule corresponds to the positive flow increment when the moving average of the time derivative pressure data is negative or equal to zero, and The third rule corresponds to the negative flow increment when the pressure value at the current time step is equal to or greater than the predetermined maximum pressure value.

12. The system according to claim 9, wherein, The hydraulic production enhancement manager also includes the following functions: Multiple recommended flow adjustments for the pump system are determined based on the moving average and one or more predetermined flow rules; The multiple recommended traffic adjustments are presented within the graphical user interface of the user device; In response to selecting a recommended flow regulation from the plurality of recommended flow regulation options, a command is sent to the pump system to implement the recommended flow regulation.

13. The system according to claim 9, wherein, The hydraulic production enhancement manager also includes the following functions: Obtain the first observational data on the environment for hydraulic power enhancement; A first action of the pump agent is determined based on the agent strategy and the first observation data, wherein the pump agent corresponds to the pump system, and wherein the first action corresponds to the flow regulation of the pump system; In response to the pump agent performing the first action, second observational data regarding the hydraulic production enhancement environment is obtained; The reward value of the pump agent is determined based on the second observation data and the reward function; and The agent policy is updated based on the reward value to generate an updated policy. The updated strategy determines one or more actions of the pump agent.

14. The system according to any one of claims 9 to 13, further comprising: A training system that obtains training data about multiple pump agents. The pump agent is trained based on a loss function and the mismatch between the training data and observational data regarding one or more hydraulic enhancement operations. Among them, multiple policy parameters of the updated policy are adjusted based on the mismatch.

15. The system of claim 14, further comprising: A replay buffer that obtains multiple pump agent trajectories for multiple pump agents, wherein the training system updates the reward function based on the multiple pump agent trajectories.

Images