Shale gas well fracturing parameter optimization design method and system based on depth Q network

Abstract

The invention discloses a shale gas well fracturing parameter optimization design method and system based on a depth Q network, and relates to the technical field of shale gas development, and the method comprises the following steps: S1, collecting geological parameters, fracturing construction parameters and productivity data of a shale gas well in advance; s2, constructing an agent model and performing environment simulation; and S3, constructing a DQN model. According to the method, a LightGBM algorithm is adopted to construct a data-driven proxy model to simulate a real fracturing environment, an optimization model is constructed based on a deep Q network (DQN), state, action, reward and epsilon-greedy strategies are defined, and technologies such as a variable step size search mechanism, experience playback and target network soft update are combined, so that the real fracturing environment is simulated. The problems that a traditional optimization method is weak in local search capability and low in convergence speed are effectively solved, multivariable synchronous optimization of the unit perforation length proppant dosage and the fracturing fluid dosage is achieved, and a global optimal parameter combination can be rapidly explored.

Description

Technical Field

[0001] This invention relates to the field of shale gas development technology, and more specifically, to a method and system for optimizing fracturing parameters in shale gas wells based on deep Q-networks. Background Technology

[0002] Shale gas, as an important unconventional oil and gas resource in the global energy structure, has reservoirs characterized by ultra-low permeability, high organic matter content, and complex fracture development. Under natural conditions, its recovery rate is extremely low, necessitating a combination of horizontal drilling and hydraulic fracturing technologies for industrial development. Hydraulic fracturing, through high-pressure injection of fracturing fluid and proppant, creates a complex network of fractures in the reservoir, thereby improving the seepage channels for shale gas. The rationality of its construction parameters directly determines the fracture geometry, conductivity, and final gas production efficiency. Therefore, the optimized design of fracturing parameters is a core element in shale gas development, aimed at increasing production capacity and reducing costs, and is crucial for achieving the economical and efficient exploitation of shale gas resources.

[0003] Traditional fracturing parameter optimization methods are mainly divided into two categories: numerical simulation and traditional machine learning optimization methods, but both have obvious limitations. Numerical simulation methods describe the fracturing fluid flow, fracture propagation, and gas reservoir seepage processes by establishing physical models. Although they can reflect certain physical mechanisms, they rely on a large number of ideal assumptions and are difficult to accurately characterize complex phenomena such as reservoir heterogeneity, fracture dynamic evolution, and inter-well pressure crossflow, leading to deviations between predicted results and actual production capacity. At the same time, numerical simulation requires a large amount of computing resources and depends on complete field geological and engineering data support. The model verification cycle is long, and the flexibility is insufficient, failing to meet the needs of rapid optimization of field construction parameters. Traditional machine learning optimization methods, such as genetic algorithms and particle swarm optimization, optimize parameters by mining potential patterns in historical data. However, these methods suffer from weak local search capabilities, slow convergence speeds, and difficulty in achieving simultaneous optimization of multiple variables. That is, there is a complex nonlinear coupling relationship between fracturing parameters, such as proppant dosage and fracturing fluid dosage, and geological parameters, such as total organic carbon content and brittle mineral content. Adjusting a single variable often fails to achieve the global optimum. Furthermore, traditional algorithms are prone to getting trapped in local optima when dealing with high-dimensional and strongly coupled optimization problems, making it difficult to adapt to the personalized optimization needs of shale gas wells under different geological conditions.

[0004] Meanwhile, in recent years, the application of machine learning technology in the oil and gas development field has gradually expanded. Some scholars have attempted to use models such as random forests, support vector machines, and long short-term memory networks for production capacity prediction, or combine them with genetic algorithms and Bayesian optimization for parameter adjustment, achieving some progress. However, these methods still lack the ability to dynamically adjust strategies and cannot optimize parameter combinations in real time based on reservoir response. Reinforcement learning, as a model-free machine learning paradigm, learns optimal strategies through continuous interaction between the agent and the environment. It has advantages in long-term cumulative benefit optimization and handling complex high-dimensional problems, and has shown good results in fields such as traffic control and energy dispatch. However, in the field of shale gas well fracturing parameter optimization, the application of reinforcement learning is still in the exploratory stage. The core bottleneck lies in the accurate modeling of the real fracturing environment. That is, the fracturing process involves multi-physics coupling, which is difficult to completely replicate through analytical models. Data-driven environmental modeling also faces problems such as high data quality requirements and insufficient generalization ability. At present, a mature reinforcement learning-based fracturing parameter optimization method and system has not yet been formed, which cannot effectively solve the shortcomings of traditional methods in multivariate synchronous optimization and dynamic adaptation to different geological conditions, thus restricting the further improvement of shale gas development efficiency.

[0005] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0006] To address the problems in related technologies, this invention proposes a method and system for optimizing fracturing parameters in shale gas wells based on deep Q-networks, in order to overcome the aforementioned technical problems existing in existing related technologies.

[0007] The technical solution of this invention is implemented as follows:

[0008] One aspect of the present invention:

[0009] A method and system for optimizing fracturing parameters in shale gas wells based on deep Q-networks includes the following steps:

[0010] Step S1 involves pre-collecting geological parameters, fracturing operation parameters, and production capacity data for shale gas wells. The geological parameters include: vertical depth, TOC content, porosity, thickness of high-quality reservoir, gas saturation, pressure coefficient, and BMC. The fracturing operation parameters include proppant usage per unit perforation length and fracturing fluid usage per unit perforation length. The production capacity data is represented by a production capacity index indicating the maximum average daily gas production Qg in the first year. The collected data is then filtered to remove outliers and divided into training and testing sets.

[0011] Step S2: Construct an agent model and simulate the environment, including calibrating the LightGBM algorithm to construct an agent model based on the training set and test set, and using the agent model as the environment for reinforcement learning to receive agent actions and provide corresponding states and rewards.

[0012] Step S3, construct the DQN model, including the following steps:

[0013] Define the state, including calibrating the state space as follows: ,in, For current fracturing capacity, The amount of fracturing fluid used per unit perforation length. Propionage amount per unit perforation length;

[0014] Define the action, including the action variables. and Each variable includes three change strategies: increase, decrease, and remain unchanged, forming nine action combinations. The actions are adjusted using a variable step size method, with the step size based on... The design ensures that the parameter values ​​do not exceed the preset upper and lower limits after the action is executed.

[0015] Define the reward, including the reward value being calculated based on the rate of change between the optimized Qg and the initial Qg. A positive rate of change results in a positive reward, with a higher rate of change leading to a higher reward value. A negative rate of change results in a negative reward, and a zero rate of change results in a zero reward value.

[0016] The strategy is defined, and an ε-greedy strategy is adopted for action selection. The ε value gradually decreases from the initial value of 1 to 0.01 through exponential decay. The balance model is explored and utilized.

[0017] Step S4: Perform model training, initialize the weights of the Q network and target network of DQN, set up an experience replay buffer, and convert the training data into a transition tuple of state, action, reward and next state and store it in the buffer; during iterative training, randomly sample small batches of data from the buffer, calculate the target Q value and minimize the loss function through gradient descent, and update the target network weights using a soft update method until the model converges.

[0018] Step S5: Optimize fracturing parameters by inputting the initial parameters of the shale gas well to be optimized into the trained DQN model. The model then interacts with the surrogate model to output the optimal proppant dosage and fracturing fluid dosage per unit perforation length.

[0019] Wherein, the variable step size adjustment of the action in step S3 satisfies , Furthermore, when the parameters are adjusted beyond this range, they are automatically truncated to the corresponding boundary value.

[0020] The calculation method for the reward value in step S3 is expressed as follows:

[0021]

[0022] in, This refers to the fracturing capacity after the action is executed; This refers to the fracturing capacity obtained after implementing the action strategy. This represents the initial gas production of each well.

[0023] The network structure of the DQN model includes an input layer, two hidden layers, and an output layer. The input layer has a dimension of [1,3], the hidden layer has 64 nodes, the activation function is ReLU, and the output layer has a dimension of [9], corresponding to the Q-values ​​of 9 actions. The training parameters are set as follows: discount factor γ=0.99, learning rate α=0.0005, mini-batch size=64, experience replay buffer size=10000, target network soft update parameter τ=0.001, maximum number of training rounds=500, and maximum number of time steps per round=2000.

[0024] Another aspect of the present invention:

[0025] A shale gas well fracturing parameter optimization design system based on deep Q-networks, used in the aforementioned shale gas well fracturing parameter optimization design method based on deep Q-networks, includes:

[0026] The data acquisition module is used to collect geological parameters, fracturing operation parameters, and production capacity data of shale gas wells, and to perform preprocessing.

[0027] The proxy model module is used to build a production capacity prediction model based on the LightGBM algorithm, simulate the real environment of shale gas well fracturing, receive action inputs and output the corresponding production capacity prediction results.

[0028] The DQN optimization module is used to build the DQN module, including a state encoding unit, action generation unit, reward calculation unit, policy execution unit, and network training unit, which are used to implement state definition, action generation, reward feedback, policy selection, and model training.

[0029] The parameter output module is used to combine the trained DQN model with the initial parameters of the well to be optimized and output the optimal combination of fracturing parameters.

[0030] The storage module is used to store raw data, training sets, test sets, surrogate model parameters, DQN model parameters, and optimization results.

[0031] Furthermore, the data acquisition module includes a data filtering unit and a data partitioning unit. The data filtering unit is used to remove abnormal data, and the data partitioning unit divides the training set and the test set in a 7:3 ratio.

[0032] Furthermore, the network training unit in the DQN optimization module uses the Adam optimizer to minimize the loss function and eliminates sample correlation through an empirical replay mechanism to improve training stability.

[0033] The beneficial effects of this invention are:

[0034] This invention employs the LightGBM algorithm to construct a data-driven surrogate model to simulate the real fracturing environment. Compared to traditional numerical simulation methods, it does not rely on complex physical assumptions, can more accurately capture the nonlinear coupling relationship between geological parameters, fracturing parameters, and production capacity, and has lower modeling costs and faster response speed, providing efficient and reliable environmental support for the optimization process. The optimization model, based on a deep Q-network (DQN), effectively solves the problems of weak local search ability and slow convergence speed in traditional optimization methods by defining states, actions, rewards, and an ε-greedy strategy, combined with a variable step-size search mechanism, experience replay, and soft update of the target network. It also achieves multi-variable synchronization of proppant dosage per unit perforation length and fracturing fluid dosage. This optimization method enables rapid exploration of globally optimal parameter combinations. The method and system possess strong versatility and adaptability, eliminating the need to adjust the core framework for different geological conditions. It can be adapted to the personalized optimization needs of various shale gas wells through a data-driven approach, breaking the dependence of traditional methods on specific reservoirs. Simultaneously, the system integrates full-process functions such as data processing, model training, parameter optimization, and result output, providing on-site engineers with intuitive and accurate guidance for fracturing parameter design. This reduces errors caused by experience-based decision-making, facilitating the scientific and personalized formulation of fracturing construction plans. Ultimately, while improving fracturing effectiveness and production capacity, it optimizes construction cost input, providing strong technical support for the economical and efficient development of shale gas resources. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating the shale gas well fracturing parameter optimization design method based on a deep Q-network according to an embodiment of the present invention.

[0037] Figure 2 The present invention relates to a method for optimizing fracturing parameters of shale gas wells based on deep Q-networks according to an embodiment of the present invention, which compares the predicted gas production capacity with the actual Qg, wherein (a) is the LR method; (b) is the SVM method; and (c) is the LightGBM method.

[0038] Figure 3 This is a schematic diagram of the shale gas well fracturing construction parameter optimization process based on DQN, according to an embodiment of the present invention.

[0039] Figure 4 This is a learning curve during the well_1 training of the shale gas well fracturing parameter optimization design method and system based on deep Q-network according to an embodiment of the present invention;

[0040] Figure 5 This is a comparison chart of the Qg changes predicted by the DQN agent of well_1 in different training rounds of the shale gas well fracturing parameter optimization design method and system based on deep Q network according to an embodiment of the present invention.

[0041] Figure 6 This is a comparison chart of the variation curves of the maximum Qg per round and the corresponding fracturing construction parameters of the shale gas well fracturing parameter optimization design method and system based on deep Q network according to an embodiment of the present invention.

[0042] Figure 7 This is a learning curve during the Well_2 training of the shale gas well fracturing parameter optimization design method and system based on deep Q network according to an embodiment of the present invention.

[0043] Figure 8 This is a comparison chart of the changes in Qg predicted by the well_2 DQN agent of the shale gas well fracturing parameter optimization design method and system based on deep Q-network according to an embodiment of the present invention, under different training rounds;

[0044] Figure 9 This is a comparison chart of the variation curves of the maximum Qg per round and the corresponding fracturing construction parameters of the shale gas well fracturing parameter optimization design method and system based on deep Q network according to an embodiment of the present invention.

[0045] Figure 10 This is a learning curve during well_3 training of a shale gas well fracturing parameter optimization design method and system based on a deep Q-network according to an embodiment of the present invention.

[0046] Figure 11 This is a comparison chart of the Qg changes predicted by the DQN agent of well_3 in different training rounds of the shale gas well fracturing parameter optimization design method and system based on deep Q network according to an embodiment of the present invention.

[0047] Figure 12 This is a comparison chart of the variation curves of the maximum Qg per round and the corresponding fracturing construction parameters of the well_3 of the shale gas well fracturing parameter optimization design method and system based on the depth Q network according to an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0049] According to an embodiment of the present invention, a method for optimizing fracturing parameters of shale gas wells based on depth Q-networks is provided.

[0050] like Figure 1 As shown, the shale gas well fracturing parameter optimization design method based on depth Q-network according to an embodiment of the present invention includes the following steps:

[0051] Step S1 involves pre-collecting shale gas well data from shale gas development blocks such as the Weiyuan Block in the Sichuan Basin, covering three main categories: geological parameters, fracturing operation parameters, and production capacity data. Geological parameters include vertical depth, TOC (total organic carbon) content, porosity, thickness of high-quality reservoirs, gas saturation, pressure coefficient, and BMC (brittle mineral content). Fracturing operation parameters focus on two core controllable parameters: proppant usage per unit perforation length and fracturing fluid usage per unit perforation length. The production capacity target is the maximum average daily gas production Qg in the first year.

[0052] Specifically, outliers are removed from the collected data to ensure data quality. The data is then divided into training and testing sets in a 7:3 ratio for subsequent model training and performance validation.

[0053] Step S2: Based on the training and test sets, construct three capacity prediction models: LR (Linear Regression), SVM (Support Vector Machine), and LightGBM (Light Gradient Boosting Machine). MSE (Mean Squared Error) is used as the model performance evaluation metric. The MSE calculation formula is as follows:

[0054]

[0055] Where N represents the number of shale gas wells. This is the actual Qg value. To predict the Qg value.

[0056] like Figure 2 As shown, by comparing the MSE values ​​of the three algorithms, the LightGBM algorithm with the best prediction accuracy was selected to construct the surrogate model. This surrogate model serves as the environment for reinforcement learning, capable of receiving fracturing parameter adjustment actions output by the agent, simulating the real fracturing process, and providing corresponding production capacity status and rewards. During model training, five-fold cross-validation was used to adjust hyperparameters, ensuring that the final surrogate model's MSE is less than 0.01.

[0057] Step S3, construct the DQN (Deep Q-Network) model, such as... Figure 3 As shown, it includes the following steps:

[0058] Step S301: Define the state, and design its state space as a three-dimensional vector. ,in, Represents current fracturing capacity. This represents the current fracturing fluid consumption per unit perforation length. This represents the proppant usage per unit perforation length. During model initialization, the initial parameters of the well to be optimized are input as the initial state.

[0059] Step S302, define the action, to and For each action variable, three adjustment strategies are designed: increase, decrease, and remain unchanged, resulting in 9 possible action combinations, represented by an action space of size 9. Due to construction constraints, and The range of variation must not exceed its maximum and minimum values. The specific parameter changes for each action are shown in Table 1, as detailed below:

[0060] Table 1. Reference Table for the Operation of the DQN Model

[0061] Step S303: Define the reward. In reinforcement learning, each action receives feedback from the environment in the form of a reward or penalty. The system's goal is to maximize the reward and achieve optimal decision-making by optimizing the action policy. The reward represents the agent's learning ability and whether the agent made the best choice in any given situation. Specifically, it is given based on the rate of change of the optimized Qg obtained by the agent relative to the initial Qg, expressed as:

[0062]

[0063] in, This refers to the fracturing capacity obtained after the system executes the action strategy of the intelligent agent. This is the initial gas production of each well, i.e., the unoptimized production capacity. The environment is automatically refreshed at the end of each training session. As one of the initial states.

[0064] Among them, if the agent's decision-making is reduced Compared to the initial fracturing capacity, a negative reward will be obtained. The greater the decrease, the larger the absolute value of the negative reward. This tells the agent that this strategy has a negative impact, so the strategy should be changed to try to obtain a positive reward. Conversely, the higher the ratio, the larger the positive reward, and the agent will be encouraged to take such actions to increase the reward value, expressed as:

[0065]

[0066] Step S304: Define the strategy and adopt... - A greedy strategy is used to select actions. The initial value is 1, representing full exploration, and it gradually decreases to 0.01 using an exponential decay formula, as shown below:

[0067] ;

[0068] in, This is the initial value of the greedy factor, set to 1, meaning a random exploration strategy is used at the beginning. The value at the end of the decay is... , This represents the decay rate of the greed factor; the larger the value, the slower the decay. A value of 50 is used. In the current round, the balance model's ability to explore unknown actions and utilize known optimal actions.

[0069] Step S4: Model training constructs a four-layer neural network as the Q-network and target network. The input layer dimension is [1,3], the two hidden layers each have 64 nodes, the activation function is ReLU, and the output layer dimension is [9], used to calculate the Q-values ​​for the nine actions. The network weights are initialized, and an experience replay buffer with a capacity of 10000 is set. The state, action, reward, and next state transition tuples from the training process are stored in the buffer. During training, each iteration consists of 2000 steps. 64 samples are randomly sampled from the buffer, and the target Q-value is calculated, expressed as:

[0070]

[0071] in, It is to optimize the target value. Is the agent in a state? Take action The maximum Q value at that time As a discount factor, For target network parameters;

[0072] Learning rate through Adam optimizer Minimize the loss function, expressed as:

[0073]

[0074] in, It is the current Q value;

[0075] The target network weights are updated using a soft update method, expressed as:

[0076]

[0077] Here, τ is a user-specified parameter, τ=0.001, which improves training stability.

[0078] The maximum number of training rounds is set to 500. Training will automatically terminate if the model converges early.

[0079] Step S5: Optimize fracturing parameters by inputting the initial geological parameters and initial fracturing operation parameters of the shale gas well to be optimized into the trained DQN model. The DQN model interacts with the LightGBM proxy model to continuously adjust the action strategy and finally outputs the optimal combination of proppant and fracturing fluid dosage per unit perforation length that maximizes Qg.

[0080] Among these options, on-site engineers can combine the cost of fracturing materials to select the most cost-effective solution from multiple potential optimal combinations.

[0081] According to an embodiment of the present invention, a shale gas well fracturing parameter optimization design system based on a depth Q network is provided.

[0082] The shale gas well fracturing parameter optimization design system based on deep Q-network according to an embodiment of the present invention includes: a data acquisition module, a surrogate model module, a DQN optimization module, a parameter output module, and a storage module, wherein;

[0083] The data acquisition module includes a data filtering unit and a data partitioning unit. The data filtering unit is responsible for collecting geological parameters, fracturing operation parameters, and production capacity data of shale gas wells, and removing outliers; the data partitioning unit divides the preprocessed data into training and testing sets at a 7:3 ratio to provide data support for model training.

[0084] The proxy model module constructs a production capacity prediction model based on the LightGBM algorithm. After training on a training set and validating on a test set, it simulates the real shale gas well fracturing environment. This module receives action commands from the DQN optimization module and outputs the corresponding production capacity prediction results, providing a basis for reward calculation.

[0085] The DQN optimization module is used to build the DQN model and includes a state encoding unit, an action generation unit, a reward calculation unit, a policy execution unit, and a network training unit. Specifically, the state encoding unit encodes fracturing capacity and parameter data into model-recognizable state vectors; the action generation unit generates nine parameter adjustment actions according to a preset action space; the reward calculation unit calculates the reward value based on the capacity change rate; the policy execution unit selects actions using an ε-greedy policy; and the network training unit completes model training and updates network weights through experience replay and gradient descent.

[0086] The parameter output module receives the optimal parameter combination output by the trained DQN model. Combined with the fracturing material cost database, it provides users with two options: optimal production capacity and optimal economic benefits, facilitating decision-making by on-site engineers.

[0087] The storage module is used to store raw data, preprocessed training and test sets, surrogate model parameters, DQN model weights, training process data, and final optimization results, supporting data backtracking and model iterative optimization.

[0088] Using the above technical solution, in the implementation process, taking well_1, a shale gas well with moderate geological conditions and low fracturing production capacity in the Weiyuan Block of the Sichuan Basin, as an example, the fracturing parameters are optimized using the method of this invention, as detailed below:

[0089] Data collection and preprocessing: Geological parameters of the well were collected, including vertical depth of 2400.00m, TOC content of 4.80%, porosity of 6.60%, thickness of high-quality reservoir of 6.00m, gas saturation of 77.00%, pressure coefficient of 1.45, and BMC of 86.40%. Initial fracturing parameters were also collected, including fracturing fluid consumption per unit perforation length of 21.06m³. 3 / m, proppant dosage per unit perforation length of 0.89t / m, and initial Qg are expressed as 10.30×10 4 m 3 / d, the data has been filtered and no outliers were found.

[0090] The proxy model is invoked using a pre-trained LightGBM proxy model with MSE=0.008 as the environment.

[0091] The DQN model was trained with the following parameters: discount factor 0.99, learning rate 0.0005, mini-batch size 64, experience replay buffer capacity 10000, target network soft update parameter 0.001, maximum training epochs 500, and the model converged around the 200th epoch.

[0092] The optimization results show that the optimal fracturing parameters are 29.04 m³ / m of fracturing fluid per unit perforation length and 1.81 t / m of proppant per unit perforation length. After optimization, Qg is increased to 11.95 × 10⁻⁶. 4 m 3 / d, an increase of 15.9%.

[0093] Specifically, such as Figures 4-6 As shown, its well_1 learning curve is as follows: Figure 4The figure shows the curve of the reward obtained by the agent in each round. Since the time step is set to 2000 steps per round, the reward in each round is the average reward obtained in each time step. During training, the total reward gradually increases and fluctuates within a certain range. In this case, the shape of the total reward curve indicates that the DQN agent can learn to choose actions with higher rewards from repeated interactions with the environment, and the training performance becomes stable over large rounds. The total reward of well_1 stabilizes around the 200th round, meaning that the model has basically learned a combination of variables to maximize the total reward of the system. The reason for the fluctuation of the total reward curve during later learning is that the ε-greedy policy maintains the probability of allowing the agent to explore the environment, which means that a certain degree of randomness in action selection is normal.

[0094] The initial Qg of well_1 is 10.30 × 10 4 m 3 / d, Figure 5 The graph shows the curve of Qg over time steps when the agent adopts different fracturing operation parameter variation strategies in different training rounds. Without any prior knowledge, the agent's initial strategy is extremely difficult to improve Qg, resulting in a relatively low reward. As the agent learns, Qg gradually increases, and the reward value increases accordingly.

[0095] Record the maximum Qg obtained in each round and the values ​​of the corresponding two variables, as shown in the curve. Figure 6 As shown. At the end of round 212, the agent achieved the largest Qg across all rounds, with a value of 11.95 × 10. 4 m 3 / d, an increase of 15.9% compared to the initial value. At this point, the values ​​of the two construction variables are 29.04m. 3 / m and 1.81t / m, a significant increase compared to the original values, with the proppant per unit perforation length more than double. Specifically, in Figure 6 The image is marked with a red line. The results indicate that the initial parameters for this well were inappropriate, as the formation potential was not fully realized. There is room for improvement in fracturing production capacity.

[0096] Meanwhile, it can be observed that during the learning process, strategies include continuously increasing the value of one variable while decreasing the value of another, focusing on increasing the value of one variable while reducing the growth range of the other, or simultaneously and significantly increasing the values ​​of both variables. The reason Qg does not continuously increase with the increase of variable values ​​is the limitation of reservoir resources. Due to the limited shale gas reserves in the formation, the optimization of the two variables in this paper can only improve the shale gas production capacity after fracturing to a certain extent, but it is impossible to increase the original gas reserves. Furthermore, since the two variables have different weights in their influence on Qg, a significant increase in the value of one variable has little impact on improving the gas production rate. Therefore, the agent will gradually learn to abandon this strategy and find a more balanced value for the two variables to improve Qg.

[0097] In addition, during another implementation process, taking well_2, a shale gas well in the Weiyuan block with poor geological conditions and extremely low fracturing production capacity, as an example, the details are as follows:

[0098] Initial parameters are expressed as follows: vertical depth 3500.00 m, TOC content 7.00%, porosity 8.60%, and initial fracturing fluid volume 33.75 m³. 3 / m, initial proppant dosage 1.71t / m, initial Qg 5.79×10 4 m 3 / d.

[0099] The model was trained and optimized, and converged in the 100th round, outputting the optimal parameters as fracturing fluid dosage of 35.70 m³. 3 / m, proppant dosage 1.76t / m, optimized Qg reaches 6.42×10 4 m 3 / d, an increase of 10.9%.

[0100] Specifically, such as Figures 7-9 As shown, well_2 has the lowest initial Qg, indicating poor geological conditions. Figure 7 The reward curve in the image shows that the well converges around round 100. From Figure 8 It is evident that Qg gradually increases and stabilizes at a certain value, indicating that the agent's learning process has been successful. Figure 9 It can be seen that during the training process, well_2 achieved the maximum Qg of 6.42 × 10⁻⁶ four times across all epochs. 4 m 3 / d, an increase of 10.9% compared to the initial value. The values ​​of the two corresponding fracturing operation parameters are different each time; in other words, the same fracturing capacity can be achieved by applying different parameter design schemes. However, in practical applications, engineers can choose a set of parameters that maximizes economic benefits for production based on the cost of the injected fluid and proppant. The optimized parameter values ​​selected in this study are 35.70 m³ / m and 1.76 t / m.

[0101] Well_2 is more expensive to develop than Well_1 due to higher usage of fracturing fluid and proppant, but its Qg (quantity, yield, and output) is lower. Although Well_2's Qg improved after optimization, the growth rate was not ideal, resulting in lower economic efficiency compared to Well_1. The fundamental reason for the difference in production capacity between the two wells is their different geological properties, or in other words, Well_1 has better inherent conditions than Well_2. This also demonstrates the importance of geological classification for shale gas wells. For wells with favorable geological conditions, increased investment can be achieved to obtain higher gas production, while for wells with poor geological conditions, reduced investment can be used to improve the input-output ratio and economic efficiency.

[0102] In addition, during another implementation process, taking well_3, a shale gas well in the Weiyuan block with good geological conditions and high fracturing production capacity, as an example, the details are as follows:

[0103] Initial parameters are expressed as follows: vertical depth 3050.00m, TOC content 5.50%, porosity 7.60%, and initial fracturing fluid volume 23.58m³. 3 / m, initial proppant dosage 2.39t / m, initial Qg 26.52×10 4 m 3 / d.

[0104] The model was trained and optimized, and it converged rapidly in the 50th round, with the output Qg corresponding to the optimal parameters reaching 30.99 × 10⁻⁶. 4 m 3 / d, an increase of 16.9%.

[0105] like Figures 10-12 As shown, Figure 10 The data shows that the well converged around the 50th cycle, which is much faster than the other two wells. Figure 12 The changing trend of Qg in the model demonstrates the agent's rapid learning ability. From... Figure 11 It can be seen that during the training process, well_3 achieved the maximum Qg30.99×10 twice in all rounds. 4 m 3 / d, an increase of 16.9% compared to the initial value. Furthermore, well_3 has the highest economic cost because the amount of fracturing fluid and proppant used is significantly higher than the other two wells. Therefore, for wells with favorable geological conditions like well_3, good fracturing results can be achieved by controlling the amount of fracturing material used within a certain range.

[0106] In summary, by utilizing the above-mentioned technical solutions of this invention, the present invention employs the LightGBM algorithm to construct a data-driven surrogate model to simulate the real fracturing environment. Compared with traditional numerical simulation methods, it does not rely on complex physical assumptions, can more accurately capture the nonlinear coupling relationship between geological parameters, fracturing parameters, and production capacity, and has lower modeling costs and faster response speed, providing efficient and reliable environmental support for the optimization process. The optimization model based on Deep Q-Network (DQN), by defining states, actions, rewards, and an ε-greedy strategy, combined with a variable step size search mechanism, experience replay, and soft update of the target network, effectively solves the problems of weak local search ability and slow convergence speed of traditional optimization methods, and achieves optimal proppant dosage per unit perforation length and fracturing efficiency. Multivariate synchronous optimization of fluid usage can quickly explore the globally optimal parameter combination. This method and system have strong versatility and adaptability, eliminating the need to adjust the core framework for different geological conditions. It can be adapted to the personalized optimization needs of various shale gas wells through a data-driven approach, breaking the dependence of traditional methods on specific reservoirs. At the same time, the system integrates full-process functions such as data processing, model training, parameter optimization, and result output, providing on-site engineers with intuitive and accurate guidance for fracturing parameter design, reducing errors caused by experience-based decision-making, and helping to achieve scientific and personalized fracturing construction plans. Ultimately, while improving fracturing effect and production capacity, it optimizes construction cost input, providing strong technical support for the economical and efficient development of shale gas resources.

[0107] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Those skilled in the art, upon considering the disclosure in the specification and embodiments, will readily conceive of other embodiments of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0108] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for optimizing fracturing parameters in shale gas wells based on deep Q-networks, characterized in that, Includes the following steps: Step S1: Collect geological parameters, fracturing operation parameters, and production capacity data of shale gas wells in advance; The geological parameters include: vertical depth, TOC content, porosity, thickness of high-quality reservoir, gas saturation, pressure coefficient, and BMC; the fracturing operation parameters include proppant usage per unit perforation length and fracturing fluid usage per unit perforation length; the production capacity data is represented by the production capacity index, indicating the maximum average daily gas production Qg in the first year; and the collected data is screened, outliers are removed, and then divided into training and testing sets. Step S2: Construct an agent model and simulate the environment, including calibrating the LightGBM algorithm to construct an agent model based on the training set and test set, and using the agent model as the environment for reinforcement learning to receive agent actions and provide corresponding states and rewards. Step S3, construct the DQN model, including the following steps: Define the state, including calibrating the state space as follows: ,in, For current fracturing capacity, The amount of fracturing fluid used per unit perforation length. Propionage amount per unit perforation length; Define the action, including the action variables. and Each variable includes three change strategies: increase, decrease, and remain unchanged, forming nine action combinations. The actions are adjusted using a variable step size method, with the step size based on... The design ensures that the parameter values ​​do not exceed the preset upper and lower limits after the action is executed. Define the reward, including the reward value being calculated based on the rate of change between the optimized Qg and the initial Qg. A positive rate of change results in a positive reward, with a higher rate of change leading to a higher reward value. A negative rate of change results in a negative reward, and a zero rate of change results in a zero reward value. The strategy is defined, and an ε-greedy strategy is adopted for action selection. The ε value gradually decreases from the initial value of 1 to 0.01 through exponential decay. The balance model is explored and utilized. Step S4: Perform model training, initialize the weights of the Q network and target network of DQN, set up an experience replay buffer, and convert the training data into a transition tuple of state, action, reward and next state and store it in the buffer; during iterative training, randomly sample small batches of data from the buffer, calculate the target Q value and minimize the loss function through gradient descent, and update the target network weights using a soft update method until the model converges. Step S5: Optimize fracturing parameters by inputting the initial parameters of the shale gas well to be optimized into the trained DQN model. The model then interacts with the surrogate model to output the optimal proppant dosage and fracturing fluid dosage per unit perforation length.

2. The method for optimizing fracturing parameters of shale gas wells based on depth Q-networks according to claim 1, characterized in that, The variable step size adjustment of the action in step S3 satisfies , Furthermore, when the parameters are adjusted beyond this range, they are automatically truncated to the corresponding boundary value.

3. The shale gas well fracturing parameter optimization design method based on depth Q-network according to claim 1, characterized in that, The calculation method for the reward value in step S3 is expressed as follows: ； in, This refers to the fracturing capacity after the action is executed; This refers to the fracturing capacity obtained after implementing the action strategy. This represents the initial gas production of each well.

4. The shale gas well fracturing parameter optimization design method based on depth Q-network according to claim 1, characterized in that, The network structure of the DQN model includes: an input layer, two hidden layers, and an output layer. The input layer has a dimension of [1,3], the hidden layer has 64 nodes, the activation function is ReLU, and the output layer has a dimension of [9], corresponding to the Q values ​​of 9 actions. The training parameters are set as follows: discount factor γ=0.99, learning rate α=0.0005, mini-batch size=64, experience replay buffer size=10000, target network soft update parameter τ=0.001, maximum number of training rounds=500, and maximum number of time steps per round=2000.

5. A shale gas well fracturing parameter optimization design system based on a depth Q-network, used in the system of the shale gas well fracturing parameter optimization design method based on a depth Q-network as described in any one of claims 1-4, characterized in that, include: The data acquisition module is used to collect geological parameters, fracturing operation parameters, and production capacity data of shale gas wells, and to perform preprocessing. The proxy model module is used to build a production capacity prediction model based on the LightGBM algorithm, simulate the real environment of shale gas well fracturing, receive action inputs and output the corresponding production capacity prediction results. The DQN optimization module is used to build the DQN module, including a state encoding unit, action generation unit, reward calculation unit, policy execution unit, and network training unit, which are used to implement state definition, action generation, reward feedback, policy selection, and model training. The parameter output module is used to combine the trained DQN model with the initial parameters of the well to be optimized and output the optimal combination of fracturing parameters. The storage module is used to store raw data, training sets, test sets, surrogate model parameters, DQN model parameters, and optimization results.

6. The shale gas well fracturing parameter optimization design system based on deep Q-networks according to claim 5, characterized in that, The data acquisition module includes a data filtering unit and a data partitioning unit. The data filtering unit is used to remove abnormal data, and the data partitioning unit divides the training set and the test set in a 7:3 ratio.

7. The shale gas well fracturing parameter optimization design system based on deep Q-networks according to claim 5, characterized in that, The network training unit in the DQN optimization module uses the Adam optimizer to minimize the loss function and eliminates sample correlation through an empirical replay mechanism to improve training stability.

Images