An oilfield production system decision optimization method based on reinforcement learning

Abstract

The application discloses a kind of oilfield production system decision optimization methods based on forced learning, comprising: collecting the dynamic production data of oilfield production site, establishing the data cube of reservoir production optimization;The preset machine learning model is trained based on data cube, and the agent model of injection-production system based on forced learning for predicting oil production according to the dynamic production data of production site is obtained;Construct the evaluation function for gas injection reservoir production optimization;In the production optimization process, the forced constraint model based on input parameter and boundary constraint condition are established;With constraint model and boundary constraint condition as constraint, based on the agent model of injection-production system of reservoir, with evaluation function as optimization direction, find reservoir production optimization scheme, obtain optimal production scheme.The present application can solve the technical problems of limitations in calculation accuracy and time of traditional physical-based methods.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of oilfield development, and particularly relates to an oilfield production system decision optimization method based on forced learning. BACKGROUND

[0002] Production optimization is a key part of reservoir closed-loop management, and can directly affect the sustainability, efficiency, safety and economy of reservoir development. How to increase oil production as much as possible under the conditions of ensuring production safety and environmental protection by adjusting the injection mode of injection wells has been the focus of oilfield sites.

[0003] The existing reservoir production optimization method is mainly based on a complex geological model and a multiphase flow mechanism to construct a reservoir numerical simulator. The yield of the numerical simulator is matched with the actual yield through a large number of parameter adjustments, and on this basis, the injection parameters are adjusted to predict future yield, and then production optimization is completed. The reservoir numerical simulator is a reservoir model developed in dependence on geological interpretation and current understanding of the general physical laws of fluid flow in porous media, and contains many assumptions, simplifications, experiences and preconceived concepts. Due to the complexity of reservoir geology, the uncertainty of multiphase flow and the limitations of exploration technology, the reservoir numerical simulator is difficult to completely simulate the real reservoir, and often cannot accurately restore the real situation. Meanwhile, the simulator involves iterative calculation of tens of thousands of grids, and it takes a lot of time to establish the simulator and subsequent production optimization. The iterative error based on tens of thousands of grids cannot be ignored.

[0004] Therefore, the traditional physical-based method has limitations in calculation accuracy and time. SUMMARY

[0005] The present application provides an oilfield production system decision optimization method based on forced learning to solve the technical problem of limitations in calculation accuracy and time of the traditional physical-based method.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] An oilfield production system decision optimization method based on forced learning comprises:

[0008] Collecting field dynamic production data to establish a data cube for reservoir production optimization;

[0009] Training a preset machine learning model based on the data cube to obtain an oil reservoir injection-production system agent model based on forced learning for predicting oil production according to field dynamic production data;

[0010] Constructing an evaluation function for gas injection reservoir production optimization;

[0011] establishing a forced constraint model based on input parameters and boundary constraint conditions in the production optimization process;

[0012] searching for an oil reservoir production optimization scheme based on the reservoir injection-production system proxy model and the evaluation function as the optimization direction, and obtaining an optimal production scheme.

[0013] Further, after obtaining the reservoir injection-production system proxy model, the method further comprises:

[0014] quantitatively characterizing the connectivity between the injection wells and the production wells based on the reservoir injection-production system proxy model.

[0015] Further, the quantitatively characterizing the connectivity between the injection wells and the production wells comprises:

[0016] First, the importance of each input variable of the reservoir injection-production system proxy model to the model is calculated, that is, the order of the data of a variable in the data set is randomly changed, other variables are kept unchanged, a new training set is constructed, the model is trained by using the new data set, and the error of the corresponding prediction result is calculated;

[0017] Then, the difference between the prediction error of the model trained based on the new data set and the prediction error of the model trained based on the original data set is calculated as the sensitivity of the corresponding variable in the data set to the model;

[0018] The connectivity between the injection wells and the production wells is calculated by the following formula:

[0019]

[0020] wherein, L j represents the connectivity factor between the jth injection well and the production well, E(IV j ) represents the sensitivity of the gas injection rate of the jth injection well to the model, E(IP j ) represents the sensitivity of the injection pressure of the jth injection well to the model, and E(IM j ) represents the sensitivity of the injection mode of the jth injection well to the model.

[0021] Further, the field dynamic production data is collected to establish a data cube for the oil reservoir production optimization, comprising:

[0022] The dynamic production data of the oil reservoir production wells and the surrounding adjacent injection wells are collected; wherein the dynamic production data of the production wells comprises oil production rate, gas production rate, gas-oil ratio, well opening and closing state, and choke size; and the dynamic production data of the injection wells comprises gas injection rate, injection pressure, and injection mode.

[0023] The dynamic production data collected is used to establish a data cube for reservoir production optimization.

[0024] Further, a preset machine learning model is trained based on the data cube to obtain an oil reservoir injection-production system agent model based on forced learning for predicting oil production according to field dynamic production data, including:

[0025] A data set required for model training is extracted from the data cube, and the data set is divided into a training set and a test set according to a ratio of 9:1 to obtain the training set and the test set; wherein the opening and closing states of the production wells and the choke size in the data set, and the gas injection amount, the injection pressure and the injection mode of the injection well are used as inputs of the model, and the oil production, the gas production and the gas-oil ratio of the production well in the data set are used as outputs of the model;

[0026] A machine learning model based on a deep neural network is constructed; wherein the number of hidden layers of the machine learning model is 3, and the number of neurons in each layer is 60;

[0027] The machine learning model is trained by using the training set, and the trained machine learning model is tested by using the test set to obtain an oil reservoir injection-production system agent model based on forced learning.

[0028] Further, the expression of the evaluation function is:

[0029]

[0030] wherein O(x) is the evaluation function, Qo (f(D,W,θ)) is the oil production, GOR (f(D,W,θ)) is the gas-oil ratio.

[0031] Further, the establishment of the forced constraint model based on the input parameters and the boundary constraint condition includes:

[0032] A physical constraint model between the injection amount and the injection pressure is established, that is, the injection amount is used as the input, the injection pressure is used as the output, and an intelligent constraint model S for predicting the injection pressure by using the injection amount is constructed by the machine learning model, therefore, the relationship between the injection amount and the injection pressure is represented as:

[0033] IP_pred w,t =S(IV w,t ,W,θ)

[0034] wherein IP_pred w,t is a predicted value of the injection pressure of the wth injection well at t time, IV w,tThe injection amount of the wth injection well at time t, W is the weight between neurons of the machine learning model, and θ is the threshold value in the neuron. In addition, for the input variables corresponding to the injection well, the following boundary constraint condition is set:

[0035] IV w,t ∈{a*Ave(IV),b*Max(IV)},IP w,t ∈{Min(IP),Max(IP)}

[0036] Wherein, a and b are constraint factors, the value range is a∈(0, 1), b∈(0.5, 2);IP w,t The injection pressure of the wth injection well at time t; Ave(IV) is the average value of the injection amount, Max(IV) is the maximum value of the injection amount, Min(IP) is the minimum value of the injection pressure; Max(IP) is the maximum value of the injection pressure;

[0037] In addition, the boundary constraint condition of the choke size in the production measure is expressed as:

[0038] CS t ∈{0,AVE(CS)+c*(MAX(CS)-MIN(CS))}

[0039] Wherein, CS t The choke size of the production well at time t; Ave(CS) is the average value of the choke size; c is the floating coefficient, Max(CS) is the maximum value of the choke size, and Min(CS) is the minimum value of the choke size.

[0040] Further, based on the constraint model and the boundary constraint condition, the oil reservoir injection-production system agent model is used to find the oil reservoir production optimization scheme in the optimization direction of the evaluation function, and the optimal production scheme is obtained, including:

[0041] An injection-production optimization model based on a particle swarm algorithm is established, and the input of the injection-production optimization model is the injection amount, injection pressure and injection mode of each injection well, and the output is the oil production and gas-oil ratio of the target well;

[0042] The relationship model between the injection amount and the injection pressure and the boundary constraint condition are used as constraints, and the oil reservoir injection-production system agent model is used as the basis to find the oil reservoir production optimization scheme in the optimization direction of the evaluation function;

[0043] Different injection schemes are optimized, and the oil production and gas-oil ratio of the production well under the corresponding conditions are calculated based on the optimized injection scheme using the injection-production system agent model, and the Pareto frontier for the oil reservoir production optimization problem is found, and based on the Pareto frontier, the scheme is selected and optimized according to different requirements.

[0044] The technical scheme provided by the present application has at least the following beneficial effects:

[0045] 1. The present application designs a calculation framework for decision optimization of an oilfield production system based on forced learning, first uses injection-production data to establish an injection-production system agent model, can accurately predict dynamic production data of a production well according to injection parameters, completes historical fitting of field monitoring data, and based on the model, constructs an intelligent production optimization model, and finds an optimal injection mode to complete production optimization in the oilfield development process.

[0046] 2. The present application proposes an agent model of an injection-production system, uses injection variables of an injection well and production measures to accurately predict production well yield and gas-oil ratio and other parameters based on a machine learning method, without complex geological parameters and geological models.

[0047] 3. The present application introduces a method for evaluating interwell connectivity in a forced learning manner, analyzes the importance of input variables based on the injection-production agent model, and defines interwell connectivity by using the influence degree of different injection wells on production wells.

[0048] 4. The present application establishes an injection-production optimization method for an oil reservoir, the input is injection volume, injection pressure and injection mode of each injection well, the output includes oil production and gas-oil ratio of a target well, the relationship model between injection volume and injection pressure and boundary conditions are used as constraints, the injection-production system agent model is used as a basis, and the evaluation function is used as an optimization direction to find an oil reservoir production optimization scheme, and the optimal solution under different targets is analyzed according to a Pareto front solution set. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0050] Figure 1 is an execution flow schematic diagram of the oilfield production system decision optimization method based on forced learning provided by the embodiment of the present application;

[0051] Figure 2 is an injection-production agent model daily oil production prediction result schematic diagram provided by the embodiment of the present application;

[0052] Figure 3 is an input parameter importance analysis and interwell connectivity schematic diagram provided by the embodiment of the present application;

[0053] Figure 4is a production development optimization direction diagram for gas injection reservoirs provided by the embodiment of the present application;

[0054] Figure 5 is a production optimization pareto frontier solution set diagram for reservoirs provided by the embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the present application more clear, the embodiment of the present application will be further described in detail below with reference to the drawings.

[0056] The embodiment provides a production system decision optimization method for oilfields based on forced learning, which is completely based on real monitoring data on site, considers physical and operation restrictions on the basis of an integrated machine learning method, establishes an injection-production system agent model and a production optimization framework, solves injection optimization problems in the development process of oilfields, provides more and more effective choices and guidance for decision makers, and achieves the purpose of cost reduction and benefit increase.

[0057] The production system decision optimization method for oilfields in the embodiment first establishes an injection-production system agent model for reservoirs based on machine learning based on real monitoring data provided on site, replaces a complex traditional numerical simulator, and does not need to consider complex geological structures and multiphase percolation mechanisms, then determines the sensitivity of input parameters to the model based on the agent model, and further provides an interwell connectivity evaluation method. Then, an evaluation function for production optimization of gas injection reservoirs is constructed, the relationship model between injection rate and injection pressure and boundary conditions are used as constraints, the injection-production system agent model is used as the basis, and the evaluation function is used as the optimization direction to find a production optimization scheme for reservoirs, and an intelligent injection-production optimization model is established; a technical idea diagram is shown as Figure 1 The embodiment includes the following steps:

[0058] S1, collecting on-site dynamic production data, and establishing a data cube for reservoir production optimization;

[0059] S2, training a preset machine learning model based on the established data cube, and obtaining an injection-production system agent model for reservoirs based on forced learning, which is used for predicting oil production according to on-site dynamic production data;

[0060] S3, quantitatively characterizing the connectivity between injection wells and production wells based on the injection-production system agent model for reservoirs;

[0061] S4, constructing an evaluation function for production optimization of gas injection reservoirs;

[0062] S5, establishing a forced constraint model based on input parameters and boundary constraint conditions in the production optimization process;

[0063] S6, based on the reservoir injection-production system agent model, and taking the constraint model and the boundary constraint condition as constraints, and taking the evaluation function as an optimization direction, an optimal production scheme is found, and an optimal production scheme is obtained.

[0064] The following will be described in detail in combination with Figures 2 to 5 each of the above steps.

[0065] First, collect the dynamic production data on site to establish a data cube for reservoir production optimization; first, collect the dynamic production data of the reservoir production well (target well) and the surrounding adjacent injection well (gas injection well), wherein the data of the production well includes: oil production (Qo), gas production (Qg), gas-oil ratio (GOR), well opening and closing state (SI), choke size (CS); the data of the injection well includes: injection volume (IV), injection pressure (IP), injection mode (IM). The well opening and closing state is represented by 0 / 1, and the injection mode is represented by 001 / 010 / 100, which represents stop injection / gas injection / water injection. This experiment takes a five-spot well pattern as an example, that is, one production well and four surrounding injection wells, and the data set of the production well can be represented as A = [Qo t , Qg t , GOR t , SI t , CS t ], and the data set of the injection well can be represented as B = [IV w,t , IP w,t , IM w,t ], wherein w and t are well numbers (respectively #1, #2, #3, #4) and development cycle time (4500 days), forming a data cube for reservoir production optimization.

[0066] Second, based on the established data cube, a preset machine learning model is trained to obtain a reservoir injection-production system agent model based on forced learning for predicting oil production according to the dynamic production data on site; the data set D = [A, B] required for model training is extracted from the constructed data cube, and the total number of samples is 4500. The data set is divided into a training set and a test set according to a ratio of 9:1, to obtain the training set D train = [A 1-4000 , B 1-4000 ] and the test set D test = [A 4001 -4500 , B 4001-4500 ], wherein 1-4000 represents a time range from the 1st day to the 4000th day; at the same time, both for the training set and the test set, the input and the output need to be defined for the training and verification of the model, that is, the training set is D train = [X train , Y train ] and the test set is Dtest = [X test , Y test ], wherein X and Y represent input and output parameters respectively, X = [SI t , CS t , IV w,t , IP w,t , IM w,t ] and Y = [Qo t , Qg t , GOR t ]. A machine learning model based on deep neural network is constructed, wherein the number of hidden layers is 3 and the number of neurons in each layer is 60, which can be expressed as F(D; W, θ); wherein W represents the connection weight between neurons, θ represents the threshold value in each neuron, and D represents the data set required by the machine learning model. The core of the machine learning model is to continuously adjust the weight and threshold value in the model by using the input and output parameters of the data set through the back propagation algorithm, so that the prediction result can continuously approach the true value. Finally, the oil reservoir injection-production system agent model F based on forced learning can be obtained, and the prediction results of the training set and the test set can be expressed as:

[0067] Y_pred train = [Qo_pred t_train , Qg_pred t_train , GOR_pred t_train ]

[0068] Y_pred test = [Qo_pred t_test , Qg_pred t_test , GOR_pred t_test ]

[0069] As shown in Figure 2 , the prediction result of the injection-production agent model for daily oil production is shown, wherein the horizontal axis is the actual daily oil production and the vertical axis is the predicted daily oil production. It can be seen that the relative error is basically within 15%, and both the test set and the training set show very good prediction performance.

[0070] Thirdly, based on the injection-production system agent model, the connectivity between injection wells and production wells is quantitatively characterized; and a well connectivity evaluation method based on the injection-production system agent model is formed. The injection-production system agent model provides a forced learning way to perform history matching on true field monitoring data. Comparative analysis based on the prediction results proves that the model has learned the nonlinear characteristic relationship between the data, and has high accuracy and reliability. The input data of the model includes the injection rate, injection pressure and injection mode of four injection wells, and the production measures including the choke size and the well switching state of one production well, totaling 14 variables. First, the importance of each variable to the model is calculated, that is, the order of the data of a certain variable in the input set is randomly changed, and the other variables remain unchanged, thereby constructing a new training set, such as D_SI=[R(SI t ),CS t ,IV w,t ,IP w,t ,IM w,t ], R is a random function, and D_SI represents the data set in which the well switching state (SI) is changed. The machine learning model is trained using the new data set, and the error of the corresponding prediction result is calculated. The difference between the error and the prediction error of the machine learning model based on the original data set can be represented as:

[0071] E(SI)=MAE[F(D_SI,W,θ)--MAE[F(D,W,θ)

[0072] Wherein, E(SI) represents the sensitivity of the parameter well switching state, MAE is the mean absolute error function, and F is the injection-production system agent model. Based on the above method, the sensitivity of each parameter to the model can be calculated, and the connectivity between the injection wells and the production wells can be represented as:

[0073]

[0074] Wherein, L #01 is the connectivity factor between the first injection well and the production well, and there are four injection wells in the five-spot well pattern, so the maximum value of i is 4. The connectivity between the injection wells and the production wells can be quantitatively characterized by using the above method. As shown in Figure 3 , for a group of five-spot well patterns, the sensitivity of the input parameters to the model and the connectivity between the injection wells and the production wells are calculated.

[0075] Fourthly, construct the evaluation function for production optimization of gas injection reservoirs; high gas-oil ratio (GOR) in gas injection development reservoirs will have safety hazards, some corrosive acid gases will damage pipelines and other infrastructure, and are prone to cause gas channeling and other phenomena, which is not conducive to the sustainable development of the reservoir. Therefore, during the development of oil and gas fields, it is often desirable to increase oil production as much as possible under the condition of low gas-oil ratio, and to increase economic benefits under the premise of sustainable development. Therefore, one goal of this project is to increase oil production, which can be represented as Obj_01 = MAX (Qo), and the other goal is to reduce the gas-oil ratio, which can be represented as Obj_02 = MIN (GOR). In order to meet the above two goals, the evaluation function for production optimization of gas injection reservoirs is constructed as follows:

[0076]

[0077] Where O(x) is the evaluation function, Qo is the oil production, GOR is the gas-oil ratio, and exp{} is the exponential function, which can be used to amplify the degree of change in the ratio.

[0078] Based on the evaluation function, the optimization direction of the optimization problem can be determined. As shown in the following figure, the optimization direction of the reservoir production optimization based on the evaluation function under the two objectives is shown in detail, which facilitates the search for the optimal optimization solution set. Figure 4

[0079] Fifthly, establish a forced constraint model based on input parameters and boundary constraint conditions during production optimization; in actual production, the injection rate and injection pressure of each injection well have an implicit relationship, and the larger the injection rate, the higher the injection pressure will be. However, during optimization, the injection rate and injection pressure will be randomly selected as two independent variables, so the injection rate and injection pressure of different injection wells obtained under this optimization mechanism obviously do not meet the actual production requirements, and therefore a physical constraint model between injection rate and injection pressure needs to be established, i.e. taking injection rate as input and injection pressure as output, and an intelligent constraint model S can be constructed to predict injection pressure using injection rate through machine learning model. Therefore, the relationship between injection rate and injection pressure can be represented as:

[0080] IP_pred w,t = S(IV w,t ,W, θ)

[0081] Where IP_pred w,t is the predicted value of the injection pressure of the wth injection well at time t, w is the well number, t is the production time, IV w,t ​The injection amount of the wth injection well at time t, W is the weight between neurons of the machine learning model, and θ is the threshold value in the neuron. In addition, for the input variables corresponding to the injection well in the optimization model, boundary constraint conditions are still needed, which can be expressed as:

[0082] IV w,t ∈{a*Ave(IV),b*Max(IV)},IP w,t ∈{Min(IP),Max(IP)}

[0083] Wherein, a and b are constraint factors, the value range is a∈(0, 1), b∈(0.5, 2); IP w,t is the injection pressure of the wth injection well at time t; Ave(IV) is the average value of the injection amount, Max(IV) is the maximum value of the injection amount, Min(IP) is the minimum value of the injection pressure; Max(IP) is the maximum value of the injection pressure.

[0084] In addition, the constraint condition of the choke size in the production measure can be expressed as:

[0085] CS t ∈{0,AVE(CS)+c*(MAX(CS)-MIN(CS))}

[0086] Wherein, CS t is the choke size of the production well at time t; AVE(CS) is the average value of the choke size; c is the floating coefficient, MAX(CS) is the maximum value of the choke size, and MIN(CS) is the minimum value of the choke size.

[0087] In the sixth step, the constraint model and the boundary constraint condition are used as constraints, the injection-production optimization model based on the reservoir injection-production system agent model is used to find the optimal production scheme, and the optimal production scheme is obtained. The input of the injection-production optimization model based on the particle swarm algorithm includes the injection amount, injection pressure and injection mode of each injection well, and the output result includes the oil production and gas-oil ratio of the target well. The relationship model between the injection amount and the injection pressure and the boundary condition are used as constraints, the injection-production system agent model is used as the basis, and the evaluation function is used as the optimization direction to find the optimal production scheme of the reservoir. Further, different injection schemes can be optimized, and the injection-production agent model is used to calculate the oil production and gas-oil ratio of the production well under the corresponding conditions based on the optimized injection mode, and the Pareto frontier for the reservoir production optimization problem is found. The Pareto frontier is the optimal solution set of the optimization problem, and the scheme selection and optimization can be performed according to different development requirements. For example Figure 5The prediction effect of different injection modes after optimization by the intelligent injection optimization model is shown, the horizontal axis is oil production, the vertical axis is gas oil ratio, the pareto front solution set can be obtained according to the optimization direction, the oil production obtained according to the injection mode of injection mode 1 is the largest, the optimal gas oil ratio can be obtained according to the injection mode of injection mode 2, and the injection scheme can be adjusted according to different needs of the field by using the solution set.

[0088] In summary, the embodiment innovatively proposes an oilfield production system decision optimization method based on forced learning, which does not need to consider complex geological models and flow mechanisms, and establishes an injection-production system agent model based on real monitoring data on site, can replace the traditional numerical simulator, and the optimization algorithm based on forced learning can quickly and accurately determine the optimal injection mode, has high prediction accuracy and strong adaptability, the prediction speed can reach seconds, and the complex oilfield production optimization problem can be well solved.

[0089] In addition, it should be noted that the present application can be provided as a method, device or computer program product. Therefore, the embodiments of the present application can adopt a completely hardware embodiment, a completely software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer usable storage media containing computer usable program codes.

[0090] The embodiments of the present application are described with reference to flowcharts and / or block diagrams according to the method, terminal device (system) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The means for implementing the functions specified in one flow or multiple flows and / or blocks.

[0091] These computer program instructions can also be stored in a computer readable storage medium which can guide the computer or other programmable data processing terminal device to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1The computer program instructions can also be loaded onto a computer or other programmable data processing terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal device provide steps for implementing the flow Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal device provide steps for implementing the flow Figure 1 The computer program instructions can also be loaded onto a computer or other programmable data processing terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal device provide steps for implementing the flow

[0092] It should also be noted that, in the present document, the terms "comprising", "comprising" or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or terminal device that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or terminal device. Without further limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or terminal device that includes the element.

[0093] Finally, it should be noted that the above description is of the preferred embodiments of the present application, and it should be pointed out that, although the preferred embodiments of the present application have been described, for those skilled in the art, once the basic creative concept of the present application is known, several improvements and refinements can be made without departing from the principles of the present application, and these improvements and refinements should also be considered as falling within the scope of protection of the present application. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

Claims

1. A method for decision optimization of an oilfield production system based on reinforcement learning, the method comprising: The method comprises the following steps: Collecting dynamic production data of a production well and a surrounding adjacent injection well; wherein, the dynamic production data of the production well comprises oil production, gas production, gas-oil ratio, well opening and closing state and choke size; the dynamic production data of the injection well comprises gas injection volume, injection pressure and injection mode; a data cube for production optimization of the oil reservoir is established by using the collected dynamic production data; Extracting a data set required for model training from the data cube, dividing the data set into a training set and a test set according to a ratio of 9:1, obtaining the training set and the test set; wherein, the well opening and closing state and the choke size of the production well, and the gas injection volume, the injection pressure and the injection mode of the injection well in the data set are used as inputs of the model, and the oil production, the gas production and the gas-oil ratio of the production well in the data set are used as outputs of the model; a machine learning model based on a deep neural network is constructed; wherein, the number of hidden layers of the machine learning model is 3, and the number of neurons in each layer is 60; the machine learning model is trained by using the training set, and the trained machine learning model is tested by using the test set, so as to obtain an injection-production system proxy model of the oil reservoir based on forced learning; An evaluation function O(x) for gas injection reservoir production optimization is constructed. where Qo (f(D,W,θ)) is the oil production, GO (f(D,W,θ)) is the gas-oil ratio; In the production optimization process, a forced constraint model based on input parameters and a boundary constraint condition are established; Based on the injection-production system proxy model of the oil reservoir, an optimal production scheme is searched by taking the constraint model and the boundary constraint condition as constraints and taking the evaluation function as an optimization direction, so as to obtain the optimal production scheme.

2. The forced learning based oilfield production system decision optimization method of claim 1, wherein, After obtaining the injection-production system proxy model of the oil reservoir, the method further comprises: Based on the injection-production system proxy model of the oil reservoir, the connectivity between the injection well and the production well is quantitatively characterized.

3. The forced learning based oilfield production system decision optimization method of claim 2, wherein, The quantitatively characterized connectivity between the injection well and the production well comprises: Firstly, the importance of each input variable of the injection-production system proxy model of the oil reservoir to the model is calculated, that is, the order of the data of a certain variable in the data set is randomly changed, other variables are kept unchanged, a new training set is constructed, the model is trained by using the new data set, and the error of the corresponding prediction result is calculated; Then, the difference between the prediction error of the model trained based on the new data set and the prediction error of the model trained based on the original data set is calculated as the sensitivity of the corresponding variable in the data set to the model; The connectivity between the injection well and the production well is calculated by the following formula: where L j represents the connectivity factor between the jth injection well and the production well, E(IV j ) represents the sensitivity of the gas injection rate of the jth injection well to the model, E(IP j ) represents the sensitivity of the injection pressure of the jth injection well to the model, E(IM j ) represents the sensitivity of the injection mode of the jth injection well to the model.

4. The forced learning based oilfield production system decision optimization method of claim 1, wherein, The establishment of the forced constraint model based on input parameters and the boundary constraint condition comprises: A physical constraint model between the injection volume and the injection pressure is established, that is, the injection volume is taken as an input, the injection pressure is taken as an output, the model is constructed by using the machine learning model, and an intelligent constraint model S for predicting the injection pressure by using the injection volume is constructed, therefore, the relationship between the injection volume and the injection pressure is represented as: IP_pred w,t = S(IV w,t , W, θ) wherein, IP_pred w,t is the injection pressure prediction value of the wth injection well at time t, IV w,t is the injection rate of the wth injection well at time t, W is the weight between neurons of the machine learning model, and θ is the threshold value in the neuron; in addition, for the input variables corresponding to the injection well, the following boundary constraint conditions are set: IV w,t ∈{a*Ave(IV),b*Max(IV)},IP w,t ∈{Min(IP),Max(IP)} wherein a and b are constraint factors, a ∈ (0, 1), b ∈ (0.5, 2); IP w,t is the injection pressure of the wth injection well at time t; Ave(IV) is the average of the injection volume, Max(IV) is the maximum of the injection volume, Min(IP) is the minimum of the injection pressure, and Max(IP) is the maximum of the injection pressure. In addition, the boundary constraint condition of the choke size in the production measure is represented as: CS t ∈ {0, AVE(CS) + c * (MAX(C) - MIN(C))} where CS t is the choke size at time t for the production well; AVE(C) is the average choke size; c is a float factor, MAX(CS) is the maximum choke size, and MIN(CS) is the minimum choke size.

5. The forced learning based oilfield production system decision optimization method of claim 4, wherein, Based on the injection-production system proxy model of the oil reservoir, an optimal production scheme is searched by taking the constraint model and the boundary constraint condition as constraints and taking the evaluation function as an optimization direction, so as to obtain the optimal production scheme, which comprises: An injection-production optimization model based on a particle swarm algorithm is established, the input of the injection-production optimization model is the injection volume, the injection pressure and the injection mode of each injection well, and the output is the oil production and the gas-oil ratio of the target well; The relationship model between injection volume and injection pressure and the boundary constraint condition are taken as constraints, the reservoir injection-production system agent model is taken as a basis, and an evaluation function is taken as an optimization direction to find a reservoir production optimization scheme; Different injection schemes are optimized, the oil production and the gas-oil ratio of a production well under corresponding conditions are calculated based on the optimized injection schemes and the injection-production system agent model, and a Pareto front for the reservoir production optimization problem is found based on the oil production and the gas-oil ratio, and a scheme is selected and optimized according to different requirements based on the Pareto front.

Images