A prediction model and its prediction method for the optimal injection-production plan of oil reservoirs using a deep reinforcement learning model

Abstract

The present invention provides a prediction model and a prediction method for an optimal injection-production plan of a reservoir using a deep reinforcement learning model, a simulated waterflood reservoir model for fitting historical data of an actual production environment so as to simulate the actual production environment for an agent model to learn; a waterflood injection-production plan optimization model for interacting with the simulated waterflood reservoir model and the agent model, thereby achieving the purpose of optimizing the agent model; an agent model for designing a corresponding injection-production plan according to the current production state and continuously optimizing according to the reward value feedback by the waterflood injection-production plan optimization model, so as to output an optimal injection-production plan; and an injection-production index migration optimization plan model for improving the accuracy of the prediction result of the simulated waterflood reservoir model. The present invention uses the trained model to provide an optimal injection-production plan for future oilfield production, thereby achieving maximum oil production.

Description

Technical Field

[0001] The present invention relates to the technical field of oil reservoir production and injection allocation, and more specifically to a prediction model for the optimal injection and production plan of an oil reservoir using a deep reinforcement learning model and a prediction method thereof. Background Art

[0002] The optimal injection and production plan model of an oil reservoir is divided into three main parts: "hidden environment", "explicit environment", and "agent". Among them, the hidden environment is a simulation water drive oil reservoir model, which inputs data such as injection volume and wellhead pressure. The intermediate hidden layer can output the injection splitting of injection wells and the lateral splitting of each production well, which can reflect the underground situation, and finally output the daily liquid production and daily oil production data of each production well. Since the hidden environment can only fit the underground situation and cannot score the injection and production plan of the agent, the explicit environment is used to interact with the hidden environment and the agent. The input of the explicit environment is the injection and production plan generated by the agent, and the output of the explicit environment is the oil production, which is used as the reward for the injection and production plan generated by the agent. As the name implies, the agent is the model that finally provides the injection and production plan. Its input is the output of the intermediate hidden layer of the hidden environment, and the output is the specific injection volume and the liquid production corresponding to the production well. And it uses the oil production output by the explicit environment as a reward to continuously optimize the agent using reinforcement learning, so that it can make the optimal decision in different environments. Summary of the Invention

[0003] The present invention overcomes the deficiencies in the prior art and provides a prediction model for the optimal injection and production plan of an oil reservoir using a deep reinforcement learning model and a prediction method thereof. The present invention uses the trained model to provide the optimal injection and production plan for future oilfield production, so as to maximize oil production.

[0004] The object of the present invention is achieved by the following technical solutions.

[0005] A prediction model for the optimal injection and production plan of an oil reservoir using a deep reinforcement learning model includes a simulation water drive oil reservoir model, a water drive injection and production plan optimization model, an agent model, and an injection and production index migration optimization plan model.

[0006] The simulation water drive oil reservoir model is used to fit the historical data of the real production environment, so as to simulate the actual production environment for the agent model to learn.

[0007] The water drive injection and production plan optimization model is used to interact with the simulation water drive oil reservoir model and the agent model, so as to achieve the purpose of optimizing the agent model.

[0008] The agent model is used to design the corresponding injection and production plan according to the current production state, and continuously optimize according to the reward value feedback by the water drive injection and production plan optimization model, so as to output the optimal injection and production plan.

[0009] The injection-production index migration optimization scheme model is used to improve the accuracy of the prediction results of the simulated waterflood reservoir model.

[0010] The simulated waterflood reservoir model includes three sub-models: VerticalNet, HorizonNet, and LiftNet;

[0011] The input data of VerticalNet is the daily injection volume and wellhead pressure input to the simulated waterflood reservoir model. The output data of VerrticalNet are the vertical splitting of the injection well, the pressure of the injection well in each sub-layer, the formation pressure around the injection well in each sub-layer, and the total oil displacement in each sub-layer;

[0012] The input data of HorizonNet are the output data of VerticalNet. The output data of HorizonNet are the horizontal splitting of each sub-layer, the pressure of each production well in different sub-layers, the formation pressure around different sub-layers, the oil production and oil displacement of each production well in different sub-layers;

[0013] The input data of LiftNet are the output data of HorizonNet. The output data of LiftNet are the output data of the simulated waterflood reservoir model, which are the daily liquid production and daily oil production of the production well.

[0014] The input data of the waterflood injection-production scheme optimization model is the injection-production scheme of the agent model. The output data of the waterflood injection-production scheme optimization model after being trained by the SIP model is the oil production of the production well, which is used as the reward value and fed back to the agent model. At the same time, the waterflood injection-production scheme optimization model will transfer the vertical splitting in the output data of the simulated waterflood reservoir model VerticalNet and the horizontal splitting in the output data of HorizonNet to the agent model as the input data of the agent model.

[0015] The input data of the agent model are the vertical splitting in the output data of VerticalNet and the horizontal splitting in the output data of HorizonNet transferred by the waterflood injection-production scheme optimization model. The output data of the agent model is the generated injection-production scheme. At the same time, the output data of the waterflood injection-production scheme optimization model is obtained as the reward value, and the reinforcement learning DDPG algorithm is used to optimize the agent model to obtain the optimal injection-production scheme.

[0016] The injection-production index migration optimization scheme model provides constraint conditions for the simulated waterflood reservoir model according to the actual production situation and business logic, using the existing data and the data calculated by the simulated waterflood reservoir model, as follows:

[0017] loss_mse_v: Calculate the mean square error loss of the daily injection volume and the injection splitting volume. According to the business logic, the values of the daily injection volume and the injection splitting volume should be equal;

[0018] loss_ploss: Liquid production loss per day. The liquid production per day calculated using the productivity formula should be equal to the liquid production per day fitted to the simulated waterflood reservoir model.

[0019] reg_poilloss: Oil production loss. The oil production per day calculated using the productivity formula should be equal to the oil production per day fitted to the simulated waterflood reservoir model.

[0020] reg_abse: Injection-production balance loss. Obtained according to business logic. The injection volume should be equal to the liquid production volume over a period of time.

[0021] reg_mse_vhl: Loss of the simulated waterflood reservoir model. The actual liquid production volume should be equal to the liquid production volume predicted by the simulated waterflood reservoir model.

[0022] reg_mse_v_1: Injection volume loss. The daily injection volume needs to be approximately equal to the sum of the total oil displacement volume and the edge water invasion volume.

[0023] Calculation method of edge water invasion volume: Calculate the historical edge water invasion volume according to the material balance equation and historical data to obtain the distribution of the historical edge water invasion volume. Since the distribution of the historical edge water invasion volume satisfies the Beta function distribution, the Beta function is used to fit the edge water invasion volume as the value of the injection-production index migration optimization scheme model.

[0024] A method for predicting the optimal injection-production scheme of an oil reservoir using a deep reinforcement learning model is carried out according to the following steps:

[0025] Step 1: Input the historical data of the daily injection volume and wellhead pressure into the simulated waterflood reservoir model to fit the historical production data of the real production environment.

[0026] Step 2: Input the historical production data fitted in Step 1 and the real historical production data into the injection-production index migration optimization scheme model to calculate the loss value, and return the loss value to the simulated waterflood reservoir model for the next round of training.

[0027] Step 3: Input the water injection split and liquid production split values output from the hidden layer of the trained simulated waterflood reservoir model into the waterflood injection-production scheme optimization model.

[0028] Step 4: The agent model obtains the water injection split and liquid production split from the waterflood injection-production scheme optimization model as inputs and gives an injection-production scheme.

[0029] Step 5: Input the injection-production scheme of the agent model into the waterflood injection-production scheme optimization model, and output the oil production as the reward value and return it to the agent model.

[0030] Step 6: The agent model uses the reinforcement learning DDPG algorithm to optimize its own injection-production plan based on the returned reward value until the plan reaches the optimal state.

[0031] The simulation waterflood reservoir model is carried out according to the following steps:

[0032] Step (1): Perform data preprocessing on the injection-production plan data output by the agent.

[0033] Step (2): Input the preprocessed data into VerticalNet and conduct network training.

[0034] Step (3): Transmit the output data of VerticalNet to HorizonNet as input data and conduct network training.

[0035] Step (4): Transmit the output data of the trained HorizonNet to LiftNet as input data and conduct network training.

[0036] Step (5): Calculate the historical edge water invasion volume and fit the future edge water invasion volume based on the historical edge water invasion volume distribution.

[0037] Step (6): Define the loss function to constrain the output values of each network layer.

[0038] The waterflood injection-production plan optimization model is carried out according to the following steps:

[0039] Step (1): The agent generates an injection-production plan.

[0040] Step (2): Transmit the injection-production plan generated by the agent as input data into the waterflood injection-production plan optimization model.

[0041] Step (3): After the training of the waterflood injection-production plan optimization model, the oil production of each production well is output.

[0042] Step (4): Transmit the output oil production as a reward back to the agent model and conduct the next round of training.

[0043] The agent model is carried out according to the following steps:

[0044] Step (1): Obtain the water injection splitting and liquid production splitting of the output data of the hidden layer of the simulation waterflood reservoir model.

[0045] Step (2): Generate a new injection-production plan according to the water injection splitting and liquid production splitting.

[0046] Step (3): Input the new injection-production plan into the waterflood injection-production plan optimization model.

[0047] Step (4): Use the oil production of the production well obtained as the reward value to optimize the agent model.

[0048] Step (5): Generate a new injection-production plan.

[0049] The beneficial effects of the present invention are as follows: The present invention can intelligently output the optimal injection-production plan for the oil reservoir;

[0050] By using the LSTM neural network to build a simulation of the real underground situation of the oil reservoir, it solves the problem that the agent in the oil reservoir project cannot be trained using the real production environment;

[0051] The overall calculation process is standardized, highly operable, the simulated underground oil reservoir environment is relatively close to the actual situation, and the output injection-production plan is scientific and clear;

[0052] On the basis of referring to traditional mathematical models, through a large number of calculation comparisons, the optimal injection-production big data model for the oil reservoir is finally determined, filling the domestic gap in using big data analysis and mining and machine learning technologies to predict the optimal injection-production plan for the oil reservoir. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a schematic diagram of the principle of the present invention.

[0054] For those of ordinary skill in the art, without creative efforts, other relevant drawings can be obtained according to the above drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] The technical solutions of the present invention will be further described below through specific embodiments.

[0056] Embodiment 1

[0057] An oil reservoir optimal injection-production plan prediction model using a deep reinforcement learning model, including a simulated waterflooding oil reservoir model, a waterflooding injection-production plan optimization model, an agent model, and an injection-production index migration and optimization plan model,

[0058] The simulated waterflooding oil reservoir model, also known as the hidden environment VHL model. Under normal circumstances, the learning of the agent model should be online, that is, continuously propose injection-production plans in the real production environment and observe the feedback after implementing production. However, the characteristics of oil production determine that we cannot randomly use the real production environment to train the agent to avoid waste. Therefore, a simulated waterflooding oil reservoir model is constructed to fit historical data, so as to simulate the actual production environment for the agent to learn;

[0059] The waterflood injection-production plan optimization model, also known as the explicit environment SIP model, constructs the waterflood injection-production plan optimization model to interact with the implicit environment VHL model and the agent model because the implicit environment VHL model only fits the underground situation and does not score the injection-production plans proposed by the agent, thus preventing the agent model from learning. The explicit environment SIP model passes the parameters output by the hidden layer of the implicit environment VHL model to the agent model. The agent model outputs the injection-production plan and inputs it into the explicit environment SIP model. After calculation, the oil production is obtained as the reward value and returned to the agent model for optimization;

[0060] The agent model, also known as Agent, is a model based on reinforcement learning. Its role is to determine the injection-production plan according to the current production status and continuously optimize it based on the reward value feedback by the explicit environment SIP model to output the optimal injection-production plan;

[0061] The injection-production index migration optimization plan model is used to improve the accuracy of the prediction results of the simulated waterflood reservoir model. According to the actual production situation and business logic, the existing data and the output results of the implicit environment are used to calculate the loss, and the implicit environment VHL model is optimized based on the loss.

[0062] Example Two

[0063] The optimal injection-production model of the reservoir is clearly divided into three major parts: the implicit environment, the explicit environment, and the agent. Among them, the implicit environment is responsible for simulating and fitting the real underground situation, the agent is responsible for continuously providing the injection-production plan, the explicit environment is responsible for interacting between the implicit environment and the agent, and the injection-production index migration optimization plan model is added as a constraint condition to the implicit environment for training.

[0064] The simulated waterflood reservoir model includes three sub-models: VerticalNet, HorizonNet, and LiftNet;

[0065] The role of VerticalNet is to simulate the water injection process. The input data of VerticalNet is the daily water injection volume and the wellhead pressure, which are the inputs of the simulated waterflood reservoir model. The output data of VerrticalNet are the vertical splitting of the injection well, the pressure of the injection well in each small layer, the formation pressure around the injection well in each small layer, and the total oil displacement volume of each small layer;

[0066] The role of HorizonNet is to simulate the underground oil displacement process. The input data of HorizonNet is the output data of VerticalNet. The output data of HorizonNet are the horizontal splitting of each small layer, the pressure of each production well in different small layers, the formation pressure around different small layers, the oil production volume and the oil displacement volume of each production well in different small layers;

[0067] The function of LiftNet is to simulate the process of lifting oil fluid to the ground during the actual production process. The input data of LiftNet is the output data of HorizonNet, and the output data of LiftNet is the output data of the simulated waterflood reservoir model, which is the daily liquid production and daily oil production of production wells.

[0068] The input data of the waterflood injection-production plan optimization model is the injection-production plan of the agent model. After being trained by the SIP model, the output data of the waterflood injection-production plan optimization model is the oil production of production wells, which is used as the reward value and fed back to the agent model. At the same time, the waterflood injection-production plan optimization model will pass the vertical splitting in the output data of VerticalNet and the horizontal splitting in the output data of HorizonNet of the simulated waterflood reservoir model to the agent model as the input data of the agent model.

[0069] The input data of the agent model is the vertical splitting in the output data of VerticalNet and the horizontal splitting in the output data of HorizonNet passed from the waterflood injection-production plan optimization model. The output data of the agent model is the generated injection-production plan. At the same time, the output data of the waterflood injection-production plan optimization model is obtained as the reward value, and the DDPG algorithm of reinforcement learning is used to optimize the agent model, so as to obtain the optimal injection-production plan.

[0070] The injection-production index migration optimization plan model provides constraint conditions for the simulated waterflood reservoir model according to the actual production situation and business logic, using the existing data and the data calculated by the simulated waterflood reservoir model. The specific conditions are as follows:

[0071] loss_mse_v: Calculate the mean square error loss between the daily injection volume and the injection splitting volume. According to the business logic, the values of the daily injection volume and the injection splitting volume should be equal.

[0072] loss_ploss: Daily liquid production loss. The daily liquid production calculated using the productivity formula should be equal to the daily liquid production fitted by the simulated waterflood reservoir model.

[0073] reg_poilloss: Oil production loss. The daily oil production calculated using the productivity formula should be equal to the daily oil production fitted by the simulated waterflood reservoir model.

[0074] reg_abse: Injection-production balance loss. Obtained according to the business logic, the injection volume and the liquid production volume should be equal within a certain period of time.

[0075] reg_mse_vhl: Simulated waterflood reservoir model loss. The actual liquid production volume should be equal to the liquid production volume predicted by the simulated waterflood reservoir model.

[0076] reg_mse_v_1: Injection volume loss. The daily injection volume needs to be approximately equal to the sum of the total oil displacement volume and the edge water intrusion volume.

[0077] Calculation method of edge water invasion volume: Calculate the historical edge water invasion volume according to the material balance equation and historical data to obtain the distribution of the historical edge water invasion volume. Since the distribution of the historical edge water invasion volume satisfies the Beta function distribution, the Beta function is used to fit the edge water invasion volume as the value of the injection-production index migration optimization scheme model.

[0078] Example 3

[0079] A method for predicting the optimal injection-production scheme of an oil reservoir using a deep reinforcement learning model is carried out according to the following steps:

[0080] Step 1, Input the historical data of daily injection volume and wellhead pressure into the simulation water drive oil reservoir model to fit the historical production data of the real production environment.

[0081] Step 2, Input the historical production data obtained by fitting in Step 1 and the real historical production data into the injection-production index migration optimization scheme model to calculate the loss value, and return the loss value to the simulation water drive oil reservoir model for the next round of training.

[0082] Step 3, Input the water injection split and liquid production split values output by the hidden layer of the trained simulation water drive oil reservoir model into the water drive injection-production scheme optimization model.

[0083] Step 4, The intelligent agent model obtains the water injection split and liquid production split from the water drive injection-production scheme optimization model as inputs and gives an injection-production scheme.

[0084] Step 5, Input the injection-production scheme of the intelligent agent model into the water drive injection-production scheme optimization model, and output the oil production as a reward value and return it to the intelligent agent model.

[0085] Step 6, The intelligent agent model uses the reinforcement learning DDPG algorithm to optimize its own injection-production scheme according to the returned reward value until the scheme reaches the optimal.

[0086] The simulation water drive oil reservoir model is carried out according to the following steps:

[0087] Step (1), Perform data preprocessing on the injection-production scheme data output by the intelligent agent.

[0088] Step (2), Input the preprocessed data into VerticalNet and perform network training.

[0089] Step (3), Transmit the output data of VerticalNet to HorizonNet as input data and perform network training.

[0090] Step (4): Transfer the output data of the trained HorizonNet to LiftNet as input data and conduct network training.

[0091] Step (5): Calculate the historical edge water invasion volume and fit the future edge water invasion volume based on the historical edge water invasion volume distribution.

[0092] Step (6): Define a loss function to constrain the output values of each network layer.

[0093] The water flooding injection-production plan optimization model proceeds as follows:

[0094] Step (1): The agent generates an injection-production plan.

[0095] Step (2): Transfer the injection-production plan generated by the agent as input data into the water flooding injection-production plan optimization model.

[0096] Step (3): After the training of the water flooding injection-production plan optimization model, the oil production of each production well is output.

[0097] Step (4): Transfer the output oil production as a reward back to the agent model and conduct the next round of training.

[0098] The agent model proceeds as follows:

[0099] Step (1): Obtain the water injection splitting and liquid production splitting of the output data of the hidden layer of the simulated water flooding reservoir model.

[0100] Step (2): Generate a new injection-production plan based on the water injection splitting and liquid production splitting.

[0101] Step (3): Input the new injection-production plan into the water flooding injection-production plan optimization model.

[0102] Step (4): Optimize the agent model with the obtained oil production of the production well as the reward value.

[0103] Step (5): Generate a new injection-production plan.

[0104] For ease of explanation, in the embodiments, spatial relative terms such as "upper", "lower", "left", and "right" are used to describe the relationship of one element or feature shown in the figure relative to another element or feature. It should be understood that in addition to the orientations shown in the figure, the spatial terms are intended to include different orientations during the use or operation of the device. For example, if the device in the figure is inverted, the element described as being "below" other elements or features will be positioned "above" other elements or features. Therefore, the exemplary term "lower" can include both upper and lower orientations. The device can be positioned in other ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used here can be interpreted accordingly.

[0105] Moreover, relational terms such as "first" and "second" are only used to distinguish one component with the same name from another, and do not necessarily require or imply any actual relationship or order between these components.

[0106] The present invention has been described in detail above, but the above content is only a preferred embodiment of the present invention and should not be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the application of the present invention should still fall within the scope covered by the patent of the present invention.

Claims

1. An apparatus for predicting an optimal injection-production plan model of a reservoir using a deep reinforcement learning model, characterized in that: It includes a simulated waterflood reservoir model, a waterflood injection-production plan optimization model, an agent model, and an injection-production index migration optimization plan model. The simulated waterflood reservoir model is used to fit the historical data of the real production environment, so as to simulate the actual production environment for the agent model to learn. The waterflood injection-production plan optimization model is used to interact with the simulated waterflood reservoir model and the agent model, so as to achieve the purpose of optimizing the agent model. The agent model is used to design the corresponding injection-production plan according to the current production status, and continuously optimize according to the reward value feedback by the waterflood injection-production plan optimization model, so as to output the optimal injection-production plan. The injection-production index migration optimization plan model is used to improve the accuracy of the prediction results of the simulated waterflood reservoir model. Among them, the simulated waterflood reservoir model includes three sub-models: VerticalNet, HorizonNet, and LiftNet. The input data of VerticalNet is the daily injection volume and wellhead pressure input to the simulated waterflood reservoir model. The output data of VerrticalNet are the vertical splitting of the injection well, the pressure of the injection well in each small layer, the formation pressure around the injection well in each small layer, and the total oil displacement in each small layer. The input data of HorizonNet are the output data of VerticalNet. The output data of HorizonNet are the horizontal splitting of each small layer, the pressure of each production well in different small layers, the formation pressure around different small layers, the oil production and oil displacement of each production well in different small layers. The input data of LiftNet are the output data of HorizonNet. The output data of LiftNet are the output data of the simulated waterflood reservoir model, which are the daily liquid production and daily oil production of the production well. The input data of the waterflood injection-production plan optimization model is the injection-production plan of the agent model. The output data of the waterflood injection-production plan optimization model after being trained by the SIP model is the oil production of the production well, and this is used as the reward value to feedback to the agent model. At the same time, the waterflood injection-production plan optimization model will pass the vertical splitting in the output data of VerticalNet of the simulated waterflood reservoir model and the horizontal splitting in the output data of HorizonNet to the agent model as the input data of the agent model. The input data of the agent model are the vertical splitting in the output data of VerticalNet and the horizontal splitting in the output data of HorizonNet passed by the waterflood injection-production plan optimization model. The output data of the agent model is the generated injection-production plan. At the same time, the output data of the waterflood injection-production plan optimization model is obtained as the reward value, and the DDPG algorithm of reinforcement learning is used to optimize the agent model, so as to obtain the optimal injection-production plan. The injection-production index migration optimization plan model provides constraint conditions for the simulated waterflood reservoir model according to the actual production situation and business logic, using the existing data and the data calculated by the simulated waterflood reservoir model. Specifically as follows: loss_mse_v: Calculate the mean square error loss between the daily injection volume and the injection split volume. According to the business logic, the values of the daily injection volume and the injection split volume should be equal. loss_ploss: Liquid production loss. The liquid production calculated using the productivity formula should be equal to the liquid production fitted by the simulated waterflood reservoir model; reg_poilloss: Oil production loss. The oil production calculated using the productivity formula should be equal to the oil production fitted by the simulated waterflood reservoir model; reg_abse: Injection-production balance loss. Obtained according to business logic. The injection volume should be equal to the liquid production volume within a certain period; reg_mse_vhl: Simulated waterflood reservoir model loss. The actual liquid production volume should be equal to the liquid production volume predicted by the simulated waterflood reservoir model; reg_mse_v_1: Injection volume loss. The daily injection volume should be approximately equal to the sum of the total oil displacement volume and the edge water invasion volume.

2. The device for predicting the optimal injection-production plan model of an oil reservoir using a deep reinforcement learning model according to claim 1, wherein: Calculation method of edge water invasion volume: Calculate the historical edge water invasion volume according to the material balance equation and historical data to obtain the distribution of the historical edge water invasion volume. Since the distribution of the historical edge water invasion volume satisfies the Beta function distribution, the Beta function is used to fit the edge water invasion volume as the value of the injection-production index migration optimization scheme model.

3. The reservoir optimal injection-production plan prediction method of a device for predicting a reservoir optimal injection-production plan using a deep reinforcement learning model according to any one of claims 1 or 2, characterized in that: Perform according to the following steps: Step 1, Input the historical data of the daily injection volume and wellhead pressure into the simulated waterflood reservoir model to fit the historical production data of the actual production environment; Step 2, Input the historical production data fitted in Step 1 and the actual historical production data into the injection-production index migration optimization scheme model to calculate the loss value, and return the loss value to the simulated waterflood reservoir model for the next round of training; Step 3, Input the water injection split and liquid production split values output by the hidden layer of the trained simulated waterflood reservoir model into the waterflood injection-production scheme optimization model; Step 4, The agent model takes the water injection split and liquid production split obtained from the waterflood injection-production scheme optimization model as inputs and gives an injection-production scheme; Step 5, Input the injection-production scheme of the agent model into the waterflood injection-production scheme optimization model, and output the oil production as the reward value and return it to the agent model; Step 6, The agent model uses the DDPG algorithm of reinforcement learning to optimize its own injection-production scheme according to the returned reward value until the scheme reaches the optimal.

4. The prediction method for the optimal injection-production plan of an oil reservoir according to claim 3, characterized in that: For the simulated waterflood reservoir model, perform according to the following steps: Step 1, Perform data preprocessing on the injection-production scheme data output by the agent; Step 2, Input the preprocessed data into VerticalNet and perform network training; Step 3, Pass the output data of VerticalNet to HorizonNet as input data and perform network training; Step 4, Pass the output data of the trained HorizonNet to LiftNet as input data and perform network training; Step 5, Calculate the historical edge water invasion volume and fit the future edge water invasion volume according to the historical edge water invasion volume distribution; Step 6, Define the loss function to constrain the output values of each network layer.

5. The prediction method for the optimal injection-production plan of an oil reservoir according to claim 4, characterized in that: For the waterflood injection-production scheme optimization model, perform according to the following steps: Step A, The agent generates an injection-production scheme; Step B, Input the injection-production scheme generated by the agent as input data into the waterflood injection-production scheme optimization model; Step C, After training by the waterflood injection-production scheme optimization model, output the oil production of each production well. Step D: Return the output oil production as a reward to the agent model and conduct the next round of training.

6. The prediction method for the optimal injection-production plan of an oil reservoir according to claim 5, characterized in that: The agent model proceeds as follows: Step a: Obtain the water injection split and liquid production split of the output data of the hidden layer of the simulated waterflood reservoir model; Step b: Generate a new injection-production plan based on the water injection split and liquid production split; Step c: Input the new injection-production plan into the waterflood injection-production plan optimization model; Step d: Optimize the agent model using the obtained oil production of the production wells as the reward value; Step e: Generate a new injection-production plan.

Images