A method and device for recommending a well work system based on reinforcement learning

Abstract

The application discloses a kind of based on reinforcement learning's recommended oil well work system method and device, comprising: obtaining the overall data matrix of reservoir attribute by reservoir numerical simulation model, and the first oil well geological development feature dataset is obtained by pre-processing data matrix;Based on the first oil well geological development feature dataset, and combining work system generates second oil well geological development feature dataset, and the second oil well geological development feature dataset is expanded using image enhancement technology;Based on oil well work system, construct practice network model, and train practice network model by the second oil well geological development feature dataset after expansion;The optimal work system is obtained by heuristic algorithm calling trained practice network model.This application is auxiliary with reinforcement learning, and the optimal work system of each oil well is recommended according to heuristic algorithm calling trained practice network model facing recommended oil well work system, and the optimization speed of oil well work system is improved.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a method and apparatus for recommending oil well operating regimes based on reinforcement learning. Background Technology

[0002] Recommended well operating regimes are designed to formulate optimal production strategies for each well, such as bottom hole pressure and daily fluid production, in order to maximize net present value (NPV) or oil and gas production. Optimizing the operating regimes of each well in an oilfield is a challenging task. Due to the uncertainty of reservoir understanding, the theory and calculation are difficult, and it is necessary to comprehensively consider reservoir geological development parameters (permeability, porosity, saturation, etc.). It belongs to a large-scale nonlinear optimization problem, and there are still considerable challenges in solving it efficiently and accurately.

[0003] In the field of oil extraction, the emergence of simulation software and various optimization algorithms has provided solutions for optimizing production. Simulation software can complete optimization tasks, but simulating each scheme can take anywhere from several minutes to several hours, and a single optimization often requires hundreds or thousands of simulations of different schemes. Conventional optimization algorithms are mainly divided into gradient-based methods and derivative-free methods. Although gradient-based optimization algorithms have high optimization speed, they are prone to getting trapped in local optima. Derivative-free optimization algorithms have higher flexibility and excellent global search capabilities, but this method requires a large number of simulations and has low computational efficiency. Moreover, conventional optimization methods have another unavoidable problem: they are generally only applicable to specific tasks. When encountering new tasks, they need to be re-optimized, and historical optimization experience is not remembered, resulting in a significant expenditure of computing power and time in field applications. Existing applications of reinforcement learning for well logging evaluation and energy consumption prediction are all trained on specific reservoir models, making it difficult to apply these models to the prediction of other reservoirs. Furthermore, the prediction process still involves a large amount of numerical simulation calculations, failing to fully reflect the speed advantage of reinforcement learning. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] Therefore, this invention provides a method and apparatus for recommending oil well operating regimes based on reinforcement learning to solve the problems of slow optimization speed, low computational efficiency, and high cost of existing recommended oil well operating regimes.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] In a first aspect, embodiments of the present invention provide a method for recommending oil well operating procedures based on reinforcement learning, comprising: obtaining an overall data matrix of reservoir attributes through a reservoir numerical simulation model, and preprocessing the data matrix to obtain a first oil well geological development feature dataset;

[0009] Based on the first oil well geological development feature dataset, a second oil well geological development feature dataset is generated in combination with the working system, and the second oil well geological development feature dataset is expanded using image enhancement technology.

[0010] A practice network model was constructed based on the oil well operating system, and the practice network model was trained using the expanded second oil well geological development feature dataset.

[0011] The optimal working system is obtained by calling the trained practice network model through a heuristic algorithm.

[0012] As a preferred embodiment of the reinforcement learning-based method for recommending oil well operating systems described in this invention, the overall data matrix of reservoir attributes includes: based on reservoir sample data parameters, setting the control radius of each oil well in the reservoir model with the grid where each oil well is located as the center, and extracting square areas of the same size to obtain a square area data matrix.

[0013] Each sample data parameter is a data matrix;

[0014] The data matrix includes: an effective grid matrix, a pore volume matrix, a permeability matrix, a pressure field matrix, a saturation matrix, and a bottom hole pressure matrix for each oil and water well.

[0015] As a preferred embodiment of the reinforcement learning-based method for recommending oil well operating regimes according to the present invention, the data matrix is ​​preprocessed to obtain a first oil well geological development feature dataset, including:

[0016] Based on the matrix data of the parameters, the matrix data is converted into an image, with the numerical values ​​at each position in the matrix corresponding to the colors of the pixels at each position in the image.

[0017] An image can be used as the first oil well geological development feature dataset if and only if the image of each block does not contain sample labels.

[0018] Each sample in the first oil well geological development feature dataset contains at least one oil well parameter.

[0019] As a preferred embodiment of the reinforcement learning-based method for recommending oil well operating regimes described in this invention, the method includes: generating a second oil well geological development feature dataset, comprising:

[0020] By using reinforcement learning to invoke the reservoir numerical simulation model through the first oil well geological development feature dataset, the NPV of each oil well sample operating under the oil well geological development features and working system is obtained;

[0021] The NPV of each oil well sample is normalized, and the work system is standardized to generate a labeled second oil well geological development feature dataset required by the practice network model.

[0022] As a preferred embodiment of the reinforcement learning-based method for recommending oil well operating regimes described in this invention, the second oil well geological development feature dataset is expanded using image enhancement technology, including:

[0023] First, rotate the sample by 90°, 180°, and 270° respectively;

[0024] The new samples obtained by rotation are mirrored vertically and horizontally to obtain an expanded dataset of geological development features of the second oil well, covering various geological development conditions.

[0025] As a preferred embodiment of the reinforcement learning-based method for recommending oil well operating regimes described in this invention, the method includes: constructing and training a practice network model, comprising:

[0026] The practical network model includes three convolutional layers, four fully connected layers, and the last fully connected layer is the output layer. The activation functions in the three convolutional layers and the first three fully connected layers are all ReLU functions, and the output layer has no activation function.

[0027] The expanded second oil well geological development feature dataset is divided into a first training set and a first validation set according to the proportions.

[0028] The geological development characteristics data and standardized bottom hole pressure from each practice sample in the first training set are input into the practice network model for training;

[0029] The first validation set is used to determine whether the tested and trained model meets the requirements.

[0030] If the trained model meets the accuracy requirements, save the trained model.

[0031] If the trained model does not meet the accuracy requirements, the first training set will be divided into a second training set and a second validation set proportionally, and the first validation set will be divided into a third training set and a third validation set proportionally.

[0032] The second training set is combined with the third training set to form a new fourth training set, and the second validation set is combined with the third validation set to form a new fourth validation set. The model is then retrained based on the fourth training set and the fourth validation set until the model accuracy meets the requirements.

[0033] As a preferred embodiment of the reinforcement learning-based method for recommending oil well operating regimes according to the present invention, the optimal operating regime is obtained by calling the trained practice network model through a heuristic algorithm, including:

[0034] Oil wells were operated under different bottom hole pressures, and normalized NPV was obtained.

[0035] When NPV is closer to 1, the working regime is better. At this time, the bottom hole pressure value under the maximum NPV is the optimal bottom hole pressure.

[0036] When the standardized operating system is 0, it corresponds to the default operating system value of the oil well.

[0037] When the practice network model and heuristic algorithm for oil well operating systems obtain the optimal bottom hole pressure, the practice network model is encapsulated to construct an oil well recommended operating system platform.

[0038] Secondly, embodiments of the present invention provide a device for recommending oil well operating procedures based on reinforcement learning, comprising:

[0039] The data acquisition module is used to obtain the overall data matrix of reservoir attributes through the reservoir numerical simulation model, and to preprocess the data matrix to obtain the first oil well geological development feature dataset.

[0040] The data augmentation processing module is used to generate a second oil well geological development feature dataset based on the first oil well geological development feature dataset and in combination with the working system, and to expand the second oil well geological development feature dataset using image augmentation technology.

[0041] The model building and training module is used to construct a practice network model based on the oil well working system, and to train the practice network model using the expanded second oil well geological development feature dataset.

[0042] The optimization module is used to obtain the optimal working system by calling the trained practice network model through heuristic algorithms.

[0043] Thirdly, embodiments of the present invention provide a computing device, including:

[0044] Memory and processor;

[0045] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the one or more programs are executed by the one or more processors, the one or more processors implement the reinforcement learning-based recommended oil well operating system method as described in any embodiment of the present invention.

[0046] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method for recommending oil well operating regimes based on reinforcement learning.

[0047] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention uses reinforcement learning as an auxiliary method to prepare a diverse dataset of oil well geological development features, uses image enhancement technology to expand the number of samples, and uses the samples to train a CNN network model for recommending oil well operating regimes. Based on a heuristic algorithm, it calls the trained practice network model for recommending oil well operating regimes to recommend the optimal operating regime for each oil well, thereby improving the optimization speed of oil well operating regimes and achieving high computational efficiency. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0049] Figure 1 This is a flowchart illustrating a method and apparatus for recommending oil well operating regimes based on reinforcement learning, according to an embodiment of the present invention.

[0050] Figure 2 This is a practical network model architecture diagram of a method and apparatus for recommending oil well operating regimes based on reinforcement learning, as described in one embodiment of the present invention.

[0051] Figure 3 This is an example diagram of oil well operating system control samples and related parameters, illustrating a reinforcement learning-based method and apparatus for recommending oil well operating systems according to an embodiment of the present invention.

[0052] Figure 4 This is an image enhancement flowchart of a method and apparatus for recommending oil well operating regimes based on reinforcement learning, according to one embodiment of the present invention.

[0053] Figure 5 This is a model verification result diagram of a method and apparatus for recommending oil well operating regimes based on reinforcement learning, as described in one embodiment of the present invention. Detailed Implementation

[0054] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0055] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0056] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0057] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0058] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0059] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0060] Example 1

[0061] Reference Figures 1-4As one embodiment of the present invention, this embodiment provides a method for recommending oil well operating regimes based on reinforcement learning, comprising:

[0062] S1: Obtain the overall data matrix of reservoir attributes through the reservoir numerical simulation model, and preprocess the data matrix to obtain the first oil well geological development feature dataset;

[0063] Furthermore, the overall data matrix of reservoir attributes includes: based on the reservoir sample data parameters, setting the control radius of each well in the reservoir model with the grid where each well is located as the center, and extracting square areas of the same size to obtain the square area data matrix;

[0064] Each sample data parameter is a data matrix;

[0065] The data matrix includes: effective grid matrix, pore volume matrix, permeability matrix, pressure field matrix, saturation matrix, and bottom hole pressure matrix for each oil and water well.

[0066] In one optional embodiment, Eclipse is used as the reservoir numerical simulation software, i.e., the environment, of the present invention. The state is the geological development characteristic data corresponding to each oil well, and the state is also the input of reinforcement learning; the action is the bottom hole pressure given to the small well group by the agent according to the input state, and the action is also the output of reinforcement learning; the reward is the net profit obtained by the oil well from mining for a period of time under the bottom hole flowing pressure condition.

[0067] Furthermore, the data matrix is ​​preprocessed to obtain the geological development feature dataset of the first oil well, including:

[0068] Based on the matrix data of each parameter, the matrix data is converted into an image by using the numerical values ​​at each position in the matrix to correspond to the colors of the pixels at each position in the image.

[0069] An image can be used as the first oil well geological development feature dataset if and only if the image of each block does not contain sample labels.

[0070] Each sample in the first oil well geological development feature dataset contains at least one oil well parameter.

[0071] S2: Based on the geological development feature dataset of the first oil well, and combined with the working system, a geological development feature dataset of the second oil well is generated, and the second oil well geological development feature dataset is expanded using image enhancement technology;

[0072] Furthermore, a second oil well geological development feature dataset is generated, including:

[0073] By using reinforcement learning to invoke the reservoir numerical simulation model through the first oil well geological development feature dataset, the NPV of each oil well sample operating under the oil well geological development features and working system is obtained;

[0074] The NPV of each oil well sample is normalized, and the work system is standardized to generate a labeled second oil well geological development feature dataset required by the practice network model.

[0075] Specifically, the normalized representation is as follows:

[0076]

[0077] Among them, u new For the normalized NPV, u i For the i-th NPV value, min(u i ) is the minimum NPV value, max(u i ) represents the maximum NPV value;

[0078] The standardized representation is:

[0079]

[0080] Among them, u new For the standardized work system, u i Let be the value for the i-th work system. Let σ(u) be the average value of the work system, and σ(u) be the standard deviation of the work system.

[0081] It should be noted that each data point in the labeled second oil well geological development feature dataset contains geological development feature data and standardized oil well operating procedures, and the label is the normalized NPV.

[0082] In an optional embodiment, the reinforcement learning algorithm first inputs geological development feature data, and only one grid is input at a time. The AI ​​then calls Eclipse to derive the NPV (Net Present Value) of the corresponding well under bottomhole pressure for a certain period of time, and then replaces the geological development feature data to start a new round of operation. The NPV is represented as:

[0083]

[0084] Where, N prd and N inj C represents the number of injection wells and production wells in the reservoir model, respectively. o The revenue per barrel of oil is expressed in USD / STB. In this example, C o At 70 USD / STB, C w The cost of produced water treatment is expressed in USD / STB, and in this example it is 5 USD / STB. These represent the average oil and water production of the j-th production well at the n-th time step, in units of STB / D. This represents the water injection volume of the k-th injection well at time step n, expressed in STB / D.

[0085] After n learning cycles, the Q-table stops updating, the reinforcement learning process ends, and the reinforcement learning algorithm can then derive the wellbore pressure from the image data corresponding to each block.

[0086] Furthermore, image enhancement techniques were used to expand the geological development feature dataset of the second oil well, including:

[0087] First, rotate the sample by 90°, 180°, and 270° respectively;

[0088] The new samples obtained by rotation are mirrored vertically and horizontally to obtain an expanded dataset of geological development features of the second oil well, covering various geological development conditions.

[0089] It should be noted that, due to the limited number of oil wells in a reservoir, the oil well operating system control sample set contains relatively few samples. By using image enhancement technology to diversify the oil well geological development feature dataset (in image form), a large number of oil well geological development feature datasets covering various geological development conditions are obtained.

[0090] S3: Construct a practice network model based on oil well operating system, and train the practice network model using the expanded second oil well geological development feature dataset;

[0091] Furthermore, constructing and training practical network models includes:

[0092] The practical network architecture in the practical network model includes three convolutional layers, four fully connected layers, and the last fully connected layer is the output layer. The activation functions in the three convolutional layers and the first three fully connected layers are all ReLU functions, and the output layer has no activation function.

[0093] The expanded second oil well geological development feature dataset is divided into a first training set and a first validation set according to the proportions.

[0094] The geological development characteristics data and standardized bottom hole pressure from each practice sample in the first training set are input into the practice network model for training;

[0095] The first validation set is used to determine whether the tested and trained model meets the requirements.

[0096] If the trained model meets the accuracy requirements, save the trained model.

[0097] If the trained model does not meet the accuracy requirements, the first training set will be divided into a second training set and a second validation set proportionally, and the first validation set will be divided into a third training set and a third validation set proportionally.

[0098] The second training set is combined with the third training set to form a new fourth training set, and the second validation set is combined with the third validation set to form a new fourth validation set. The model is then retrained based on the fourth training set and the fourth validation set until the model accuracy meets the requirements.

[0099] Specifically, the practice network model outputs the normalized NPV of the corresponding oil well under the given working regime, based on the input, namely the geological development feature data in the second oil well geological development feature dataset and the standardized working regime.

[0100] S4: The optimal working system is obtained by calling the trained practice network model through heuristic algorithms;

[0101] Furthermore, the optimal working regime is obtained by using heuristic algorithms to invoke the trained practice network model, including:

[0102] Oil wells were operated under different bottom hole pressures, and normalized NPV was obtained.

[0103] When NPV is closer to 1, the working regime is better. At this time, the bottom hole pressure value under the maximum NPV is the optimal bottom hole pressure.

[0104] When the standardized operating system is 0, it corresponds to the default operating system value of the oil well.

[0105] When the practice network model and heuristic algorithm for oil well operating regimes obtain the optimal bottom hole pressure, the practice network model is encapsulated to construct a platform for recommending operating regimes for oil wells. It should be noted that the heuristic algorithm, by calling the trained practice network model for recommending oil well operating regimes, changes the operating regime under unchanged practice samples to obtain the NPV under different operating regimes. The operating regime that yields the maximum NPV is the optimal operating regime.

[0106] In an optional embodiment, the input to the heuristic algorithm is a standardized bottom hole pressure value that varies within a certain range. The standardized bottom hole pressure and geological development feature data in the practice samples are used as input to the practice network. The output of the heuristic algorithm is the normalized NPV of the oil well after a period of production under different bottom hole pressures. The purpose is to obtain the bottom hole pressure value with the maximum NPV, i.e., the optimal bottom hole pressure.

[0107] The above is an illustrative scheme of a method for recommending oil well working regimes based on reinforcement learning, as described in this embodiment. It should be noted that the technical solution of this device for recommending oil well working regimes based on reinforcement learning belongs to the same concept as the technical solution of the method for recommending oil well working regimes based on reinforcement learning described above. Details not described in detail in the technical solution of the device for recommending oil well working regimes based on reinforcement learning in this embodiment can be found in the description of the technical solution of the method for recommending oil well working regimes based on reinforcement learning described above.

[0108] This embodiment provides a reinforcement learning-based device for recommending oil well operating procedures, comprising:

[0109] The data acquisition module is used to obtain the overall data matrix of reservoir attributes through the reservoir numerical simulation model, and to preprocess the data matrix to obtain the first oil well geological development feature dataset.

[0110] The data augmentation processing module is used to generate a second oil well geological development feature dataset based on the first oil well geological development feature dataset and in combination with the working system, and to expand the second oil well geological development feature dataset using image augmentation technology.

[0111] The model building and training module is used to construct a practice network model based on the oil well working system, and the practice network model is trained by the expanded second oil well geological development feature dataset;

[0112] The optimization module is used to obtain the optimal working system by calling the trained practice network model through heuristic algorithms.

[0113] This embodiment also provides a computing device applicable to reinforcement learning-based methods for recommending oil well operating regimes, including:

[0114] The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement the reinforcement learning-based recommended oil well operating regime method proposed in the above embodiments.

[0115] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0116] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the method for implementing a recommended oil well operating regime based on reinforcement learning as proposed in the above embodiments.

[0117] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0118] Example 2

[0119] Reference Figures 1-5 This is one embodiment of the present invention. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0120] In this embodiment, the reservoir model has 93 oil wells and 77 water wells. Using a gridded reservoir model, the reservoir has 238*161*1 grids, with each grid measuring 60.28m*58.88m*20.60m. Each oil well in the reservoir model is centered on a 11*11 block, i.e., 11*11 grids. The controlled well production parameter is the bottomhole pressure, such as... Figure 3 As shown, the oil well operating system control sample and related parameters are: 1: effective grid matrix data; 2: pore volume matrix data; 3: permeability matrix data; 4: pressure field matrix data; 5: saturation field matrix data; 6: injection-production field matrix data.

[0121] like Figure 4As shown, in this embodiment, the dataset before image enhancement contains 900 geological development feature data. The geological development feature data (in image form) is rotated by 90°, 180°, and 270°, expanding the sample size to 2700. Then, the images obtained from each rotation are mirrored vertically and horizontally, expanding the sample size to 10800. The geological development feature data after each operation is saved, finally generating the expanded second oil well geological development feature dataset for training the practical network model.

[0122] The validation results of the trained network model are as follows: Figure 5 As shown in the figure, the distribution of the predicted values ​​(predict_values) output by the practice network model and the 100 real values ​​(real_values) in the validation set is illustrated. The vertical axis in the figure represents the normalized NPV. As can be seen from the figure, the prediction results of the practice network model highly match the actual results. The practice network model of this invention can obtain the corresponding NPV based on the input well group and working regime. By using one or more methods such as hyperparameter optimization or changing the structure of the practice network model, the accuracy and practicality of the practice model for recommending working regimes for oil wells are further improved.

[0123] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0124] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0125] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0126] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0128] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0129] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for recommending oil well operating regimes based on reinforcement learning, characterized in that, include: The overall data matrix of reservoir attributes is obtained by using a reservoir numerical simulation model, and the data matrix is ​​preprocessed to obtain the first oil well geological development feature dataset. The overall data matrix of reservoir attributes includes: based on the reservoir sample data parameters, setting the control radius of the oil well in the reservoir model with the grid where each oil well is located as the center, and extracting square areas of the same size to obtain a square area data matrix. Each of the sample data parameters is a data matrix; The data matrix includes: effective grid matrix, pore volume matrix, permeability matrix, pressure field matrix, saturation matrix, and bottom hole pressure matrix for each oil well; Based on the first oil well geological development feature dataset, a second oil well geological development feature dataset is generated by combining the working system, and the second oil well geological development feature dataset is expanded using image enhancement technology. The generation of the second oil well geological development feature dataset includes: By using reinforcement learning to invoke the reservoir numerical simulation model through the first oil well geological development feature dataset, the NPV of each oil well sample operating under the oil well geological development features and working system is obtained; The NPV of each oil well sample is normalized, and the working system is standardized to generate a labeled second oil well geological development feature dataset required by the practice network model. Each data point in the labeled second oil well geological development feature dataset contains geological development feature data and the standardized working system of the oil well. The label is the normalized NPV. A practice network model was constructed based on the oil well operating system, and trained using an expanded second oil well geological development feature dataset. The construction and training of the practice network model included: The practice network model includes three convolutional layers, four fully connected layers, and the last fully connected layer is the output layer. The activation function in the three convolutional layers and the first three fully connected layers is the ReLU function, and the output layer has no activation function. The expanded second oil well geological development feature dataset is divided into a first training set and a first validation set according to the proportions. The geological development characteristics data and standardized bottom hole pressure from each practice sample in the first training set are input into the practice network model for training; The first validation set is used to determine whether the tested and trained model meets the requirements. If the trained model meets the accuracy requirements, save the trained model. If the trained model does not meet the accuracy requirements, the first training set will be divided into a second training set and a second validation set proportionally, and the first validation set will be divided into a third training set and a third validation set proportionally. The second training set and the third training set are combined into a new fourth training set, and the second validation set and the third validation set are combined into a new fourth validation set. The model is then retrained based on the fourth training set and the fourth validation set until the model accuracy meets the requirements. The optimal working regime is obtained by calling the trained practice network model through a heuristic algorithm, specifically including: operating the oil well under different bottom hole pressures and obtaining the normalized NPV. When NPV is closer to 1, the working regime is better. At this time, the bottom hole pressure value under the maximum NPV is the optimal bottom hole pressure. When the standardized operating system is 0, it corresponds to the default operating system value of the oil well. When the practice network model and heuristic algorithm for oil well operating systems obtain the optimal bottom hole pressure, the practice network model is encapsulated to construct an oil well recommended operating system platform.

2. The method for recommending oil well operating procedures based on reinforcement learning as described in claim 1, characterized in that, The data matrix is ​​preprocessed to obtain the first oil well geological development feature dataset, including: Based on the matrix data of the parameters, the matrix data is converted into an image, with the numerical values ​​at each position in the matrix corresponding to the colors of the pixels at each position in the image. The image can be used as the first oil well geological development feature dataset if and only if the image of each block does not contain sample labels; Each sample in the first oil well geological development feature dataset contains at least one oil well parameter.

3. The method for recommending oil well operating procedures based on reinforcement learning as described in claim 2, characterized in that, The geological development feature dataset of the second oil well was expanded using image enhancement techniques, including: First, rotate the sample by 90°, 180°, and 270° respectively; The new samples obtained by rotation are mirrored vertically and horizontally to obtain an expanded dataset of geological development features of the second oil well, covering various geological development conditions.

4. A device for recommending oil well operating procedures based on reinforcement learning, applied to the method described in any one of claims 1-3, characterized in that, include: The data acquisition module is used to obtain the overall data matrix of reservoir attributes through the reservoir numerical simulation model, and to preprocess the data matrix to obtain the first oil well geological development feature dataset. The data augmentation processing module is used to generate a second oil well geological development feature dataset based on the first oil well geological development feature dataset and in combination with the working system, and to expand the second oil well geological development feature dataset using image augmentation technology. The model building and training module is used to construct a practice network model based on the oil well working system, and to train the practice network model using the expanded second oil well geological development feature dataset. The optimization module is used to obtain the optimal working system by calling the trained practice network model through heuristic algorithms.

5. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the method for recommending oil well operating regimes based on reinforcement learning as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the reinforcement learning-based recommended oil well operating regime method according to any one of claims 1 to 3.

Images