Parameter optimization method, device and equipment for oil reservoir development and storage medium

Abstract

The invention discloses a parameter optimization method and device for oil reservoir development, equipment and a storage medium, and relates to the technical field of oil reservoir development. Comprising the steps that oil field geological parameters, well location and operation parameters, injection parameters and economic evaluation parameters are obtained and input into a numerical simulator to complete initialization, and oil reservoir state change data and development effect indexes are calculated; generating a first data set representing the corresponding relationship among the parameters, the oil reservoir state and the development effect by combining a particle swarm optimization algorithm; a second data set containing multiple expert tracks is obtained through screening and data enhancement processing, and the expert tracks reflect time sequence evolution of the oil reservoir state under the action of the injection parameters; inputting the expert track into a preset inverse reinforcement learning model, and screening a target expert track according to the strategy network update amplitude and the discriminator probability value; and finally, optimizing the input parameters based on the target expert trajectory to obtain optimal parameters. Intelligent optimization of oil reservoir parameters is realized, optimization efficiency and suitability are improved, and development benefits are guaranteed.

Description

Technical Field

[0001] The present application relates to the technical field of oil reservoir development, and in particular to a parameter optimization method, device, equipment and storage medium for oil reservoir development. Background Art

[0002] As oilfield development enters the medium-to-high water-cut stage, polymer injection and gas displacement, as core technologies for enhancing oil recovery, are becoming increasingly widespread in complex reservoir development. However, reservoir development is influenced by factors such as geological heterogeneity, dynamic changes in fluid properties, and the coupling of multiple development objectives. This significantly increases the difficulty of optimizing key parameters such as polymer injection concentration, gas injection rate, and injection-production cycle. Improper parameter settings can not only result in unsatisfactory oil recovery but can also lead to wellbore blockage, excessive injection costs, or decreased reservoir stability, severely hampering the economic benefits and sustainability of oilfield development.

[0003] Currently, reservoir development parameter optimization primarily relies on traditional numerical simulation combined with manual trial and error, swarm intelligence algorithms (such as particle swarm optimization and genetic algorithms), or expert experience-driven approaches. Traditional numerical simulation and manual trial and error methods require engineers to repeatedly adjust parameters based on experience and verify the results. This makes it difficult to systematically integrate complex factors such as multi-well coordination and reservoir dynamics, resulting in low optimization efficiency and poor stability. While swarm intelligence algorithms can perform a global search across parameter space, they are prone to falling into local optima and lack adaptability to heterogeneous reservoirs or fluctuating fluid properties. Expert experience-driven approaches rely on empirical formulas derived from historical development cases or subjective expert judgment, making it difficult to quantify the coupling relationships between multiple objectives. Furthermore, the cost of transferring experience is high and generalization is weak.

[0004] Furthermore, while traditional reinforcement learning methods attempt to construct an "agent-reservoir environment" interaction model, they require manually pre-defined reward functions. However, the fuzzy weights of multiple objectives in reservoir development, the time-varying rewards in dynamic environments, and the difficulty in making implicit experience explicit all contribute to inherent difficulties in reward function design and the tendency for optimization strategies to become disconnected from engineering practice. Therefore, overcoming the limitations of traditional methods, achieving intelligent and efficient optimization of reservoir development parameters, and improving the adaptability of parameter optimization results to complex reservoir conditions and their alignment with development objectives are pressing challenges in the field of polymer and gas injection in oilfields. Summary of the Invention

[0005] In view of this, the embodiments of the present application provide a parameter optimization method, device, equipment, and storage medium for reservoir development, which can achieve intelligent and efficient optimization of reservoir development parameters, and improve the adaptability of parameter optimization results to complex reservoir conditions and the fit with development goals. The parameter optimization method, device, equipment, and storage medium for reservoir development provided in the embodiments of the present application are implemented as follows: Acquiring input parameters during the oil reservoir development process, the input parameters including oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters; inputting the input parameters into a numerical simulator to obtain an initialized numerical simulator; the numerical simulator is used to calculate oil reservoir state change data and development effect indicators during the oil field development process, the oil reservoir state change data including at least one of formation pressure distribution, remaining oil distribution, and saturation change; and the development effect indicator including at least one of oil production, water cut, and recovery factor; generating a first data set according to the numerical simulator and the particle swarm optimization algorithm, wherein the first data set is used to characterize the corresponding relationship between the input parameters, the reservoir state change data, and the development effect index; Performing screening and data enhancement on the first data set to obtain a second data set; The second data set includes a plurality of expert trajectories, which are used to characterize the dynamic process of the injection parameter acting on the reservoir and the evolution of the reservoir state over time. The plurality of expert trajectories are respectively input into a preset inverse reinforcement learning model to obtain the update amplitudes of the policy network in the inverse reinforcement learning model and the probability values ​​of the discriminator in the inverse reinforcement learning model corresponding to the plurality of expert trajectories. When the update amplitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold, the expert trajectory with the smallest update amplitude and the largest probability value is determined as the target expert trajectory; Parameter optimization is performed on the input parameters according to the target expert trajectory to obtain optimal input parameters.

[0006] In some embodiments, after obtaining input parameters in the reservoir development process, the method further includes: Normalizing the input parameters to obtain normalized input parameters; Performing outlier processing on the input parameters, replacing outliers in the input parameters with boundary values, and supplementing missing values ​​through linear interpolation and mean filling, to obtain outlier-processed input parameters; The input parameters are encoded into an action vector to obtain a parameter vector.

[0007] In some embodiments, generating the first data set according to the numerical simulator and the particle swarm optimization algorithm includes: Initialize several particles, each particle represents a set of parameter vectors, and configure the inertia weight, learning factor, and maximum number of iterations of the particle swarm optimization algorithm; Perform iterative processing on several examples. In each round of iteration, the particle speed and position are updated according to the particle's historical speed, individual historical optimal position, and global optimal position to obtain the iterated parameter vector; Performing a boundary check on the parameter vector after the iteration, correcting the parameter vector to the lower boundary value when the parameter vector is lower than the lower boundary of the dimension, and correcting the parameter vector to the upper boundary value when the parameter vector is higher than the upper boundary of the dimension; The parameter vector is input into a numerical simulator to obtain a first data set.

[0008] In some embodiments, before inputting the plurality of expert trajectories into a preset inverse reinforcement learning model to obtain the update amplitudes of the policy network in the inverse reinforcement learning model and the probability values ​​of the discriminator in the inverse reinforcement learning model corresponding to the plurality of expert trajectories, the method further includes: Establish an initial inverse reinforcement learning model, input historical input parameters into the policy network and discriminator of the initial inverse reinforcement learning model, and obtain the initial update amplitude and initial probability value; A plurality of preset expert trajectories are obtained from an expert trajectory library, the discriminator is trained according to the obtained plurality of preset expert trajectories and the initial probability value to obtain a trained discriminator, and the policy network is trained according to the obtained plurality of preset expert trajectories and the initial update amplitude to obtain a trained policy network.

[0009] In some embodiments, the optimizing the input parameters according to the target expert trajectory to obtain the optimal input parameters further includes: A trajectory legitimacy check is performed on the target expert trajectory, and when the legitimacy of the target expert trajectory exceeds a preset legal range, the legitimacy of the target expert trajectory is corrected.

[0010] In some embodiments, performing screening and data enhancement on the first data set to obtain the second data set includes: Screening the first data set to obtain data whose development effect index is greater than a preset threshold and whose injection parameters meet physical and engineering constraints, thereby obtaining a screened first data set; The first data set after the screening process is interpolated in time steps and a preset amplitude disturbance is applied to the injection parameters to generate new data to obtain a second data set.

[0011] In some embodiments, the preset inverse reinforcement learning model is a dual-network static architecture based on generative adversarial imitation learning, the policy network is fixedly equipped with a 3-layer fully connected neural network structure, and the discriminator is fixedly equipped with a 2-layer fully connected neural network structure.

[0012] An embodiment of the present application provides a parameter optimization device for oil reservoir development, comprising: an acquisition module for acquiring input parameters during the oil reservoir development process, the input parameters including oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters, inputting the input parameters into a numerical simulator to obtain an initialized numerical simulator, the numerical simulator being used to calculate reservoir state change data and development effect indicators during the oil field development process, the reservoir state change data including at least one of formation pressure distribution, remaining oil distribution, and saturation change, and the development effect indicator at least one of oil production, water cut, and recovery rate; a generating module, configured to generate a first data set according to the numerical simulator and the particle swarm optimization algorithm, wherein the first data set is used to characterize the corresponding relationship between the input parameters, the reservoir state change data, and the development effect indicator; a processing module, configured to perform screening and data enhancement on the first data set to obtain a second data set; The processing module is further configured to: the second data set includes a plurality of expert trajectories, the expert trajectories being used to characterize the dynamic process of the injection parameter acting on the reservoir and the evolution of the reservoir state over time; input the plurality of expert trajectories into a preset inverse reinforcement learning model, obtain update amplitudes of the policy network in the inverse reinforcement learning model and probability values ​​of the discriminator in the inverse reinforcement learning model corresponding to the plurality of expert trajectories; and, when the update amplitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold, determine the expert trajectory with the smallest update amplitude and the largest probability value as the target expert trajectory; The optimization module is used to optimize the input parameters according to the target expert trajectory to obtain the optimal input parameters.

[0013] The computer device provided in an embodiment of the present application includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the method described in the embodiment of the present application is implemented.

[0014] The computer-readable storage medium provided in the embodiment of the present application stores a computer program thereon, and when the computer program is executed by a processor, the method provided in the embodiment of the present application is implemented.

[0015] The embodiment of the present application provides a parameter optimization method, device, equipment and storage medium for reservoir development. It includes: obtaining oilfield geological parameters, well location and operation parameters, injection parameters and economic evaluation parameters, inputting into a numerical simulator to complete initialization, calculating reservoir state change data and development effect indicators; then combining the particle swarm optimization algorithm to generate a first data set representing the correspondence between parameters, reservoir state and development effect; after screening and data enhancement processing, a second data set containing multiple expert trajectories is obtained, and the expert trajectories reflect the time series evolution of the reservoir state under the action of injection parameters; inputting the expert trajectories into a preset inverse reinforcement learning model, and screening the target expert trajectories based on the policy network update amplitude and the discriminator probability value; finally, optimizing the input parameters based on the target expert trajectories to obtain the optimal parameters. It realizes the intelligent optimization of reservoir parameters, improves the optimization efficiency and adaptability, ensures the development benefits, and solves the technical problems raised in the background technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments of the present application or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A schematic diagram of a process flow for implementing a parameter optimization method for reservoir development provided in an embodiment of the present application; Figure 2 A schematic diagram of an implementation flow for processing input parameters provided in an embodiment of the present application; Figure 3 A parameter optimization device for oil reservoir development is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] The following description of some of the technologies involved in the embodiments of this application is provided to facilitate understanding and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for the sake of clarity and conciseness, some descriptions of well-known functions and structures are omitted from the following description.

[0020] Figure 1 This is a schematic diagram of the implementation process of a parameter optimization method for reservoir development provided in an embodiment of the present application, including steps 101 to 105. Figure 1 This is only an execution order shown in the embodiment of the present application, and does not represent the only execution order of a parameter optimization method for reservoir development. Figure 1 The steps shown may be performed in parallel or reversed.

[0021] Step 101: obtain input parameters in the reservoir development process, input the input parameters into a numerical simulator, and obtain an initialized numerical simulator.

[0022] In the embodiments of the present application, oilfield geological parameters are obtained through core experiments and logging data interpretation, including the permeability distribution and porosity distribution of different areas within the block, as well as the formation pressure and temperature monitored in real time by bottomhole pressure gauges and temperature sensors. At the same time, the remaining oil distribution and water cut are calculated by production dynamic analysis software; well location and operating parameters are determined through field measurements and development planning, including the three-dimensional spatial coordinates of water injection wells, gas injection wells, and production wells, the injection rate recorded hourly by the wellhead flowmeter of each well, the injection time set according to the block development plan, and the well group allocation plan formulated based on the well network type and inter-well connectivity test results; injection parameters are obtained through laboratory configuration, field sampling and testing, and metering statistics, including polymer concentration and the cumulative injection volume of polymer and gas, and carbon dioxide is selected as the injection gas based on the reservoir characteristics of the block; economic evaluation parameters are determined with reference to market quotations, industry standards, and historical data, including real-time crude oil prices, polymer procurement costs, gas processing costs, benchmark discount rates and investment payback periods set according to corporate investment requirements, and single-well operating costs and tax rates calculated based on historical operating data and tax policies.

[0023] After the collected input parameters are organized, they are entered into the Computer Modeling Group Numerical Simulator (CMG) to complete simulator initialization. The initialized numerical simulator can simulate reservoir state changes (such as formation pressure distribution, remaining oil distribution, and saturation changes at different development stages) and development performance indicators (such as daily oil production, water cut, and stage recovery factor) during the development of the oilfield block based on the input parameters.

[0024] Step 102: Generate a first data set based on a numerical simulator and a particle swarm optimization algorithm.

[0025] In this embodiment, a particle swarm optimization algorithm is initiated and several particles are initialized, each corresponding to a complete set of input parameter combinations (i.e., a set of parameter solutions for polymer injection and gas development). The parameters of the particle swarm optimization algorithm are configured: the inertia weight is initially set to 0.9, decreasing linearly to 0.4 with the number of iterations. The learning factor is fixed at 2.0, and the maximum number of iterations is set to 80 to balance the algorithm's early global search capability with the accuracy of local optimization in the later stages.

[0026] In each round of iteration, the speed and position of each particle are calculated and updated based on the historical movement speed of each particle, the particle's own historical optimal position (that is, the parameter combination with the best development effect in the past iteration of the particle), and the global optimal position of all particles (that is, the parameter combination with the best development effect in the past iteration of all particles). The parameter combination after the iteration (that is, the input parameters corresponding to the particle after the iteration) is obtained.

[0027] If a parameter (such as polymer concentration) is lower than the lower limit of the parameter dimension (such as 800 mg per liter), it will be corrected to the lower limit value; if a parameter (such as gas injection rate) is higher than the upper limit of the parameter dimension (such as 8000 cubic meters per day), it will be corrected to the upper limit value.

[0028] The modified parameter combinations from each iteration are fed into an initialized numerical simulator. The simulator simulates the oilfield development process under the influence of that parameter combination and outputs the corresponding reservoir state change data and development effect indicators. The corresponding data of "input parameter-reservoir state change data-development effect indicator" obtained from all iterations are summarized to form the first data set.

[0029] Step 103: Perform screening and data enhancement processing on the first data set to obtain a second data set.

[0030] In this embodiment, development effect index thresholds are preset. Based on the development requirements of the medium-to-high permeability heterogeneous oilfield block, the recovery factor threshold is set to no less than 25%, and the daily oil production threshold is set to no less than 50 cubic meters. The input parameters corresponding to each data set are also verified to ensure compliance with physical and engineering constraints. For example, the polymer concentration must be between 800 and 2500 milligrams per liter, the injection pressure must not exceed 90% of the formation fracture pressure, and the injection rate must not exceed 95% of the rated capacity of the injection and production equipment. Data that meets both the development effect index thresholds and the parameter constraints are screened out to obtain a first filtered data set.

[0031] For development data with time steps greater than three days (i.e., the time interval between two consecutive data records exceeds three days), linear interpolation is used to supplement reservoir state change data (such as formation pressure distribution and remaining oil distribution) and input parameter data (such as polymer concentration and gas injection rate) between adjacent time steps, ensuring a uniform time step interval of 1-2 days for all data and improving the temporal continuity of the data. Furthermore, random perturbations of preset amplitudes are applied to the input parameters, such as ±8%-15% perturbations for polymer concentration, ±5%-10% perturbations for gas injection rate, and ±2%-5% perturbations for injection time. After the perturbations, the parameters are re-verified for compliance with physical and engineering constraints. Perturbed data that pass the verification are added to the filtered first dataset as new data, ultimately forming a second dataset containing multiple expert trajectories. Each expert trajectory fully characterizes the dynamic evolution of reservoir state over time after a set of injection parameters is applied to the reservoir (e.g., the continuous process of formation pressure and remaining oil distribution changing with injection parameters from early to mid-development).

[0032] In step 104, multiple expert trajectories are respectively input into a preset inverse reinforcement learning model to obtain the update amplitudes of the policy network in the inverse reinforcement learning model and the probability values ​​of the discriminator in the inverse reinforcement learning model corresponding to the multiple expert trajectories. When the update amplitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold, the expert trajectory with the smallest update amplitude and the largest probability value is determined as the target expert trajectory.

[0033] In an embodiment of the present application, the strategy network adopts a multi-layer fully connected neural network structure to learn the optimization strategy of reservoir development parameters; the discriminator also adopts a multi-layer fully connected neural network structure to determine whether the input trajectory conforms to the expert-level development logic (that is, whether it is a high-quality development parameter trajectory).

[0034] For each expert trajectory, during model operation, the policy network adjusts its own network parameters based on the reservoir status and parameter information in the trajectory, and records the update amplitude of the policy network parameters (that is, the degree of change before and after the network parameter adjustment); at the same time, the discriminator determines the "expert attributes" of the trajectory and outputs the probability value of the trajectory belonging to the expert trajectory (the higher the probability value, the closer the development parameter optimization logic of the trajectory is to the expert decision-making logic).

[0035] In combination with the model training convergence requirements, the policy network update amplitude threshold is set to 10 to the negative power of 4 (that is, when the parameter update amplitude is less than this value, it indicates that the policy network's learning of the trajectory has stabilized), and the discriminator probability value threshold is set to 0.48 (that is, when the probability value is greater than this value, it indicates that the similarity between the trajectory and the expert trajectory meets the standard).

[0036] Filter out the expert trajectories whose "policy network update amplitude is less than the preset threshold and the discriminator probability value is greater than the preset threshold". Among these qualified trajectories, further compare the policy network update amplitude and the discriminator probability value, and finally determine the expert trajectory with the "minimum policy network update amplitude and maximum discriminator probability value" and use it as the target expert trajectory (the parameter optimization logic corresponding to this trajectory is the most stable, closest to expert decision-making, and has the best development effect).

[0037] Step 105 , optimizing the input parameters according to the target expert trajectory to obtain the optimal input parameters.

[0038] In an embodiment of the present application, the correlation between the injection parameters and the reservoir status and development effect in the target expert trajectory is analyzed. For example, at a certain development stage in the trajectory, the adjustment rule of the polymer concentration with the change of the remaining oil distribution, the matching rule of the gas injection rate and the formation pressure change, etc.

[0039] Based on the above rules, the initially collected reservoir development input parameters are optimized and adjusted. If the polymer concentration in the initial injection parameters deviates from the optimal concentration under similar reservoir conditions in the target expert trajectory at a certain development stage, it is adjusted to the adaptive concentration with reference to the trajectory rules. If the formation pressure change corresponding to the initial gas injection rate does not conform to the optimal pressure change trend in the trajectory, the gas injection rate is adjusted with reference to the trajectory rules.

[0040] After the adjustment is completed, the optimized input parameters are input into the numerical simulator again for verification to confirm that the corresponding development effect indicators (such as recovery rate, daily oil production, and net present value) meet the development goals of the oil field block, and there is no risk of wellbore blockage or excessive injection cost. Finally, the optimal input parameters for reservoir development are obtained.

[0041] The embodiment of the present application realizes the intelligent and systematic parameter optimization through the collaborative architecture of "numerical simulator + particle swarm optimization + inverse reinforcement learning". By using expert trajectories to characterize the dynamic relationship of "injection parameters-reservoir state-time", combined with the implicit learning of expert experience by the inverse reinforcement learning model, the optimization strategy is more adaptable to complex conditions such as reservoir heterogeneity and fluid dynamic changes, and can more accurately match the reservoir needs of the entire development cycle than traditional methods. By screening the target expert trajectory with "the smallest update amplitude of the strategy network and the largest discriminator probability value", it is ensured that the optimized input parameters can not only meet the physical and engineering constraints of reservoir development, but also maximize core indicators such as recovery rate and net present value, and balance the development effect and economic benefits.

[0042] In the above Figure 1 Based on the above, the present application also provides a schematic diagram of the implementation process of processing input parameters, such as Figure 2 As shown, it includes steps 201 to 203: Step 201 : normalize the input parameters to obtain normalized input parameters.

[0043] In the embodiments of the present application, the parameter categories that need to be normalized are distinguished according to the type and physical meaning of the input parameters, mainly for injection parameters (such as polymer concentration, gas injection rate, cumulative polymer injection volume, and cumulative gas injection volume). Since such parameters have large dimensional differences (such as concentration units are milligrams per liter and rate units are cubic meters per day), direct use in subsequent particle swarm optimization algorithms or model training can easily lead to parameter weight imbalance, and dimensional effects need to be eliminated through normalization.

[0044] To perform this operation, the actual range of values ​​for each injection parameter in the historical development data for the oilfield block is first calculated. For example, the historical minimum and maximum values ​​for polymer concentration are 800 mg / L and 2500 mg / L, respectively; the historical minimum and maximum values ​​for gas injection rate are 1500 cubic meters per day and 8000 cubic meters per day, respectively. For each injection parameter, the original data is calculated against the historical minimum and maximum values, uniformly mapping the original data to a range between zero and one. For example, if the original polymer concentration value is 1600 mg / L, this value is converted to the corresponding value within the range between zero and one, ensuring that injection parameters of different dimensions can be used in the subsequent optimization calculations on the same scale.

[0045] For oilfield geological parameters (such as permeability and porosity), well location and operation parameters (such as well coordinates and injection time), and economic evaluation parameters (such as crude oil price and operating costs), normalization processing is not performed for the time being because their original characteristics of physical or economic significance need to be retained in the future. Only the original values ​​are recorded for subsequent steps.

[0046] Step 202 : Perform outlier processing on the input parameters, replace outliers in the input parameters with boundary values, and supplement missing values ​​through linear interpolation and mean filling, to obtain input parameters after outlier processing.

[0047] In the embodiment of the present application, for oilfield geological parameters, if the permeability data appears to be negative (not in line with physical meaning) or the porosity data is greater than 40% (exceeding the conventional range of reservoir porosity in the block), it is determined to be an outlier; the permeability outlier value is directly replaced by the lower limit value of the permeability of similar reservoirs in the block (such as 50 millidarcy), and the porosity outlier value is replaced by the upper limit value of the porosity of the block (such as 30%).

[0048] For injection parameters, if the polymer concentration is lower than 500 mg / L (lower than the lower limit of effective displacement concentration) or higher than 3000 mg / L (exceeding the upper limit of equipment configuration), or the gas injection rate is higher than 10,000 cubic meters per day (exceeding the rated capacity of the gas injection equipment), it is judged as an abnormal value; the abnormal value of polymer concentration is replaced with 500 mg / L or 3000 mg / L, and the abnormal value of gas injection rate is replaced with 10,000 cubic meters per day.

[0049] For economic evaluation parameters, if the crude oil price is lower than 50% of the lowest market quotation during the same period, and the operating cost of a single well is higher than twice the historical average cost, it will be judged as an outlier; the outlier crude oil price will be replaced with the lowest market quotation during the same period, and the outlier single well operating cost will be replaced with twice the historical average cost.

[0050] Differentiated methods are used to fill missing values ​​depending on the parameter type. For oilfield geological parameters (such as permeability and porosity), if permeability data for a particular well is missing, permeability data from three adjacent wells in the same formation are obtained and the permeability value for the missing well is calculated using linear interpolation. For example, if the permeabilities of the three adjacent wells are 60, 70, and 80 millidarcy, respectively, the permeability of the missing well is calculated to be 65 millidarcy through interpolation.

[0051] For production operation parameters (such as daily oil production and gas injection rate), if the gas injection rate data for a certain day is missing, the average daily gas injection rate of the seven days before the missing parameter is calculated, and the average value is used as the supplementary data for the missing value; for example, the gas injection rates in the seven days before the missing parameter are 4,500 cubic meters per day, 4,600 cubic meters per day, and so on, 4,800 cubic meters per day, and the calculated daily average rate is 4,650 cubic meters per day, which is used to supplement the missing data.

[0052] For economic evaluation parameters (such as polymer procurement costs), if the procurement cost data for a certain quarter is missing, the average procurement cost of the other three quarters of the year will be used to fill in the gaps to ensure the integrity of the economic parameters.

[0053] Through the above-mentioned outlier and missing value processing, a set of input parameters without abnormalities or missing values ​​is obtained.

[0054] Step 203: Encode the input parameters into an action vector to obtain a parameter vector.

[0055] In the embodiment of the present application, according to the parameter input requirements of the subsequent particle swarm optimization algorithm, the preprocessed input parameters are encoded into structured action vectors (i.e., parameter vectors) according to the "development decision dimension", ensuring that each vector corresponds to a complete set of development parameter combinations. The specific encoding logic is as follows: Taking into account the key decision parameters for reservoir development, the parameter vector is set into six dimensions, corresponding to "polymer concentration", "gas injection rate", "cumulative polymer injection volume", "cumulative gas injection volume", "injection-production cycle" and "daily water injection volume of a single well".

[0056] The preprocessed input parameters are mapped one by one into a vector in dimensional order. For example, consider a set of input parameters: polymer concentration of 1500 mg / L, gas injection rate of 5000 m3 / day, cumulative polymer injection volume of 100,000 m3, cumulative gas injection volume of 150,000 m3, injection-production cycle of 300 days, and daily water injection volume per well of 2000 m3. These parameters are sequentially mapped into the six dimensions to form a six-dimensional parameter vector.

[0057] After coding is completed, check whether the parameters of each dimension still meet the physical and engineering constraints (for example, whether the polymer concentration is within the range of 500 to 3000 mg / L). If there is a parameter deviation caused by coding errors, correct it in time and determine the final parameter vector for subsequent particle initialization of the particle swarm optimization algorithm.

[0058] The embodiments of this application eliminate dimensional differences in injection parameters (such as polymer concentration and gas injection rate) through normalization, thus avoiding parameter weight deviations during particle swarm optimization or model training due to dimensional imbalance. By replacing outliers (such as correcting physically unreasonable values ​​to boundary values) and supplementing missing values ​​(linear interpolation and mean filling), the common problems of "data incompleteness and outlier interference" in historical data are addressed, ensuring the integrity and accuracy of input parameters. The preprocessed parameters are encoded as fixed-dimensional action vectors (parameter vectors), allowing the particle swarm optimization algorithm to directly call standardized parameter combinations for search. This ensures that the input format received by the inverse reinforcement learning model is uniform, avoiding algorithm interruptions or model training deviations caused by non-standard parameter formats, and improving the smoothness and reliability of the subsequent optimization process.

[0059] In some embodiments, generating a first data set based on a numerical simulator and a particle swarm optimization algorithm includes: initializing a number of particles, each particle representing a set of parameter vectors, and configuring an inertia weight, a learning factor, and a maximum number of iterations of the particle swarm optimization algorithm.

[0060] Specifically, the number of particles in the particle swarm optimization algorithm was determined based on the development parameter search requirements for this oilfield block. Considering the strong reservoir heterogeneity within the block (permeability coefficient of variation of 0.85), a wide range of parameter combinations needed to be covered. Therefore, 50 particles were initialized, each of which uniquely corresponds to a set of six-dimensional parameter vectors (i.e., a complete set of polymer and gas injection development parameter solutions).

[0061] The initial parameter vectors for the particles are determined based on historically high-quality development plans for the block and engineering constraints. For example, the initial range for polymer concentration is 800 to 2500 mg / L, the injection rate is 1500 to 8000 cubic meters per day, and the injection-production cycle is 180 to 730 days. The remaining parameters are randomly assigned within the feasible ranges of similar projects to ensure that the initial particles cover all key parameter combinations.

[0062] The core parameters of the particle swarm optimization algorithm were configured based on the search accuracy and efficiency requirements for reservoir parameter optimization. The inertia weight was initially set to 0.9 and linearly decreased to 0.4 with the number of iterations. Maintaining a high inertia weight in the early stages of the iteration (the first thirty rounds) enhanced the algorithm's global search capability across the parameter space and avoided missing potentially high-quality parameter combinations. Lowering the inertia weight in the later stages of the iteration (the last fifty rounds) focused on localized high-quality parameter regions for refined optimization, improving parameter accuracy.

[0063] Two learning factors are set, corresponding to the learning weights of the particle's "individual historical optimal position" and "global optimal position," respectively. Both are fixed at 0.2. This ensures that particles can fully learn from their own optimal parameter experience from past iterations while simultaneously moving towards the global optimal parameters of all particles, balancing individual exploration and group collaboration.

[0064] Considering the block's parameter dimensions (six dimensions) and the number of particles (50), we set a maximum number of iterations to 80. Preliminary testing has shown that 80 iterations allows the algorithm to achieve a balance between global search and local optimization, avoiding both missing the optimal solution due to insufficient iterations and wasting computational resources due to excessive iterations.

[0065] Furthermore, several examples are iterated. In each round of iteration, the particle velocity and position are updated according to the particle historical velocity, individual historical optimal position, and global optimal position to obtain the iterated parameter vector.

[0066] Specifically, at the beginning of each iteration, a new velocity is calculated for each particle, combining its historical velocity, individual historical optimal position, and global optimal position. A particle's historical velocity refers to the rate of change of its parameter vector during the previous iteration, reflecting the particle's past search trends. Its individual historical optimal position refers to the location of the parameter vector that yielded the best development effect (recovery factor, net present value) for that particle since the start of the iteration. The global optimal position refers to the location of the parameter vector that yielded the best development effect for all particles since the start of the iteration.

[0067] During the calculation process, a learning factor is used to adjust the degree to which a particle learns from both its individual experience and its collective experience. For example, if the parameter adjustment corresponding to a particle's velocity in the previous round was small, and its individual historical optimal position differed significantly from the global optimal position, the learning factor will amplify the learning weight of the global optimal position, guiding the particle toward a more optimal parameter region and ensuring that the particle's velocity adapts to the current search requirements after each iteration.

[0068] Based on the updated particle velocity, each particle's position in the parameter space is adjusted to generate the iterated parameter vector. The particle's current parameter vector is then superimposed with the parameter adjustment corresponding to the updated velocity to generate a new parameter vector (i.e., the particle's new position). For example, if the "polymer concentration" in a particle's current parameter vector is 1500 mg / L, and the updated velocity dimension requires an increase of 100 mg / L, then the parameter for that dimension becomes 1600 mg / L after the iteration. The remaining dimensions are adjusted using the same logic, ultimately forming the complete iterated parameter vector.

[0069] In each round of iteration, 50 particles complete the velocity and position update according to the above logic, generating 50 sets of iterated parameter vectors.

[0070] Furthermore, the parameter vector after iteration is subjected to boundary checking, and is corrected to the lower boundary value when the parameter vector is lower than the lower boundary of the dimension, and is corrected to the upper boundary value when the parameter vector is higher than the upper boundary of the dimension.

[0071] Specifically, since the particle velocity update may cause the dimensions of some parameter vectors to exceed the engineering or physical constraints, it is necessary to perform boundary checking and correction on all parameter vectors after iteration to ensure that the parameters input into the numerical simulator meet the actual development requirements. Based on the equipment capabilities, reservoir characteristics and development specifications of the oil field block, the upper and lower bounds of the six-dimensional parameter vectors are preset. The lower limit of polymer concentration is 800 mg / L (below this value, the displacement efficiency decreases significantly), and the upper limit is 2500 mg / L (above this value, it is easy to cause wellbore blockage); the lower limit of gas injection rate is 1500 cubic meters per day (below this value, it is impossible to effectively establish displacement pressure), and the upper limit is 8000 cubic meters per day (above this value, it exceeds the rated capacity of the gas injection equipment); the lower limit of cumulative polymer injection volume is 50,000 cubic meters (below this value, it is impossible to form an effective plugging), and the upper limit is 300,000 cubic meters (above this value, the economic efficiency decreases); the lower limit of cumulative gas injection volume is 100,000 cubic meters (below this value, the displacement range is insufficient), and the upper limit is 500,000 cubic meters (above this value, it is easy to cause gas escape); The lower limit of the injection-production cycle is 180 days (below this value, parameter adjustments are too frequent), and the upper limit is 730 days (above this value, it is impossible to adapt to dynamic changes in the reservoir in a timely manner); the lower limit of the daily water injection rate of a single well is 1,000 cubic meters (below this value, it is impossible to maintain formation pressure), and the upper limit is 3,000 cubic meters (above this value, it is easy to cause increased formation water production).

[0072] If a certain dimension parameter is lower than the lower limit (such as the polymer concentration is 700 mg / L), the dimension parameter is directly corrected to the corresponding lower limit value (800 mg / L); if a certain dimension parameter is higher than the upper limit (such as the gas injection rate is 8500 cubic meters per day), it is corrected to the corresponding upper limit value (8000 cubic meters per day); If the parameter is within the bounds, the original value remains unchanged.

[0073] Furthermore, the parameter vector is input into a numerical simulator to obtain a first data set.

[0074] Specifically, the boundary check and corrected parameter vectors (50 sets per iteration) are input one by one into the initialized CMG numerical simulator (the simulator has been loaded with basic data such as the geological model and fluid properties of the oil field block). Based on the input parameter vector, the 180-day reservoir development process under the action of this set of parameters is simulated, and the reservoir state change data at different time nodes (such as formation pressure distribution, remaining oil distribution, and saturation change) are calculated; the simulator automatically calculates the development effect indicators corresponding to this set of parameters according to the reservoir state changes, including cumulative oil production, stage water cut, stage recovery factor, net present value, etc.; the "input parameter vector-simulated reservoir state change data-development effect indicator" are linked one by one to form a complete trajectory data.

[0075] After all 80 iterations were completed, the trajectory data generated during all iterations were aggregated and duplicate data was removed, resulting in a first dataset containing 3,800 valid trajectory data points. This dataset fully characterized the correspondence between input parameters, reservoir state change data, and development effect indicators, providing a foundation for subsequent screening and data enhancement of the first dataset.

[0076] By rationally setting the number of particles (adapting to reservoir complexity) and algorithm parameters (linearly decreasing inertia weights and synergistic learning factors), the present embodiment enables the particle swarm optimization algorithm to cover a wide parameter space (avoiding the omission of potentially high-quality parameters) while also focusing on local optimization in the later stages of iteration (improving parameter accuracy). Combined with post-iteration boundary checking (correcting out-of-bounds parameters), this ensures that the generated parameter vectors conform to physical laws and engineering constraints, preventing invalid parameters from entering the numerical simulator. Through the closed-loop process of "iterative update-boundary checking-simulator calculation," each trajectory in the first dataset accurately reflects the causal relationship between "input parameters → reservoir state changes → development effect indicators." Compared to the traditional "random parameter generation + simulation" approach, this significantly improves the effectiveness of the dataset and provides high-quality basic data for subsequent screening, processing, and data enhancement.

[0077] In some embodiments, before inputting multiple expert trajectories into a preset inverse reinforcement learning model to obtain the update amplitude of the policy network in the inverse reinforcement learning model and the probability value of the discriminator in the inverse reinforcement learning model corresponding to the multiple expert trajectories, the method further includes: establishing an initial inverse reinforcement learning model, inputting historical input parameters into the policy network and discriminator of the initial inverse reinforcement learning model, and obtaining the initial update amplitude and initial probability value.

[0078] Specifically, the initial inverse reinforcement learning model adopts a "strategy network-discriminator" dual network architecture, both of which are built based on a fully connected neural network. The strategy network adopts a three-layer fully connected neural network structure. The input layer is a twelve-dimensional reservoir state vector (the dimensions correspond to formation pressure, porosity, permeability, remaining oil distribution, water content, oil production, recovery rate, net present value, polymer concentration, gas injection rate, injection-production cycle, and daily water injection volume per well), which is used to receive reservoir dynamic state data; the output layer is a six-dimensional injection parameter decision vector, which is used to output a development parameter plan adapted to the current reservoir state; the number of hidden layer nodes is set to 64 according to the complexity of the reservoir data in this block, and the activation function uses the ReLU function that can adapt to nonlinear parameter mapping to ensure that the network can learn the complex correlation between reservoir state and injection parameters.

[0079] The discriminator adopts a two-layer fully connected neural network structure. The input layer is the "reservoir state-injection parameter" combined trajectory fragment, which is used to receive the trajectory information to be judged; the output layer is a one-dimensional probability value, which is used to characterize the possibility that the input trajectory belongs to the "expert trajectory"; the number of hidden layer nodes is set to 40, and the Sigmoid function is used as the activation function to ensure that the output probability value is stable in the range of zero to one, which is convenient for subsequent trajectory judgment and model training.

[0080] Using the "Xavier initialization" method, the initial weight matrix is ​​randomly generated based on the mean and variance of the input and output dimensions of each layer of the network. For example, the weight matrix from the input layer to the hidden layer of the policy network is calculated based on the dimensions of the 12-dimensional input layer and the 64-node dimension of the hidden layer, avoiding model training instability caused by excessively large or small initial weights. The bias parameters of all layers are initialized to 0.1 to ensure that each neuron has basic activation ability in the initial state of the network and reduce the convergence delay in the early stage of training; The optimizer is preset for subsequent training. Both use the Adam optimizer, and the initial learning rate is set to 0.01%, taking into account both training stability and convergence speed.

[0081] At this point, the initial inverse reinforcement learning model is built. The model can receive input data and output preliminary calculation results.

[0082] Historical development data from the past five years was selected from the development database of the oilfield block as the input data for the initial model. The screening criteria were that the recovery factor corresponding to the historical data was no less than 25% and the net present value was no less than the industry average for the same period, ensuring that the input data had basic reference value. The data contained complete reservoir status data (formation pressure, remaining oil distribution, etc.) and corresponding injection parameter data (polymer concentration, gas injection rate, etc.), with no key dimensions missing. The data covered a continuous development cycle of at least 360 days, forming a complete "reservoir status-injection parameter" time series trajectory to adapt to the model's time series learning requirements.

[0083] Finally, 20 groups of historical input parameters that meet the requirements (each group contains 360 consecutive days of time series data) were selected as the input source of the initial model.

[0084] Each of the 20 sets of historical input parameters was fed into the initial inverse reinforcement learning model, and the following calculations were performed. After the reservoir state data from the historical input parameters was fed into the policy network, the network output an injection parameter decision vector based on the initial weights. This output vector was compared with the actual injection parameters from the historical input parameters, the difference between the two was calculated, and the network weights were adjusted accordingly. The degree of change before and after the weight adjustment is the initial update amplitude. For example, if the polymer concentration output by the policy network differed significantly from the actual value after a certain set of historical parameters was input, the network weight adjustment amplitude would increase accordingly, and the initial update amplitude would also increase, reflecting the network's current learning bias for that parameter.

[0085] After the "reservoir state-injection parameter" trajectory segment in the historical input parameters is input into the discriminator, the discriminator determines the "expert attributes" of the trajectory based on the initial weights and outputs an initial probability value. For example, if a set of historical trajectories corresponds to excellent development results, the initial probability value output by the discriminator is close to 0.6, indicating that the initial model can initially identify high-quality trajectories, but there is still a judgment bias, which requires subsequent training and optimization.

[0086] Furthermore, a plurality of preset expert trajectories are obtained from the expert trajectory library, and the discriminator is trained according to the obtained plurality of preset expert trajectories and the initial probability value to obtain a trained discriminator, and the policy network is trained according to the obtained plurality of preset expert trajectories and the initial update amplitude to obtain a trained policy network.

[0087] Specifically, pre-set expert trajectories were retrieved from the oilfield's expert trajectory library. This library consists of high-quality development plans for the oilfield over the past decade, benchmark development cases for similar reservoirs within the industry, and optimal parameter trajectories verified by numerical simulators. The selection criteria were: the corresponding trajectory demonstrated a minimum 5% improvement in ultimate recovery and a minimum 10% improvement in net present value compared to conventional plans; the corresponding reservoir geology (permeability, porosity, and heterogeneity) must be at least 80% similar to the block in this example to ensure the adaptability of the training data; and each trajectory contained data from the complete development cycle (730 days), encompassing reservoir state changes and injection parameter adjustments during the initial, mid-, and final stages of development, covering parameter optimization requirements throughout the entire development phase. Finally, 30 pre-set expert trajectories that met the requirements were obtained from the expert trajectory library and used as the core data for model training.

[0088] With the goal of "improving the discriminator's recognition accuracy of expert trajectories", training is carried out by combining the preset expert trajectories and initial probability values. The steps are as follows: Thirty pre-set expert trajectories were split into segments based on time steps. Thirty non-expert trajectories were randomly selected from the second dataset and similarly split into segments. The two segments were mixed in a one-to-one ratio to form the training dataset. The trajectory segments from the training dataset were fed into the discriminator one by one. Using the initial probability values ​​as a benchmark, the discriminator's error in classifying expert segments as expert and non-expert segments as non-expert was calculated. If the discriminator's probability output for a particular expert trajectory segment was lower than 0.5, the discriminator's weights and bias were adjusted based on this error to reduce the error in subsequent similar classifications. Each training round processed two hundred trajectory segments, for a total of fifty rounds, with the discriminator's performance verified every ten rounds.

[0089] When the discriminator's accuracy in identifying expert trajectory segments stabilizes above 90% and its accuracy in rejecting non-expert trajectory segments stabilizes above 85%, training is terminated, resulting in the trained discriminator. At this point, the discriminator can accurately distinguish between expert and non-expert trajectories, and the output probability values ​​reliably represent the "expert attributes" of the trajectories.

[0090] With the goal of "making the injection parameter trajectory output by the policy network close to the expert trajectory", training is carried out by combining the preset expert trajectory and the initial update amplitude. The steps are as follows: The reservoir state data of thirty preset expert trajectories were associated with the corresponding injection parameter data to form a "reservoir state-expert injection parameter" paired dataset, where the reservoir state data served as the input of the strategy network and the expert injection parameters served as the target benchmark of the network output.

[0091] The reservoir status data in the paired dataset is input into the strategy network, and the network outputs an injection parameter decision vector. The difference between this output vector and the target benchmark is compared, and the network parameter update error is calculated based on the initial update amplitude. The gradient descent method is used to adjust the weights and bias of the strategy network based on the error, so that the network output gradually approaches the expert injection parameters. Fifty sets of paired data are processed in each round of training, and a total of sixty rounds of training are carried out. The network output effect is verified by a numerical simulator every ten rounds.

[0092] When the difference between the simulator development indicators corresponding to the policy network's output parameters and those corresponding to the expert trajectories is less than 3%, and the network weight update amplitude is consistently below 30% of the initial update amplitude, training is terminated, resulting in the trained policy network. At this point, the policy network can output injection parameter solutions close to expert levels based on reservoir conditions, meeting the requirements for subsequent expert trajectory input and target trajectory screening.

[0093] The embodiment of the present application obtains the initial update amplitude and initial probability value by inputting historical parameters, so that the model has basic parameter learning capabilities before formal training, avoiding the slow convergence problem caused by "training from scratch"; with the help of preset expert trajectories, the discriminator and policy network are trained in a targeted manner, allowing the model to learn expert-level development logic in advance and reduce the number of training iterations. Through the two-step process of "historical parameter initialization → expert trajectory training", the discriminator can more accurately distinguish between expert trajectories and non-expert trajectories, and the policy network can more accurately learn the mapping law of "reservoir state → injection parameter", avoiding excessive deviations in the update amplitude and probability value calculations when the expert trajectory is subsequently input due to poor initial model performance, providing reliable model support for the accurate screening of target expert trajectories.

[0094] In some embodiments, the input parameters are optimized according to the target expert trajectory to obtain the optimal input parameters, and the target expert trajectory is also subjected to trajectory legitimacy verification, and when the legitimacy of the target expert trajectory exceeds a preset legal range, the target expert trajectory is subjected to legitimacy correction.

[0095] Specifically, combining the physical laws of polymer and gas injection development, engineering equipment constraints, and oilfield development specifications, we determined three core dimensions for validating the target expert trajectory: the physical feasibility of injection parameters, the safety margin of reservoir development, and the rationality of economic indicators. Preset legal ranges were set for each dimension, as follows: The polymer concentration must be within the range of "effective displacement concentration lower limit - equipment configuration upper limit," that is, no less than 800 mg / L and no more than 2500 mg / L; The gas injection rate must be within the range of "lower limit of displacement pressure establishment - upper limit of gas injection equipment rating", that is, no less than 1,500 cubic meters per day and no more than 8,000 cubic meters per day; The gas-liquid ratio must be within the "reasonable range of miscible displacement", that is, not less than 0.3 and not more than 1.2.

[0096] The formation pressure must be within the range of "upper limit of formation fracture pressure - lower limit of bottom hole flow pressure", that is, no higher than 90% of the formation fracture pressure and no lower than 1.2 times of the bottom hole flow pressure; The increase in moisture content in a single month shall not exceed 5%; The decrease in residual oil saturation in a single month shall not exceed 1%.

[0097] The net present value must be no less than 95% of the target net present value of the development of the block; The investment payback period must not exceed five years; The unit oil production cost must not exceed 110% of the industry average unit oil production cost during the same period.

[0098] The logic of "time series segmentation verification - dimension item-by-item verification - anomaly marking" is used to verify the legitimacy of the full development cycle data of the target expert trajectory. The specific steps are as follows: Considering the differences in parameter characteristics and constraints at different stages of reservoir development, the 730-day development cycle of the target expert trajectory was divided into three time periods: early development, mid-development, and late development. Verification was then performed separately. For example, in the early development phase, the focus was on verifying whether formation pressure met safety limits, while in the late development phase, the focus was on verifying whether economic indicators met standards.

[0099] For the trajectory data of each time series segment, we verify them one by one in the order of "physical feasibility of injection parameters → safety margin of reservoir development → rationality of economic indicators": Extract daily data on polymer concentration, injection rate, and gas-to-liquid ratio within the time series and compare them with the pre-set legal range. For example, on a certain day in the middle of development, the polymer concentration in the trajectory was 2700 mg / L, exceeding the upper limit of "no more than 2500 mg / L". This data was marked as "injection parameter physical feasibility anomaly"; Extract daily formation pressure, water cut increase, and remaining oil saturation decrease data within the time series and compare them with the preset legal range. For example, in a certain month during the early stages of development, if the water cut in the trajectory increases by 6% per month, exceeding the upper limit of "no more than 5%", the data will be marked as "reservoir development safety boundary anomaly"; Net present value, payback period, and unit oil production cost data for each quarter are extracted and compared to pre-set legal ranges. For example, in a late-stage development quarter, the unit oil production cost in the trajectory exceeded the industry average by 115%, exceeding the upper limit of "no more than 110%." This data was marked as "economic indicator rationality anomaly."

[0100] After completing the full time-series and full-dimensional verification, all abnormal data are summarized and the legality status is divided into three categories according to the "abnormal impact range": "completely legal", "partial abnormality (legal range)", and "serious abnormality (outside the legal range)". Completely legal, no dimensional data exceeds the preset legal range, and the trajectory can be directly used for parameter optimization; Local anomaly (legal range): Only a single dimension of data in a single time series segment exceeds the preset legal range, and the magnitude of the anomaly is small, so the impact on overall development is controllable; Severe anomalies (outside the legal range) mean that multiple dimensional data in at least two time series segments exceed the preset legal range, or a single abnormal data exceeds the upper limit by more than 10%, which has a significant negative impact on development results, reservoir safety or economic benefits.

[0101] Based on the anomaly types discovered during verification, and combining reservoir development rules with expert experience, we use the logic of "parameter adaptation and correction - chain impact compensation - effect verification" to carry out corrections. This ensures that the corrected trajectory is within the legal range while retaining the high-quality development logic of the original trajectory. The specific steps are as follows: To address mid-development anomalies where polymer concentrations exceeded the upper limit, a "gradient reduction correction" approach was employed. This correction benchmark was set, using the upper limit of the pre-set legal range as the target value. The trend of polymer concentration changes during the anomaly period was analyzed. The original trajectory showed a slowly decreasing concentration of 2700 mg / L on the first day of the anomaly period, gradually decreasing to 2600 mg / L over the following nine days. Gradient adjustment parameters were used to adjust the concentration to 2500 mg / L on the first day of the anomaly period, then decreasing by 50 mg / L per day over the following nine days, ultimately to 2500 mg / L. This ensured that the corrected concentration remained within the legal range of 800 to 2500 mg / L. Correlated parameters were also adjusted in a coordinated manner, as a decrease in polymer concentration could lead to a decrease in displacement efficiency. The gas injection rate was then adjusted incrementally from 7000 cubic meters per day in the original trajectory to 7500 cubic meters per day over the ten days following the concentration correction. This increased gas displacement intensity compensated for the loss in displacement efficiency caused by the decrease in polymer concentration.

[0102] In response to the abnormality of unit oil production cost exceeding the upper limit in the late stage of development, the "cost-output coordinated optimization" method is adopted: Tracing back the trajectory data, we found that the abnormal unit oil production cost was caused by a temporary increase in polymer procurement costs, which caused the cost to exceed the upper limit; While ensuring that the recovery rate does not decline, the injection volume of high-cost polymers will be reduced. The cumulative polymer injection volume in the late development period will be reduced from the original trajectory of 300,000 cubic meters to 280,000 cubic meters. At the same time, by optimizing the injection-production cycle and reducing the operating time of each well, labor and maintenance costs will be reduced.

[0103] When the revised parameters were input into the numerical simulator, simulation calculations showed that the unit oil production cost dropped to 105% of the industry average, while the net present value remained at 98% of the target value, and the recovery rate only dropped by 0.3%.

[0104] All target expert trajectories after abnormal correction are re-checked according to the dimensions of "physical feasibility of injection parameters → safety boundary of reservoir development → rationality of economic indicators" to confirm that no data exceeds the preset legal range; the difference in development effect indicators between the corrected trajectory and the original trajectory is less than one percent; and the corrected injection parameters are feasible on site.

[0105] Based on the revised legal target expert trajectory, the optimal injection parameter combination for each development stage in the trajectory is extracted, and adaptive adjustments are made based on the current actual reservoir status of the block to obtain the final optimal input parameters.

[0106] The above optimal input parameters were input into a numerical simulator for verification. The results showed that the recovery factor increased by 2.5% compared with the pre-optimization level, the net present value met the requirements, and there were no safety and engineering constraints. These parameters can be used as the final implementation parameters for polymer injection and gas development in this block.

[0107] The embodiments of this application utilize multi-dimensional verification of "physical feasibility of injection parameters, safety margins for reservoir development, and rationality of economic indicators" to avoid the problem of target expert trajectories merely satisfying theoretical optimization results while violating physical laws, exceeding safety limits, or failing to meet economic benefit standards, thereby ensuring that the optimization results are suitable for field implementation. Targeted corrections are made to trajectories that exceed the legal range, ensuring that the corrected target expert trajectories retain the original high-quality development logic while complying with engineering constraints. This avoids wellbore blockage, formation damage, or economic losses caused by parameter violations, thereby improving the safety and economic efficiency of field development.

[0108] In some embodiments, the first data set is screened and data enhanced to obtain a second data set, including: screening the first data set to obtain data whose development effect indicators are greater than a preset threshold and whose injection parameters meet physical and engineering constraints, to obtain the screened first data set.

[0109] Specifically, the screening process aims to "retain high-quality development data and eliminate invalid / illegal data," and is implemented based on the dual standards of "meeting development performance indicators" and "complying with injection parameters." The requirements include a recovery factor of no less than 25%, daily oil production of no less than 50 cubic meters, and a net present value of no less than 90% of the development target net present value for the block. Polymer concentration must be no less than 800 mg / L and no more than 2,500 mg / L; gas injection rate must be no less than 1,500 cubic meters per day and no more than 8,000 cubic meters per day; injection pressure must be no more than 90% of the formation fracture pressure; and gas-to-liquid ratio must be no less than 0.3 and no more than 1.2.

[0110] The screening was carried out using the logic of "track-by-track verification - double-standard judgment - data retention". The four thousand tracks in the first data set were split according to the principle of "independent verification of a single track". Each track was required to pass the double verification of development effect indicators and injection parameters separately; the recovery rate, daily oil production, and net present value data of a single track were extracted and compared with the preset thresholds. For example, if the recovery rate of a certain track is 24% and the daily oil production is 46 cubic meters, both indicators are lower than the threshold, and it is directly judged as "invalid data" and eliminated; if the recovery rate of a certain track is 26%, the daily oil production is 52 cubic meters, and the net present value is 92% of the target value, and all three indicators meet the standards, then the next step of injection parameter verification will be entered; for tracks that meet the development effect standards, the polymer concentration, gas injection rate, gas injection pressure, and gas-liquid ratio data of the entire cycle are extracted, and each parameter is compared with the constraint range. For example, the recovery rate of a certain trajectory meets the standard, but the polymer concentration on a certain ten-day period is 2700 mg / L and the injection pressure reaches 93% of the formation fracture pressure. These data are judged as "illegal data" and are discarded. If all injection parameters of a certain trajectory meet the constraint range throughout the entire cycle, they are judged as "valid data" and retained.

[0111] Furthermore, time step interpolation is performed on the filtered first data set and a preset amplitude disturbance is applied to the injection parameters to generate new data, thereby obtaining a second data set.

[0112] Specifically, data augmentation processing aims to "expand data volume and improve data diversity and continuity." To address the issues of "discontinuous time steps" and "insufficient parameter combination coverage" in the filtered data set, new data is generated using "time step interpolation" and "injection of parameter perturbations." In the filtered data set, some trajectories have time step intervals greater than three days due to the calculation step size setting of the numerical simulator. This results in insufficient temporal coherence between reservoir state changes and parameter adjustments, necessitating interpolation to supplement intermediate data: The data that need to be interpolated include "reservoir state change data" and "injection parameter data"; the "linear interpolation method" is used, that is, based on the known data of two adjacent time steps, the data of the missing time step in the middle is calculated according to the time interval ratio.

[0113] After interpolation, all trajectories are uniformly time-stepped to one to two days. For example, a trajectory originally time-stepped to three days is supplemented with one or two intermediate time steps; a trajectory originally time-stepped to two days remains unchanged.

[0114] In the filtered data set, some injection parameter combinations were duplicated or had a narrow coverage range. This required generating new parameter combinations through "parameter perturbation" to enrich the data diversity. Injection parameters that significantly impacted development outcomes and had room for adjustment were selected for perturbation, including polymer concentration, gas injection rate, and injection time.

[0115] After the perturbation generates new parameters, it is necessary to verify again whether they meet the "physical and engineering constraints of the injection parameters".

[0116] The new parameters after perturbation are associated with the reservoir state change data and development effect indicators of the original trajectory to generate a new trajectory. For example, after the parameters of an original trajectory are perturbed, 3-5 new trajectories are generated, and each new trajectory corresponds to a set of differentiated injection parameter combinations. Through time-step interpolation, the original 1,800 screened trajectories were supplemented with 2,200 time-series continuous trajectories. By injecting parameter perturbations, 3,500 trajectories with differentiated parameter combinations were added to the 2,200 trajectories. Ultimately, a second dataset of 5,700 valid trajectories was formed. This dataset retains the core characteristics of the original high-quality data while improving its continuity and diversity through interpolation and perturbation, fully supporting the subsequent inverse reinforcement learning model's learning of expert trajectory patterns.

[0117] This embodiment of the application addresses the problem of "uneven data quality" in the first dataset by screening data that "meets development effect indicators and injection parameters," eliminating invalid trajectories with poor development effects and out-of-bounds parameters. This ensures that subsequent model training is based only on high-quality data and reduces the interference of invalid data on model learning. Time-step interpolation improves the temporal continuity of trajectories, addressing the problems of "many time breakpoints and discontinuous reservoir state evolution" in traditional datasets. Injecting parameter perturbations increases the diversity of parameter combinations, avoiding the problem of "insufficient parameter coverage" in datasets leading to one-sided model learning.

[0118] Although this application provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on routine or non-creative work. The order of steps listed in this embodiment is only one way of executing the steps among many, and does not represent the only execution order. When an actual device or client product executes, the method shown in this embodiment or the accompanying drawings may be executed sequentially or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0119] like Figure 3As shown, the embodiment of the present application also provides a parameter optimization device 300 for reservoir development. The device includes: An acquisition module 301 is used to acquire input parameters during the oil reservoir development process. The input parameters include oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters. The input parameters are input into a numerical simulator to obtain an initialized numerical simulator. The numerical simulator is used to calculate reservoir state change data and development effect indicators during the oil field development process. The reservoir state change data includes at least one of formation pressure distribution, remaining oil distribution, and saturation change, and the development effect indicators include at least one of oil production, water cut, and recovery factor. A generating module 302 is configured to generate a first data set based on a numerical simulator and a particle swarm optimization algorithm, wherein the first data set is configured to characterize the corresponding relationship between input parameters, reservoir state change data, and development effect indicators; A processing module 303 is configured to perform screening and data enhancement on the first data set to obtain a second data set; Processing module 303 is further configured to: when the second data set includes multiple expert trajectories, the expert trajectories are used to characterize the dynamic process of the injection parameter acting on the reservoir and the evolution of the reservoir state over time; input the multiple expert trajectories into a preset inverse reinforcement learning model; obtain the update amplitudes of the policy network in the inverse reinforcement learning model and the probability values ​​of the discriminator in the inverse reinforcement learning model corresponding to the multiple expert trajectories; and when the update amplitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold, determine the expert trajectory with the smallest update amplitude and the largest probability value as the target expert trajectory; The optimization module 304 is used to optimize the input parameters according to the target expert trajectory to obtain the optimal input parameters.

[0120] In some embodiments, the processing module 303 is further configured to perform normalization processing on the input parameters to obtain normalized input parameters; The processing module 303 is further configured to perform outlier processing on the input parameters, replace outliers in the input parameters with boundary values, and supplement missing values ​​through linear interpolation and mean filling, thereby obtaining input parameters after outlier processing. The processing module 303 is further configured to encode the input parameters into an action vector to obtain a parameter vector.

[0121] In some embodiments, the generation module 302 is further used to initialize a number of particles, each particle representing a set of parameter vectors, and configure the inertia weight, learning factor, and maximum number of iterations of the particle swarm optimization algorithm; The processing module 303 is further used to iteratively process several examples. In each iteration, the particle velocity and position are updated according to the particle's historical velocity, the individual historical optimal position, and the global optimal position to obtain an iterated parameter vector. The processing module 303 is further configured to perform a boundary check on the parameter vector after the iteration, and correct the parameter vector to the lower boundary value when the parameter vector is lower than the lower boundary of the dimension, and correct the parameter vector to the upper boundary value when the parameter vector is higher than the upper boundary of the dimension; The processing module 303 is further configured to input the parameter vector into the numerical simulator to obtain a first data set.

[0122] In some embodiments, the generation module 302 is further configured to establish an initial inverse reinforcement learning model, input the historical input parameters into the policy network and the discriminator of the initial inverse reinforcement learning model, and obtain an initial update amplitude and an initial probability value; The acquisition module 301 is further used to obtain multiple preset expert trajectories from the expert trajectory library, train the discriminator based on the obtained multiple preset expert trajectories and the initial probability value to obtain a trained discriminator, and train the policy network based on the obtained multiple preset expert trajectories and the initial update amplitude to obtain a trained policy network.

[0123] In some embodiments, the processing module 303 is further configured to perform a trajectory legitimacy check on the target expert trajectory, and perform a legitimacy correction on the target expert trajectory when the legitimacy of the target expert trajectory exceeds a preset legal range.

[0124] In some embodiments, the processing module 303 is further configured to filter the first data set to obtain data whose development effect index is greater than a preset threshold and whose injection parameters meet physical and engineering constraints, thereby obtaining a filtered first data set. The processing module 303 is further configured to perform time step interpolation on the filtered first data set and apply a preset amplitude disturbance to the injection parameters to generate new data, thereby obtaining a second data set.

[0125] Some modules in the apparatus described herein may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0126] The apparatuses or modules illustrated in the above application examples can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above apparatuses are described as various modules with functions. In the implementation of the application examples, the functions of the modules can be implemented in one or more software and / or hardware. Of course, the modules with certain functions can also be implemented by a combination of multiple sub-modules or sub-units.

[0127] The methods, apparatuses or modules described in the present application can be implemented in a computer readable program code in any appropriate manner, for example, the controller can take the form of, for example, a microprocessor or a processor and a computer readable medium storing computer readable program code (such as software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit; abbreviated as: ASIC), programmable logic controllers and embedded microcontrollers, examples of the controller include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a pure computer readable program code, the same function can also be implemented by logically programming the method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, such a controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the devices for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0128] The embodiments of the present application also provide a device, which comprises: a processor; a memory for storing processor executable instructions; and the processor executes the executable instructions to implement the method as described in the embodiments of the present application.

[0129] The embodiments of the present application also provide a non-volatile computer readable storage medium, which stores a computer program or instructions, and when the computer program or instructions are executed, the method as described in the embodiments of the present application is implemented.

[0130] In addition, the functional modules in each of the embodiments of the present application can be integrated in one processing module, or each module can exist independently, or two or more modules can be integrated in one module.

[0131] The storage medium described above includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a cache, a hard disk (HDD), or a memory card. The storage medium can be used to store computer program instructions.

[0132] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary hardware. Based on such an understanding, the technical solutions of the present application can be embodied in the form of a software product or can be embodied in the form of data migration during implementation. The computer software product can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0133] The various embodiments in the specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. The whole or part of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, small computers, large computers, distributed computing environments including any of the above systems or devices, etc.

[0134] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the present application.

Claims

1. A parameter optimization method for oil reservoir development, characterized in that: include: Acquiring input parameters during the oil reservoir development process, the input parameters including oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters; inputting the input parameters into a numerical simulator to obtain an initialized numerical simulator; the numerical simulator is used to calculate oil reservoir state change data and development effect indicators during the oil field development process, the oil reservoir state change data including at least one of formation pressure distribution, remaining oil distribution, and saturation change; and the development effect indicator including at least one of oil production, water cut, and recovery factor; generating a first data set according to the numerical simulator and the particle swarm optimization algorithm, wherein the first data set is used to characterize the corresponding relationship between the input parameters, the reservoir state change data, and the development effect index; Performing screening and data enhancement on the first data set to obtain a second data set; The second data set includes a plurality of expert trajectories, which are used to characterize the dynamic process of the injection parameter acting on the reservoir and the evolution of the reservoir state over time. The plurality of expert trajectories are respectively input into a preset inverse reinforcement learning model to obtain the update amplitudes of the policy network in the inverse reinforcement learning model and the probability values ​​of the discriminator in the inverse reinforcement learning model corresponding to the plurality of expert trajectories. When the update amplitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold, the expert trajectory with the smallest update amplitude and the largest probability value is determined as the target expert trajectory; Parameter optimization is performed on the input parameters according to the target expert trajectory to obtain optimal input parameters.

2. The method according to claim 1, characterized in that After obtaining the input parameters in the reservoir development process, the following steps are also included: Normalizing the input parameters to obtain normalized input parameters; Performing outlier processing on the input parameters, replacing outliers in the input parameters with boundary values, and supplementing missing values ​​through linear interpolation and mean filling, to obtain input parameters after outlier processing; The input parameters are encoded into an action vector to obtain a parameter vector.

3. The method according to claim 2, characterized in that The generating of the first data set according to the numerical simulator and the particle swarm optimization algorithm includes: Initialize several particles, each particle represents a set of parameter vectors, and configure the inertia weight, learning factor, and maximum number of iterations of the particle swarm optimization algorithm; Perform iterative processing on several examples. In each round of iteration, the particle speed and position are updated according to the particle's historical speed, individual historical optimal position, and global optimal position to obtain the iterated parameter vector; Performing a boundary check on the parameter vector after the iteration, correcting the parameter vector to the lower boundary value when the parameter vector is lower than the lower boundary of the dimension, and correcting the parameter vector to the upper boundary value when the parameter vector is higher than the upper boundary of the dimension; The parameter vector is input into a numerical simulator to obtain a first data set.

4. The method according to claim 1, wherein Before inputting the plurality of expert trajectories into a preset inverse reinforcement learning model to obtain the update amplitudes of the policy network in the inverse reinforcement learning model and the probability values ​​of the discriminator in the inverse reinforcement learning model corresponding to the plurality of expert trajectories, the method further includes: Establish an initial inverse reinforcement learning model, input historical input parameters into the policy network and discriminator of the initial inverse reinforcement learning model, and obtain the initial update amplitude and initial probability value; A plurality of preset expert trajectories are obtained from an expert trajectory library, the discriminator is trained according to the obtained plurality of preset expert trajectories and the initial probability value to obtain a trained discriminator, and the policy network is trained according to the obtained plurality of preset expert trajectories and the initial update amplitude to obtain a trained policy network.

5. The method according to claim 1, wherein Optimizing the input parameters according to the target expert trajectory to obtain optimal input parameters further includes: A trajectory legitimacy check is performed on the target expert trajectory, and when the legitimacy of the target expert trajectory exceeds a preset legal range, the legitimacy of the target expert trajectory is corrected.

6. The method according to claim 1, characterized in that The performing screening and data enhancement processing on the first data set to obtain a second data set includes: Screening the first data set to obtain data whose development effect index is greater than a preset threshold and whose injection parameters meet physical and engineering constraints, thereby obtaining a screened first data set; The first data set after the screening process is interpolated in time steps and a preset amplitude disturbance is applied to the injection parameters to generate new data to obtain a second data set.

7. The method according to claim 1, characterized in that The preset inverse reinforcement learning model is a dual-network static architecture based on generative adversarial imitation learning. The policy network is fixedly equipped with a 3-layer fully connected neural network structure, and the discriminator is fixedly equipped with a 2-layer fully connected neural network structure.

8. A parameter optimization device for oil reservoir development, characterized in that: include: an acquisition module for acquiring input parameters during the oil reservoir development process, the input parameters including oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters, inputting the input parameters into a numerical simulator to obtain an initialized numerical simulator, the numerical simulator being used to calculate reservoir state change data and development effect indicators during the oil field development process, the reservoir state change data including at least one of formation pressure distribution, remaining oil distribution, and saturation change, and the development effect indicator at least one of oil production, water cut, and recovery rate; a generating module, configured to generate a first data set according to the numerical simulator and the particle swarm optimization algorithm, wherein the first data set is used to characterize the corresponding relationship between the input parameters, the reservoir state change data, and the development effect indicator; a processing module, configured to perform screening and data enhancement on the first data set to obtain a second data set; The processing module is further configured to: the second data set includes a plurality of expert trajectories, the expert trajectories being used to characterize the dynamic process of the injection parameter acting on the reservoir and the evolution of the reservoir state over time; input the plurality of expert trajectories into a preset inverse reinforcement learning model; obtain update amplitudes of the policy network in the inverse reinforcement learning model and probability values ​​of the discriminator in the inverse reinforcement learning model corresponding to the plurality of expert trajectories; and, when the update amplitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold, determine the expert trajectory with the smallest update amplitude and the largest probability value as the target expert trajectory; The optimization module is used to optimize the input parameters according to the target expert trajectory to obtain the optimal input parameters.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images