A method, apparatus, equipment, and storage medium for optimizing parameters in reservoir development.

Abstract

This application discloses a method, apparatus, equipment, and storage medium for optimizing parameters in oil reservoir development, relating to the field of oil reservoir development technology. It includes: acquiring oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters; inputting these parameters into a numerical simulator for initialization; calculating reservoir state change data and development effect indicators; then generating a first dataset representing the correspondence between parameters, reservoir state, and development effect using a particle swarm optimization algorithm; obtaining a second dataset containing multiple expert trajectories after screening and data augmentation, whereby the expert trajectories reflect the temporal evolution of the reservoir state under the influence of injection parameters; inputting the expert trajectories into a preset inverse reinforcement learning model; and selecting target expert trajectories based on the policy network update magnitude and discriminator probability value; finally, optimizing the input parameters based on the target expert trajectories to obtain the optimal parameters. This achieves intelligent optimization of reservoir parameters, improves optimization efficiency and adaptability, and ensures development benefits.

Description

Technical Field

[0001] This application relates to the field of reservoir development technology, and in particular to a method, apparatus, equipment and storage medium for optimizing parameters in reservoir development. Background Technology

[0002] As oilfield development enters the medium-to-high water-cut stage, polymer injection and gas displacement, as core technologies for enhancing oil recovery, are increasingly widely used in the development of complex reservoirs. However, the reservoir development process is affected by factors such as the heterogeneity of geological conditions, dynamic changes in fluid properties, and the coupling of multiple development objectives, significantly increasing the difficulty of optimizing key parameters such as polymer injection concentration, gas injection rate, and injection-production cycle. Inappropriate parameter settings may not only lead to unsatisfactory oil recovery results but also cause problems such as wellbore blockage, excessive injection costs, or decreased reservoir stability, severely restricting the economic benefits and sustainability of oilfield development.

[0003] Currently, reservoir development parameter optimization mainly relies on traditional numerical simulation combined with manual trial and error, swarm intelligence algorithms (such as particle swarm optimization and genetic algorithms), or expert experience-driven methods. Traditional numerical simulation and manual trial and error methods require engineers to repeatedly adjust parameters and verify effects based on experience, making it difficult to systematically integrate complex factors such as multi-well collaboration and reservoir dynamic changes, resulting in low optimization efficiency and poor result stability. Although swarm intelligence algorithms can achieve global search of the parameter space, they are prone to getting trapped in local optima, and the optimization results are not adaptable enough to heterogeneous reservoirs or changes in fluid properties. Expert experience-driven methods rely on empirical formulas derived from historical development cases or subjective judgments by experts, making it difficult to quantify the coupling relationship between multiple objectives, and the cost of experience transfer is high and the generalization ability is weak.

[0004] Furthermore, while traditional reinforcement learning methods attempt to construct an "agent-reservoir environment" interaction model, they require manually pre-setting the reward function. However, in reservoir development, the fuzzy weights of multiple objectives, the time-varying nature of dynamic environmental rewards, and the difficulty in explicitly expressing implicit experiences lead to inherent difficulties in reward function design, making optimization strategies prone to disconnect from engineering practice. Therefore, how to overcome the limitations of traditional methods, achieve intelligent and efficient optimization of reservoir development parameters, and improve the adaptability of parameter optimization results to complex reservoir conditions and their alignment with development goals is a pressing issue in the field of polymer and gas injection development in oilfields. Summary of the Invention

[0005] In view of this, the parameter optimization method, apparatus, equipment, and storage medium for reservoir development provided in this application can achieve intelligent and efficient optimization of reservoir development parameters, and improve the adaptability of the parameter optimization results to complex reservoir conditions and the degree of conformity with development goals. The parameter optimization method, apparatus, equipment, and storage medium for reservoir development provided in this application are implemented as follows:

[0006] The process involves acquiring input parameters during reservoir development, including oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters. These input parameters are then input into a numerical simulator to obtain an initialized numerical simulator. The numerical simulator is used to calculate reservoir state change data and development performance indicators during the oilfield development process. The reservoir state change data includes at least one of formation pressure distribution, remaining oil distribution, and saturation change. The development performance indicators include at least one of oil production, water cut, and recovery rate.

[0007] A first dataset is generated based on the numerical simulator and the particle swarm optimization algorithm. The first dataset is used to characterize the correspondence between the input parameters, the reservoir state change data, and the development effect indicators.

[0008] The first dataset is filtered and augmented to obtain the second dataset.

[0009] The second dataset includes multiple expert trajectories, which are used to characterize the dynamic process of injection parameters acting on the reservoir and the evolution of the reservoir state over time. The multiple expert trajectories are input into a preset inverse reinforcement learning model to obtain the update magnitude of the policy network in the inverse reinforcement learning model and the probability value of the discriminator in the inverse reinforcement learning model corresponding to the multiple expert trajectories. When the update magnitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold, the expert trajectory with the smallest update magnitude and the largest probability value is determined as the target expert trajectory.

[0010] The input parameters are optimized based on the target expert trajectory to obtain the optimal input parameters.

[0011] In some embodiments, after obtaining the input parameters during the reservoir development process, the method further includes:

[0012] The input parameters are normalized to obtain normalized input parameters;

[0013] The input parameters are processed for outliers by replacing outliers with boundary values ​​and by supplementing missing values ​​through linear interpolation and mean imputation, resulting in outlier-processed input parameters.

[0014] The input parameters are encoded into action vectors to obtain parameter vectors.

[0015] In some embodiments, generating the first dataset based on the numerical simulator and the particle swarm optimization algorithm includes:

[0016] Initialize several particles, each representing a set of parameter vectors, and configure the inertia weight, learning factor, and maximum number of iterations for the particle swarm optimization algorithm.

[0017] Several examples are processed iteratively. In each iteration, the particle velocity and position are updated based on the particle's historical velocity, individual historical best position, and global best position to obtain the parameter vector after iteration.

[0018] Boundary checks are performed on the iterated parameter vector, correcting it to the lower bound value when the parameter vector is below the lower bound of the dimension, and correcting it to the upper bound value when the parameter vector is above the upper bound of the dimension.

[0019] The parameter vector is input into the numerical simulator to obtain the first dataset.

[0020] In some embodiments, before inputting multiple expert trajectories into a preset inverse reinforcement learning model to obtain the update magnitude of the policy network in the inverse reinforcement learning model corresponding to each expert trajectory and the probability value of the discriminator in the inverse reinforcement learning model, the method further includes:

[0021] An initial inverse reinforcement learning model is established by inputting historical input parameters into the policy network and discriminator of the initial inverse reinforcement learning model to obtain the initial update magnitude and initial probability value.

[0022] Multiple preset expert trajectories are obtained from the expert trajectory library. The discriminator is trained based on the multiple preset expert trajectories and the initial probability value to obtain the trained discriminator. The policy network is trained based on the multiple preset expert trajectories and the initial update magnitude to obtain the trained policy network.

[0023] In some embodiments, the step of optimizing the input parameters based on the target expert trajectory to obtain the optimal input parameters further includes:

[0024] The target expert trajectory is validated for legality. If the legality of the target expert trajectory exceeds a preset legality range, the legality of the target expert trajectory is corrected.

[0025] In some embodiments, the step of filtering and data augmenting the first dataset to obtain a second dataset includes:

[0026] The first dataset is filtered to obtain data whose development performance indicators are greater than a preset threshold and whose injection parameters meet physical and engineering constraints, thus obtaining the filtered first dataset.

[0027] The first dataset after filtering is interpolated at time steps, and the injected parameters are perturbed by a preset amplitude to generate new data, thus obtaining the second dataset.

[0028] In some embodiments, the preset inverse reinforcement learning model is a dual-network static architecture based on generative adversarial imitation learning, wherein the policy network is fixed to a 3-layer fully connected neural network structure, and the discriminator is fixed to a 2-layer fully connected neural network structure.

[0029] This application provides a parameter optimization device for reservoir development, comprising:

[0030] The acquisition module is used to acquire input parameters during the reservoir development process. The input parameters include oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters. The input parameters are input into the numerical simulator to obtain an initialized numerical simulator. The numerical simulator is used to calculate reservoir state change data and development effect indicators during the oilfield development process. The reservoir state change data includes at least one of formation pressure distribution, remaining oil distribution, and saturation change. The development effect indicators include at least one of oil production, water cut, and recovery rate.

[0031] The generation module is used to generate a first dataset based on the numerical simulator and the particle swarm optimization algorithm. The first dataset is used to characterize the correspondence between the input parameters, the reservoir state change data and the development effect indicators.

[0032] The processing module is used to perform filtering and data augmentation on the first dataset to obtain the second dataset;

[0033] The processing module is further configured to: input the multiple expert trajectories into a preset inverse reinforcement learning model, thereby obtaining the update magnitude of the policy network in the inverse reinforcement learning model and the probability value of the discriminator in the inverse reinforcement learning model corresponding to the multiple expert trajectories; and, if the update magnitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold, determine the expert trajectory with the smallest update magnitude and the largest probability value as the target expert trajectory.

[0034] The optimization module is used to optimize the input parameters based on the target expert trajectory to obtain the optimal input parameters.

[0035] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0036] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method provided in this application embodiment.

[0037] This application provides a method, apparatus, equipment, and storage medium for optimizing parameters in oil reservoir development. The method includes: acquiring oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters; inputting these parameters into a numerical simulator for initialization; calculating reservoir state change data and development performance indicators; generating a first dataset representing the correspondence between parameters, reservoir state, and development performance using a particle swarm optimization algorithm; obtaining a second dataset containing multiple expert trajectories after filtering and data augmentation, whereby the expert trajectories reflect the temporal evolution of the reservoir state under the influence of injection parameters; inputting the expert trajectories into a preset inverse reinforcement learning model; filtering target expert trajectories based on the policy network update magnitude and discriminator probability value; and finally optimizing the input parameters based on the target expert trajectories to obtain the optimal parameters. This method achieves intelligent optimization of reservoir parameters, improves optimization efficiency and adaptability, ensures development benefits, and solves the technical problems mentioned in the background art. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A schematic diagram illustrating the implementation process of a parameter optimization method for reservoir development provided in this application embodiment;

[0040] Figure 2 A schematic diagram illustrating the implementation process of processing input parameters provided in an embodiment of this application;

[0041] Figure 3 This application provides a parameter optimization device for reservoir development. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.

[0044] Figure 1 This is a schematic flowchart illustrating the implementation of a parameter optimization method for reservoir development provided in this application, including steps 101 to 105. Wherein, Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order for a parameter optimization method for reservoir development. Where the final result can be achieved, Figure 1 The steps shown can be performed in parallel or in reverse order.

[0045] Step 101: Obtain the input parameters during the reservoir development process, input the input parameters into the numerical simulator, and obtain the initialized numerical simulator.

[0046] In this embodiment, oilfield geological parameters are obtained through core experiments and well logging data interpretation, including permeability and porosity distribution in different areas of the block, formation pressure and formation temperature monitored in real time by bottom hole pressure gauges and temperature sensors, and residual oil distribution and water cut calculated by production dynamic analysis software. Well locations and operating parameters are determined through field measurements and development planning, including the three-dimensional spatial coordinates of water injection wells, gas injection wells, and production wells, the injection rate recorded per hour by wellhead flow meters, the injection time set according to the block development plan, and the well group allocation scheme based on well network type and well connectivity test results. Injection parameters are obtained through laboratory configuration, field sampling and testing, and measurement statistics, including polymer concentration and cumulative injection volume of polymer and gas, and carbon dioxide is selected as the injection gas according to the reservoir characteristics of the block. Economic evaluation parameters are determined with reference to market quotations, industry standards, and historical data, including real-time crude oil prices, polymer procurement costs, gas processing costs, benchmark discount rates and investment payback periods set according to enterprise investment requirements, and single-well operating costs and tax rates calculated based on historical operating data and tax policies.

[0047] After processing the collected input parameters, they are input into the CMG (Computer Modelling Group Numerical Simulator) to complete the simulator initialization. Once initialized, the numerical simulator can simulate and calculate reservoir state changes (such as formation pressure distribution, remaining oil distribution, and saturation changes at different development stages) and development performance indicators (such as daily oil production, water cut, and staged recovery rate) during the development of the oilfield block, based on the input parameters.

[0048] Step 102: Generate the first dataset based on the numerical simulator and particle swarm optimization algorithm.

[0049] In this embodiment, a particle swarm optimization algorithm is initiated, and several particles are initialized. Each particle corresponds to a complete set of input parameter combinations (i.e., a set of parameter schemes for polymer and gas development). The parameters of the particle swarm optimization algorithm are configured as follows: the initial value of the inertia weight is set to 0.9, which decreases linearly to 0.4 with the number of iterations; the learning factor is fixed at 2.0; and the maximum number of iterations is set to 80 to balance the global search capability in the early stage of the algorithm with the local optimization accuracy in the later stage.

[0050] In each iteration, based on the historical velocity of each particle, its own historical best position (i.e. the parameter combination with the best development effect in the past iterations for that particle), and the global best position of all particles (i.e. the parameter combination with the best development effect in the past iterations for all particles), the velocity and position of each particle are calculated and updated to obtain the parameter combination after iteration (i.e. the input parameters corresponding to the particles after iteration).

[0051] If a parameter (such as polymer concentration) is below the lower bound of that parameter dimension (e.g., 800 mg / L), it is corrected to the lower bound value; if a parameter (such as gas injection rate) is above the upper bound of that parameter dimension (e.g., 8000 cubic meters per day), it is corrected to the upper bound value.

[0052] Each iteration inputs the corrected parameter combinations into an initialized numerical simulator, which then simulates the oilfield development process under the influence of these parameter combinations, outputting corresponding reservoir state change data and development performance indicators. The data representing the correspondence between "input parameters - reservoir state change data - development performance indicators" obtained from all iterations are then compiled to form the first dataset.

[0053] Step 103: Perform filtering and data augmentation on the first dataset to obtain the second dataset.

[0054] In this embodiment, preset development performance indicator thresholds are established. Based on the development needs of this medium-to-high permeability heterogeneous oilfield block, a recovery rate threshold of no less than 25% and a daily oil production threshold of no less than 50 cubic meters are set. Simultaneously, the input parameters corresponding to each data point are verified to meet physical and engineering constraints, such as a polymer concentration of 800-2500 mg / L, an injection pressure not exceeding 90% of the formation fracturing pressure, and an injection rate not exceeding 95% of the rated capacity of the injection and production equipment. Data that simultaneously meets the development performance indicator thresholds and parameter constraints are selected to obtain the first dataset after filtering.

[0055] For development data with a time step interval greater than 3 days (i.e., the time interval between two adjacent data records exceeds 3 days), linear interpolation is used to supplement reservoir state change data (such as formation pressure distribution and remaining oil distribution) and input parameter data (such as polymer concentration and gas injection rate) between adjacent time steps, unifying the time step interval of all data to 1-2 days and improving the temporal continuity of the data. Simultaneously, random perturbations of preset amplitudes are applied to the input parameters, such as ±8%-15% perturbation for polymer concentration, ±5%-10% perturbation for gas injection rate, and ±2%-5% perturbation for injection time. After perturbation, the parameters are re-verified to ensure compliance with physical and engineering constraints. Perturbed data that passes the verification is added as new data to the first filtered dataset, ultimately forming a second dataset containing multiple expert trajectories. Each expert trajectory fully represents the dynamic process of reservoir state evolution over time after a set of injection parameters is applied to the reservoir (e.g., the continuous process of formation pressure and remaining oil distribution changing with injection parameters from the early to the middle stages of development).

[0056] Step 104: Input multiple expert trajectories into the preset inverse reinforcement learning model to obtain the update magnitude of the policy network and the probability value of the discriminator in the inverse reinforcement learning model corresponding to each expert trajectory. If the update magnitude of the policy network is less than the preset threshold and the probability value of the discriminator is greater than the preset threshold, the expert trajectory with the smallest update magnitude and the largest probability value is determined as the target expert trajectory.

[0057] In this embodiment, the strategy network adopts a multi-layer fully connected neural network structure to learn optimization strategies for reservoir development parameters; the discriminator also adopts a multi-layer fully connected neural network structure to determine whether the input trajectory conforms to expert-level development logic (i.e., whether it is a high-quality development parameter trajectory).

[0058] For each expert trajectory, during model operation, the strategy network adjusts its own network parameters according to the reservoir status and parameter information in the trajectory, and records the update magnitude of the strategy network parameters (i.e. the degree of change before and after the network parameter adjustment); at the same time, the discriminator determines the "expert attribute" of the trajectory and outputs the probability value that the trajectory belongs to the expert trajectory (the higher the probability value, the closer the development parameter optimization logic of the trajectory is to the expert decision logic).

[0059] Based on the model training convergence requirements, the policy network update magnitude threshold is set to 10 to the power of -4 (i.e., when the parameter update magnitude is less than this value, it indicates that the policy network's learning of the trajectory has become stable), and the discriminator probability value threshold is set to 0.48 (i.e., when the probability value is greater than this value, it indicates that the similarity between the trajectory and the expert trajectory meets the standard).

[0060] Expert trajectories that meet the criteria of "policy network update magnitude less than a preset threshold and discriminator probability value greater than a preset threshold" are selected. Among these trajectories that meet the criteria, the policy network update magnitude and discriminator probability value are further compared. Finally, the expert trajectory that has "minimum policy network update magnitude and maximum discriminator probability value" is determined and used as the target expert trajectory (the parameter optimization logic corresponding to this trajectory is the most stable, closest to the expert decision, and has the best development effect).

[0061] Step 105: Optimize the input parameters based on the target expert trajectory to obtain the optimal input parameters.

[0062] In the embodiments of this application, the correlation between injection parameters and reservoir status and development effect in the target expert trajectory is analyzed. For example, the adjustment law of polymer concentration with the change of remaining oil distribution in a certain development stage of the trajectory, the matching law of gas injection rate with formation pressure change, etc.

[0063] Based on the above rules, the initial reservoir development input parameters are optimized and adjusted. If the polymer concentration in the initial injection parameters deviates from the optimal concentration of the same reservoir state in the target expert trajectory at a certain development stage, it is adjusted to a suitable concentration according to the trajectory rules. If the formation pressure change corresponding to the initial gas injection rate does not conform to the optimal pressure change trend in the trajectory, the gas injection rate is adjusted according to the trajectory rules.

[0064] After the adjustment is completed, the optimized input parameters are input into the numerical simulator again for verification. It is confirmed that the corresponding development performance indicators (such as recovery rate, daily oil production, and net present value) meet the development goals of the oilfield block and there is no risk of wellbore blockage or excessive injection cost. Finally, the optimal input parameters for reservoir development are obtained.

[0065] This application's embodiments achieve intelligent and systematic parameter optimization through a collaborative architecture of "numerical simulator + particle swarm optimization + inverse reinforcement learning." By leveraging expert trajectories to represent the dynamic correlation between "injection parameters - reservoir state - time," and combining this with the implicit learning of expert experience through an inverse reinforcement learning model, the optimization strategy becomes more adaptable to complex conditions such as reservoir heterogeneity and fluid dynamic changes. Compared to traditional methods, it can more accurately match reservoir needs throughout the entire development cycle. By selecting the target expert trajectory with "the smallest strategy network update amplitude and the largest discriminator probability value," it ensures that the optimized input parameters not only meet the physical and engineering constraints of reservoir development but also maximize core indicators such as recovery rate and net present value, balancing development effectiveness and economic benefits.

[0066] In the above Figure 1 Based on the above, this application also provides a schematic diagram of the implementation process for processing input parameters, as shown below. Figure 2 As shown, steps 201 to 203 are included:

[0067] Step 201: Normalize the input parameters to obtain normalized input parameters.

[0068] In this embodiment, the types of parameters that need to be normalized are distinguished according to the type and physical meaning of the input parameters. This mainly targets injection parameters (such as polymer concentration, gas injection rate, cumulative polymer injection volume, and cumulative gas injection volume). These parameters have large differences in dimensions (such as concentration in milligrams per liter and rate in cubic meters per day). Directly using them in subsequent particle swarm optimization algorithms or model training can easily lead to an imbalance in parameter weights. Therefore, normalization is needed to eliminate the influence of dimensions.

[0069] In practice, the actual value range of each injection parameter in the historical development data of the oilfield block is first calculated. For example, the historical minimum value of polymer concentration is 800 mg / L, and the historical maximum value is 2500 mg / L; the historical minimum value of gas injection rate is 1500 m³ / day, and the historical maximum value is 8000 m³ / day. For each injection parameter, its original data is calculated against its historical minimum and maximum values ​​to uniformly map the original data to the interval between zero and one. If the original value of a certain polymer concentration is 1600 mg / L, it is converted into a corresponding value within the zero-to-one interval through calculation, ensuring that injection parameters of different dimensions can participate in subsequent optimization calculations on the same dimension.

[0070] For oilfield geological parameters (such as permeability and porosity), well location and operation parameters (such as well coordinates and injection time), and economic evaluation parameters (such as crude oil price and operating cost), normalization will not be performed for the time being, as the original physical or economic characteristics need to be preserved in the future. Only the original values ​​will be recorded for subsequent steps.

[0071] Step 202: Perform outlier processing on the input parameters by replacing outliers with boundary values ​​and supplementing missing values ​​through linear interpolation and mean imputation to obtain the outlier-processed input parameters.

[0072] In this embodiment of the application, for oilfield geological parameters, if the permeability data is negative (not in accordance with physical meaning) or the porosity data is greater than 40% (exceeding the normal range of reservoir porosity in the block), it is determined to be an anomaly. The permeability anomaly is directly replaced with the lower limit value of the permeability of the same type of reservoir in the block (such as 50 millidarcy), and the porosity anomaly is replaced with the upper limit value of the porosity in the block (such as 30%).

[0073] For injection parameters, if the polymer concentration is below 500 mg / L (below the lower limit of effective displacement concentration) or above 3000 mg / L (exceeding the upper limit of equipment configuration), or the gas injection rate is above 10000 cubic meters per day (exceeding the rated capacity of the gas injection equipment), it is judged as an abnormal value; the abnormal polymer concentration value is replaced with 500 mg / L or 3000 mg / L respectively, and the abnormal gas injection rate value is replaced with 10000 cubic meters per day.

[0074] Regarding the economic evaluation parameters, if the crude oil price is lower than 50% of the lowest market price in the same period and the single well operating cost is higher than twice the historical average cost, it is judged as an outlier; the outlier crude oil price is replaced with the lowest market price in the same period, and the outlier single well operating cost is replaced with twice the historical average cost.

[0075] Depending on the parameter type, a differentiated method is used to supplement missing values. For oilfield geological parameters (such as permeability and porosity), if permeability data for a certain well location is missing, the permeability data of the three adjacent wells in the same formation are obtained, and the permeability value of the missing well location is calculated using linear interpolation. For example, if the permeabilities of the three adjacent wells are 60 millidarcs, 70 millidarcs, and 80 millidarcs, the permeability of the missing well location is calculated to be 65 millidarcs through interpolation.

[0076] For production-related operational parameters (such as daily oil production and gas injection rate), if the gas injection rate data for a certain day is missing, the average daily gas injection rate for the seven days prior to the missing date is calculated, and this average value is used as supplementary data for the missing value. For example, if the gas injection rates for the seven days prior to the missing date are 4500 cubic meters per day, 4600 cubic meters per day, ..., 4800 cubic meters per day, the calculated average daily rate is 4650 cubic meters per day, which is used to supplement the missing data.

[0077] For economic evaluation parameters (such as polymer procurement costs), if procurement cost data for a certain quarter is missing, the average of procurement costs for the other three quarters of the year is used to fill the gap, ensuring the completeness of the economic parameters.

[0078] Through the above outlier and missing value processing, a set of input parameters without outliers or missing values ​​is obtained.

[0079] Step 203: Encode the input parameters into action vectors to obtain parameter vectors.

[0080] In this embodiment, based on the parameter input requirements of the subsequent particle swarm optimization algorithm, the preprocessed input parameters are encoded into structured action vectors (i.e., parameter vectors) according to the "development decision dimension," ensuring that each vector corresponds to a complete set of development parameters. The specific encoding logic is as follows:

[0081] Based on the key decision parameters for reservoir development, the parameter vector is set into six dimensions, corresponding to "polymer concentration", "gas injection rate", "cumulative polymer injection volume", "cumulative gas injection volume", "injection-production cycle" and "daily water injection volume per well".

[0082] The preprocessed input parameters are mapped one by one into a vector according to their dimensional order. For example, in a certain set of input parameters, the polymer concentration is 1500 mg / L, the gas injection rate is 5000 m³ / day, the cumulative polymer injection volume is 100000 m³, the cumulative gas injection volume is 150000 m³, the injection-production cycle is 300 days, and the daily water injection volume per well is 2000 m³. These parameters are filled into the six dimensions in order to form a six-dimensional parameter vector.

[0083] After encoding, check whether the parameters of each dimension still meet the physical and engineering constraints (such as whether the polymer concentration is within the range of 500 to 3000 mg / L). If there are parameter deviations caused by encoding errors, correct them in time and determine the final parameter vector for particle initialization in subsequent particle swarm optimization algorithms.

[0084] This application's embodiments eliminate dimensional differences in injection parameters (such as polymer concentration and gas injection rate) through normalization processing, avoiding parameter weight deviations in particle swarm optimization or model training due to dimensional imbalances. By replacing outliers (e.g., correcting physically unreasonable values ​​to boundary values) and supplementing missing values ​​(linear interpolation, mean imputation), it addresses the common problems of "incomplete data and outlier interference" in historical data, ensuring the integrity and accuracy of input parameters. The preprocessed parameters are encoded into fixed-dimensional action vectors (parameter vectors), allowing the particle swarm optimization algorithm to directly call standardized parameter combinations for searching. This also ensures a uniform input format received by the inverse reinforcement learning model, avoiding algorithm interruptions or model training deviations caused by non-standard parameter formats, and improving the smoothness and reliability of subsequent optimization processes.

[0085] In some embodiments, generating a first dataset based on a numerical simulator and a particle swarm optimization algorithm includes: initializing a number of particles, each particle representing a set of parameter vectors, and configuring the inertia weight, learning factor, and maximum number of iterations for the particle swarm optimization algorithm.

[0086] Specifically, based on the development parameter search requirements of this oilfield block, the number of particles in the particle swarm optimization algorithm was determined. Considering the strong reservoir heterogeneity within the block (permeability variation coefficient 0.85), and the need to cover a wide range of parameter combinations, 50 particles were initialized, with each particle uniquely corresponding to a set of six-dimensional parameter vectors (i.e., a complete set of polymer injection and gas development parameter schemes).

[0087] The initial parameter vector values ​​for the particles are determined based on historical successful development schemes and engineering constraints for the block. For example, the initial value range for the polymer concentration dimension is 800 to 2500 mg / L, the gas injection rate dimension is 1500 to 8000 cubic meters per day, the injection-production cycle dimension is 180 to 730 days, and the remaining dimensions are randomly assigned values ​​within the feasible range of similar engineering projects to ensure that the initial particles cover the main parameter combination types.

[0088] Based on the requirements for search accuracy and efficiency in reservoir parameter optimization, the core parameters of the particle swarm optimization algorithm are configured. The initial value of the inertia weight is set to 0.9, and it decreases linearly to 0.4 with the number of iterations. In the early stage of iteration (the first thirty rounds), a high inertia weight is maintained to enhance the algorithm's global search capability in the parameter space and avoid missing potential high-quality parameter combinations. In the later stage of iteration (the last fifty rounds), the inertia weight is reduced to focus on fine optimization of local high-quality parameter regions and improve parameter accuracy.

[0089] Two learning factors are set, corresponding to the learning weights of the particle for its "individual historical best position" and "global best position," respectively, both fixed at 0.2. This ensures that the particle can fully learn from its own optimal parameter experience in past iterations while simultaneously converging towards the global best parameters of all particles, balancing individual exploration and group collaboration.

[0090] Based on the block parameter dimension (six-dimensional) and the number of particles (50), the maximum number of iterations is set to eighty rounds. Preliminary testing has verified that eighty iterations allow the algorithm to achieve a balance between global search and local optimization, avoiding both the omission of optimal solutions due to insufficient iterations and the waste of computational resources caused by excessive iterations.

[0091] Furthermore, several examples are iteratively processed. In each iteration, the particle velocity and position are updated based on the particle's historical velocity, individual historical best position, and global best position to obtain the iterative parameter vector.

[0092] Specifically, at the start of each iteration, a new velocity is calculated for each particle, taking into account its historical velocity, individual historical best position, and global best position. The particle's historical velocity refers to the rate of change of its parameter vector in the previous iteration, reflecting its past search trend; its individual historical best position refers to the position of the parameter vector corresponding to the optimal development effect (recovery rate, net present value) since the start of the iteration; and the global best position refers to the position of the parameter vector corresponding to the optimal development effect for all particles since the start of the iteration.

[0093] During the calculation, the degree to which particles learn from "individual experience" and "collective experience" is adjusted by a learning factor. For example, if the parameter adjustment of a particle's velocity in the previous round was small, and its historical best position differs significantly from the global best position, the learning factor will amplify the learning weight of the global best position, guiding the particle to move towards a region with better parameters, ensuring that the particle's velocity can adapt to the current search requirements after each iteration.

[0094] Based on the updated particle velocities, the position of each particle in the parameter space is adjusted to generate an iterated parameter vector. Using the current parameter vector of a particle as a base, the parameter adjustment corresponding to the updated velocity is superimposed to obtain a new parameter vector (i.e., the new position of the particle). If the "polymer concentration" in a particle's current parameter vector is 1500 mg / L, the updated velocity needs to increase that dimension by 100 mg / L, so after iteration, the parameter for that dimension becomes 1600 mg / L. The other dimensions are adjusted according to the same logic, ultimately forming a complete iterated parameter vector.

[0095] In each iteration, all 50 particles update their velocity and position according to the above logic, generating 50 sets of parameter vectors after iteration.

[0096] Furthermore, a boundary check is performed on the iterated parameter vector. If the parameter vector is below the lower bound of the dimension, it is corrected to the lower bound value, and if the parameter vector is above the upper bound of the dimension, it is corrected to the upper bound value.

[0097] Specifically, since particle velocity updates may cause some parameter vector dimensions to exceed the engineering or physical constraints, it is necessary to perform boundary checks and corrections on all parameter vectors after iteration to ensure that the parameters input to the numerical simulator meet the actual development requirements. Based on the equipment capabilities, reservoir characteristics and development specifications of the oilfield block, the upper and lower bounds of the six-dimensional parameter vector are preset. The lower limit for polymer concentration is 800 mg / L (below this value, displacement efficiency decreases significantly), and the upper limit is 2500 mg / L (above this value, wellbore blockage is likely to occur); the lower limit for gas injection rate is 1500 m³ / day (below this value, displacement pressure cannot be effectively established), and the upper limit is 8000 m³ / day (above this value, the rated capacity of the gas injection equipment is exceeded); the lower limit for cumulative polymer injection volume is 50000 m³ (below this value, effective sealing cannot be formed), and the upper limit is 300000 m³ (above this value, economic efficiency decreases); the lower limit for cumulative gas injection volume is 100000 m³ (below this value, the displacement range is insufficient), and the upper limit is 500000 m³ (above this value, gas escape is likely to occur).

[0098] The lower limit of the injection-production cycle is 180 days (below this value, parameter adjustments are too frequent), and the upper limit is 730 days (above this value, it is impossible to adapt to the dynamic changes of the reservoir in a timely manner); the lower limit of the daily water injection volume of a single well is 1000 cubic meters (below this value, it is impossible to maintain formation pressure), and the upper limit is 3000 cubic meters (above this value, it is easy to cause the formation water production to intensify).

[0099] If a certain dimension parameter is below the lower limit (e.g., polymer concentration is 700 mg / L), then the dimension parameter is directly corrected to the corresponding lower limit value (800 mg / L); if a certain dimension parameter is above the upper limit (e.g., gas injection rate is 8500 cubic meters per day), then it is corrected to the corresponding upper limit value (8000 cubic meters per day).

[0100] If the parameter is within the boundary range, then keep the original value unchanged.

[0101] Furthermore, the parameter vector is input into the numerical simulator to obtain the first dataset.

[0102] Specifically, the boundary-checked and corrected parameter vectors (50 sets per iteration) are input one by one into the initialized CMG numerical simulator (which has loaded the geological model, fluid properties, and other basic data of the oilfield block). Based on the input parameter vectors, the reservoir development process under the action of these parameters is simulated for 180 days, and the reservoir state change data at different time points (such as formation pressure distribution, remaining oil distribution, and saturation changes) are calculated. The simulator automatically calculates the development effect indicators corresponding to these parameters based on the reservoir state changes, including cumulative oil production, stage water cut, stage recovery rate, and net present value. The three elements of "input parameter vectors - simulated reservoir state change data - development effect indicators" are linked one by one to form a complete trajectory data.

[0103] After all 80 iterations are completed, the trajectory data generated during all iterations are aggregated, and duplicate data is removed, resulting in the first dataset containing 3,800 valid trajectory data. This dataset fully represents the correspondence between input parameters, reservoir state change data, and development performance indicators, providing a foundation for subsequent selection and data augmentation processing of the first dataset.

[0104] This application's embodiments, by reasonably setting the number of particles (adapting to reservoir complexity) and algorithm parameters (linearly decreasing inertia weights and synergistic learning factors), enable the particle swarm optimization algorithm to cover a broad parameter space (avoiding the omission of potential high-quality parameters) while focusing on local optimization in the later stages of iteration (improving parameter accuracy). Combined with boundary checks after iteration (correcting out-of-bounds parameters), it ensures that the generated parameter vectors conform to physical laws and engineering constraints, preventing invalid parameters from entering the numerical simulator. Through a closed-loop process of "iterative update - boundary check - simulator calculation," each trajectory in the first dataset accurately reflects the causal relationship of "input parameters → reservoir state changes → development effect indicators." Compared with the traditional "random parameter generation + simulation" method, this significantly improves the effectiveness of the dataset, providing high-quality basic data for subsequent screening and data augmentation.

[0105] In some embodiments, before inputting multiple expert trajectories into a preset inverse reinforcement learning model to obtain the update magnitude of the policy network and the probability value of the discriminator in the inverse reinforcement learning model corresponding to the multiple expert trajectories, the method further includes: establishing an initial inverse reinforcement learning model, inputting historical input parameters into the policy network and discriminator of the initial inverse reinforcement learning model to obtain the initial update magnitude and the initial probability value.

[0106] Specifically, the initial inverse reinforcement learning model adopts a dual-network architecture of "policy network-discriminator," both of which are built on fully connected neural networks. The policy network adopts a three-layer fully connected neural network structure. The input layer is a twelve-dimensional reservoir state vector (the dimensions correspond to formation pressure, porosity, permeability, remaining oil distribution, water cut, oil production, recovery rate, net present value, polymer concentration, gas injection rate, injection-production cycle, and daily water injection per well), used to receive dynamic reservoir state data. The output layer is a six-dimensional injection parameter decision vector, used to output development parameter schemes adapted to the current reservoir state. The number of hidden layer nodes is set to 64 according to the complexity of the reservoir data in this block. The activation function is the ReLU function, which can adapt to nonlinear parameter mapping, to ensure that the network can learn the complex correlation between reservoir state and injection parameters.

[0107] The discriminator adopts a two-layer fully connected neural network structure. The input layer is a trajectory segment combining "reservoir state-injection parameters" to receive the trajectory information to be judged. The output layer is a one-dimensional probability value to represent the probability that the input trajectory belongs to the "expert trajectory". The number of hidden layer nodes is set to 40, and the activation function is the Sigmoid function to ensure that the output probability value is stable in the range of zero to one, which is convenient for subsequent trajectory judgment and model training.

[0108] The "Xavier initialization" method is adopted, which randomly generates the initial weight matrix based on the mean and variance of the input and output dimensions of each layer of the network. For example, the weight matrix from the input layer to the hidden layer of the policy network is calculated based on the twelve dimensions of the input layer and the sixty-four nodes of the hidden layer, thus avoiding model training instability caused by initial weights that are too large or too small.

[0109] The bias parameters of all layers are initialized to 0.1 to ensure that each neuron has basic activation ability in the initial state of the network and to reduce convergence delay in the early stage of training.

[0110] For subsequent training, an optimizer was preset, and both were selected using the Adam optimizer with an initial learning rate of 0.01% to balance training stability and convergence speed.

[0111] At this point, the initial inverse reinforcement learning model has been built. The model can receive input data and output preliminary calculation results.

[0112] Historical development data from the past five years were selected from the development database of this oilfield block as input data for the initial model. The selection criteria were that the recovery rate corresponding to the historical data was not less than 25% and the net present value was not less than the industry average level during the same period, ensuring that the input data had basic reference value. The data included complete reservoir status data (formation pressure, remaining oil distribution, etc.) and corresponding injection parameter data (polymer concentration, gas injection rate, etc.), with no missing key dimensions. The data covered a continuous development cycle of at least 360 days, which could form a complete time-series trajectory of "reservoir status - injection parameters", which was suitable for the time-series learning requirements of the model.

[0113] Finally, 20 sets of historical input parameters that met the requirements (each set containing 360 consecutive days of time series data) were selected as the input source for the initial model.

[0114] Twenty sets of historical input parameters were input into the initial inverse reinforcement learning model, and the following calculations were performed. After the reservoir state data from the historical input parameters was input into the policy network, the network output an injection parameter decision vector based on the initial weights. This output vector was compared with the actual injection parameters from the historical input parameters, the difference between the two was calculated, and the network weights were adjusted according to the difference. The degree of change in weights before and after adjustment is the initial update magnitude. For example, if the polymer concentration output by the policy network differs significantly from the actual value after inputting a certain set of historical parameters, the network weight adjustment magnitude will increase accordingly, and the initial update magnitude will rise, reflecting the network's current learning bias towards this type of parameter.

[0115] After the "reservoir state-injection parameter" trajectory segment from the historical input parameters is input into the discriminator, the discriminator judges the "expert attribute" of the trajectory based on the initial weight and outputs the initial probability value. For example, if a certain set of historical trajectories corresponds to excellent development results, the initial probability value output by the discriminator is close to 0.6, indicating that the initial model can initially identify high-quality trajectories, but there are still judgment biases, which need to be further trained and optimized.

[0116] Furthermore, multiple preset expert trajectories are obtained from the expert trajectory library. The discriminator is trained based on the obtained multiple preset expert trajectories and initial probability values ​​to obtain the trained discriminator. The policy network is also trained based on the obtained multiple preset expert trajectories and initial update magnitude to obtain the trained policy network.

[0117] Specifically, pre-set expert trajectories are retrieved from the expert trajectory database of the oilfield. This database comprises high-quality development plans from the past decade, benchmark development cases of similar reservoirs in the industry, and optimal parameter trajectories verified through numerical simulators. The selection criteria are: the final recovery rate corresponding to the trajectory is at least 5% higher than the conventional plan, and the net present value is at least 10% higher; the reservoir geological conditions (permeability, porosity, heterogeneity) corresponding to the trajectory have at least 80% similarity to the block in this example, ensuring the suitability of the training data; each trajectory contains complete development cycle (730 days) data, including reservoir state changes and injection parameter adjustments in the early, middle, and late stages of development, covering parameter optimization needs throughout the entire development phase. Finally, thirty qualified pre-set expert trajectories are obtained from the database as the core data for model training.

[0118] With the goal of "improving the discriminator's accuracy in recognizing expert trajectories," training is conducted by combining preset expert trajectories with initial probability values. The steps are as follows:

[0119] Thirty preset expert trajectories are divided into trajectory segments according to time steps. Simultaneously, thirty non-expert trajectories are randomly selected from a second dataset and similarly divided into trajectory segments. The two types of segments are mixed in a 1:1 ratio to form a training dataset. The trajectory segments from the training dataset are successively input into the discriminator. Based on the initial probability value, the error of the discriminator in classifying "expert trajectory segments" as "expert attributes" and "non-expert trajectory segments" as "non-expert attributes" is calculated. If the probability value of an expert trajectory segment output by the discriminator is lower than 0.5, the weights and biases of the discriminator are adjusted based on this error to reduce the error of subsequent similar classifications. Two hundred trajectory segments are processed in each training round, for a total of fifty rounds. The discriminator performance is verified every ten rounds.

[0120] Training stops when the discriminator's accuracy in recognizing expert trajectory segments stabilizes above 90% and its accuracy in excluding non-expert trajectory segments stabilizes above 85%, resulting in the trained discriminator. At this point, the discriminator can accurately distinguish between expert and non-expert trajectories, and the output probability value can reliably characterize the "expert attribute" of the trajectory.

[0121] With the goal of "making the injected parameter trajectory output by the policy network approximate the expert trajectory", training is carried out by combining the preset expert trajectory and the initial update magnitude. The steps are as follows:

[0122] The reservoir state data of thirty preset expert trajectories are associated with the corresponding injection parameter data to form a "reservoir state-expert injection parameter" paired dataset, in which the reservoir state data serves as the input of the strategy network and the expert injection parameters serve as the target benchmark for the network output.

[0123] The reservoir state data from the paired dataset is input into the policy network, and the network outputs the injection parameter decision vector. The difference between this output vector and the target baseline is compared, and the parameter update error of the network is calculated in combination with the initial update magnitude. The gradient descent method is used to adjust the weights and biases of the policy network based on the error, so that the network output gradually approaches the expert-injected parameters. Fifty sets of paired data are processed in each training round, and a total of sixty rounds of training are carried out. The network output effect is verified every ten rounds through a numerical simulator.

[0124] Training stops when the difference between the simulator development metric corresponding to the output parameters of the policy network and the metric corresponding to the expert trajectory is less than 3%, and the update magnitude of the network weights is consistently lower than 30% of the initial update magnitude, resulting in the trained policy network. At this point, the policy network can output injection parameter schemes close to expert levels based on the reservoir conditions, meeting the requirements for subsequent expert trajectory input and target trajectory selection.

[0125] This application embodiment obtains the initial update magnitude and initial probability value by inputting historical parameters, enabling the model to possess basic parameter learning capabilities before formal training, avoiding the slow convergence problem caused by "training from scratch". It also utilizes preset expert trajectories to specifically train the discriminator and policy network, allowing the model to learn expert-level development logic in advance, reducing the number of training iterations. Through the two-step process of "historical parameter initialization → expert trajectory training", the discriminator can more accurately distinguish between expert and non-expert trajectories, and the policy network can more accurately learn the mapping pattern of "reservoir state → injection parameters". This avoids excessive deviations in the calculation of update magnitude and probability values ​​when subsequently inputting expert trajectories due to poor initial model performance, providing reliable model support for the accurate selection of target expert trajectories.

[0126] In some embodiments, the input parameters are optimized based on the target expert trajectory to obtain the optimal input parameters. The method further includes: performing trajectory validity verification on the target expert trajectory, and correcting the validity of the target expert trajectory if the validity exceeds a preset validity range.

[0127] Specifically, combining the physical laws of polymer and gas injection development, engineering equipment constraints, and oilfield development specifications, three core dimensions are determined for verifying the legality of the target expert trajectory: physical feasibility of injection parameters, reservoir development safety boundaries, and rationality of economic indicators. For each dimension, a preset legal range is set, specifically as follows: the polymer concentration must be within the range of "effective displacement concentration lower limit - equipment configuration upper limit," i.e., not less than 800 mg / L and not more than 2500 mg / L.

[0128] The gas injection rate must be within the range of "lower limit of displacement pressure establishment - upper limit of rated gas injection equipment", that is, not less than 1,500 cubic meters per day and not more than 8,000 cubic meters per day;

[0129] The gas-liquid ratio must be within the "reasonable range of miscibility and displacement", that is, not less than 0.3 and not more than 1.2.

[0130] The formation pressure must be within the range of "upper limit of formation fracture pressure - lower limit of bottom hole flowing pressure", that is, not higher than 90% of the formation fracture pressure and not lower than 1.2 times the bottom hole flowing pressure;

[0131] The monthly moisture content increase shall not exceed 5%;

[0132] The monthly decrease in remaining oil saturation does not exceed 1%.

[0133] The net present value must be no less than 95% of the net present value of the development target for this block;

[0134] The investment payback period should not exceed five years;

[0135] The unit oil production cost should not exceed 110% of the industry average unit oil production cost during the same period.

[0136] The logic of "time-series segmented verification - dimension-by-dimensional verification - anomaly marking" is used to perform legality verification on the full development cycle data of the target expert trajectory. The specific steps are as follows:

[0137] Considering the differences in parameter characteristics and key constraints at different stages of reservoir development, the 730-day development cycle of the target expert trajectory is divided into three time segments: "early development, mid-development, and late development," and verification is carried out separately for each segment. For example, in the early development stage, the focus is on verifying whether the formation pressure meets the safety boundary, while in the late development stage, the focus is on verifying whether the economic indicators meet the standards.

[0138] For the trajectory data of each time series, each item was verified in the following order: "physical feasibility of injection parameters → reservoir development safety boundary → rationality of economic indicators":

[0139] Extract the daily polymer concentration, injection rate, and gas-liquid ratio data within this time period and compare them with the preset legal range. For example, on a certain day during the middle of development, the polymer concentration in the trajectory is 2700 mg / L, which exceeds the upper limit of "not higher than 2500 mg / L". This data is marked as "physical feasibility of injection parameters is abnormal".

[0140] Extract daily data on formation pressure, water cut increase, and remaining oil saturation decrease within this time period and compare them with preset legal ranges. For example, if in a certain month during the early stages of development, the water cut in the trajectory increases by 6% in a single month, exceeding the upper limit of "not exceeding 5%", this data is marked as "reservoir development safety boundary anomaly".

[0141] The net present value, investment payback period, and unit oil production cost data for the specified time period are extracted quarterly and compared with the preset legal range. For example, in a certain quarter during the later stage of development, if the unit oil production cost in the trajectory is higher than 115% of the industry average, exceeding the upper limit of "not higher than 110%", the data is marked as "abnormal economic indicator rationality".

[0142] After completing full-time, full-dimensional verification, all abnormal data are summarized, and the legality status is categorized into three types based on the "scope of impact of the anomaly": "fully legal," "partially abnormal (within the legal scope)," and "severely abnormal (outside the legal scope)."

[0143] The data is completely legal with no dimension exceeding the preset legal range, and the trajectory can be directly used for parameter optimization.

[0144] Local anomalies (within the legal range) occur when a single dimension of data in a single time period exceeds the preset legal range, and the magnitude of the anomaly is small, so the impact on the overall development is controllable.

[0145] Serious anomalies (outside the legal range) are defined as multiple dimensions of data in at least two time periods exceeding the preset legal range, or a single abnormal data exceeding the upper limit by more than 10%, which have a significant negative impact on development results, reservoir safety, or economic benefits.

[0146] For the anomalies discovered during verification, and combining reservoir development patterns with expert experience, a correction process of "parameter adaptation correction - cascading impact compensation - effect verification" is adopted to ensure that the corrected trajectory not only conforms to the legal range but also retains the high-quality development logic of the original trajectory. The specific steps are as follows:

[0147] To address the anomaly of polymer concentration exceeding the upper limit during the development phase, a "gradient reduction correction" method was employed. The correction benchmark was determined with the upper limit of the preset legal range as the target value, and the trend of polymer concentration changes during the anomaly period was analyzed. In the original trajectory, the concentration on the first day of the anomaly period was 2700 mg / L, gradually decreasing to 2600 mg / L over the following nine days, showing a slow downward trend. Based on this trend, the gradient adjustment parameters corrected the concentration on the first day of the anomaly period to 2500 mg / L, decreasing it by 50 mg / L daily for the next nine days until it was finally corrected to 2500 mg / L, ensuring that the corrected concentration remained within the legal range of 800 to 2500 mg / L. Correlation parameters were also adjusted collaboratively, as the decrease in polymer concentration might lead to a decrease in displacement efficiency; therefore, the gas injection rate was adjusted simultaneously. During the ten days of concentration correction, the gas injection rate was gradually increased from 7000 cubic meters per day in the original trajectory to 7500 cubic meters per day, compensating for the loss of displacement efficiency caused by the decrease in polymer concentration by increasing the gas displacement intensity.

[0148] To address the anomaly of unit oil production costs exceeding the upper limit in the later stages of development, a "cost-production co-optimization" method is adopted:

[0149] Tracing the trajectory data revealed that the abnormal unit oil production cost originated from a temporary increase in polymer procurement costs, causing the cost to exceed the upper limit.

[0150] While ensuring that the recovery rate does not decline, the injection volume of high-cost polymers will be reduced. The cumulative polymer injection volume in the later stage of development will be reduced from 300,000 cubic meters in the original trajectory to 280,000 cubic meters. At the same time, by optimizing the injection and production cycle, the operating time of a single well will be reduced to lower labor and maintenance costs.

[0151] The revised parameters were input into the numerical simulator, and the simulation showed that the unit oil production cost dropped to 105% of the industry average, while the net present value remained at 98% of the target value, and the recovery rate decreased by only 0.3%.

[0152] All corrected target expert trajectories were re-verified according to the dimensions of "physical feasibility of injection parameters → reservoir development safety boundary → rationality of economic indicators" to confirm that no data exceeded the preset legal range; the difference between the corrected trajectory and the original trajectory in terms of development effect indicators was less than one percent; and the corrected injection parameters were feasible on site.

[0153] Based on the corrected legal target expert trajectory, the optimal injection parameter combination for each development stage in the trajectory is extracted and adapted to the current actual reservoir state of the block to obtain the final optimal input parameters.

[0154] The above optimal input parameters were verified by inputting them into a numerical simulator. The results showed that the recovery rate increased by 2.5% compared with that before optimization, the net present value met the target, and there were no safety or engineering constraints. They can be used as the final implementation parameters for polymer and gas development in this block.

[0155] This application's embodiments employ multi-dimensional verification of "physical feasibility of injection parameters, reservoir development safety boundaries, and economic indicator rationality" to avoid situations where the target expert trajectory merely meets theoretical optimization effects but violates physical laws, exceeds safety limits, or fails to meet economic benefits. This ensures that the optimization results are feasible for field implementation. Targeted corrections are made to trajectories exceeding legal limits, ensuring that the corrected target expert trajectory retains the original high-quality development logic while complying with engineering constraints. This avoids wellbore blockage, formation damage, or economic losses due to parameter violations, thereby improving the safety and economy of field development.

[0156] In some embodiments, the first dataset is filtered and augmented to obtain the second dataset, including: filtering the first dataset to obtain data whose development performance indicators are greater than a preset threshold and whose injection parameters meet physical and engineering constraints, thus obtaining the filtered first dataset.

[0157] Specifically, the screening process aims to "retain high-quality development data and eliminate invalid / non-compliant data," and is implemented based on the dual standards of "meeting development performance indicators" and "compliance of injection parameters." The recovery rate is no less than 25%; daily oil production is no less than 50 cubic meters; net present value (NPV) is no less than 90% of the target NPV for the block; polymer concentration is no less than 800 mg / L and no more than 2500 mg / L; gas injection rate is no less than 1500 cubic meters per day and no more than 8000 cubic meters per day; gas injection pressure is no more than 90% of the formation fracturing pressure; and the gas-liquid ratio is no less than 0.3 and no more than 1.2.

[0158] The screening process employs a logic of "trajectory-by-trajectory verification - dual-standard judgment - data retention." The four thousand trajectory data in the first dataset are split according to the principle of "independent verification of each trajectory." Each trajectory must undergo dual verification based on both development performance indicators and injection parameters. The recovery rate, daily oil production, and net present value (NPV) data for each trajectory are extracted and compared with preset thresholds. For example, if a trajectory has a recovery rate of 24% and a daily oil production of 46 cubic meters, both indicators are below the threshold, and it is directly judged as "invalid data" and removed. If a trajectory has a recovery rate of 26%, a daily oil production of 52 cubic meters, and an NPV of 92% of the target value, all three indicators meet the standards, then it proceeds to the next step of injection parameter verification. For trajectories that meet the development performance standards, their full-cycle polymer concentration, injection rate, injection pressure, and gas-liquid ratio data are extracted and compared with the constraint range for each parameter. For example, if the recovery rate of a certain trajectory meets the standard, but the polymer concentration is 2700 mg / L and the gas injection pressure reaches 93% of the formation fracture pressure on a certain ten-day period, it is judged as "non-compliant data" and removed; if all injection parameters of a certain trajectory are within the constraints throughout the entire cycle, it is judged as "valid data" and retained.

[0159] Furthermore, time-step interpolation is performed on the first dataset after filtering, and a preset amplitude perturbation is applied to the injected parameters to generate new data, thus obtaining the second dataset.

[0160] Specifically, the data augmentation process aims to "expand the amount of data and improve its diversity and continuity." Addressing the issues of "discontinuous time steps" and "insufficient parameter combination coverage" in the filtered dataset, new data is generated using two methods: "time step interpolation" and "injected parameter perturbation." In the filtered dataset, some trajectories have time step intervals exceeding three days due to the numerical simulator's calculation step size settings, resulting in insufficient temporal continuity between reservoir state changes and parameter adjustments. Interpolation is needed to supplement intermediate data.

[0161] The data to be interpolated include "reservoir state change data" and "injection parameter data"; the "linear interpolation method" is used, which is to calculate the data of the missing time step in the middle according to the time interval ratio based on the known data of two adjacent time steps.

[0162] After interpolation, the time step interval for all trajectories is standardized to one to two days. For example, for trajectories with an original time step interval of three days, 1-2 intermediate time step data are added; for trajectories with an original time step interval of two days, the interval remains unchanged.

[0163] In the filtered dataset, some injection parameter combinations exhibited duplication or narrow coverage. Therefore, new parameter combinations needed to be generated through "parameter perturbation" to enrich the data diversity. Injection parameters that significantly impacted development results and had room for adjustment were selected for perturbation, including polymer concentration, injection rate, and injection time.

[0164] After the perturbation generates new parameters, it is necessary to verify again whether they conform to the "physical and engineering constraints of the injected parameters".

[0165] The new parameters after perturbation are correlated with reservoir state change data and development performance indicators of the original trajectory to generate new trajectories. For example, after parameter perturbation, a certain original trajectory generates 3-5 new trajectories, each corresponding to a set of differentiated injection parameter combinations;

[0166] Through time-step interpolation, 2200 temporally continuous trajectories were obtained from the original 1800 selected trajectories. By injecting parameter perturbations, 3500 new trajectories with differentiated parameter combinations were added to these 2200 trajectories. Finally, a second dataset containing 5700 valid trajectories was formed. This dataset retains the core features of the original high-quality data while enhancing its continuity and diversity through interpolation and perturbation, thus fully supporting the subsequent learning of expert trajectory patterns by the inverse reinforcement learning model.

[0167] This application's embodiments address the issue of inconsistent data quality in the first dataset by filtering data that meets development performance indicators and whose injected parameters are compliant, thus eliminating invalid trajectories with poor development performance or out-of-bounds parameters. This ensures that subsequent model training is based solely on high-quality data, reducing interference from invalid data on model learning. Time-step interpolation enhances the temporal continuity of the trajectories, resolving the problems of numerous time discontinuities and discontinuous reservoir state evolution in traditional datasets. Furthermore, injecting parameter perturbations increases the diversity of parameter combinations, avoiding the problem of insufficient parameter coverage leading to biased model learning.

[0168] While this application provides the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in this embodiment is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the methods shown in this embodiment or the accompanying drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0169] like Figure 3 As shown in the illustration, this application also provides a parameter optimization device 300 for reservoir development. The device includes:

[0170] The acquisition module 301 is used to acquire input parameters during the reservoir development process. The input parameters include oilfield geological parameters, well location and operation parameters, injection parameters and economic evaluation parameters. The input parameters are input into the numerical simulator to obtain the initialized numerical simulator. The numerical simulator is used to calculate reservoir state change data and development effect indicators during the oilfield development process. The reservoir state change data includes at least one of formation pressure distribution, remaining oil distribution and saturation change. The development effect indicators include at least one of oil production, water cut and recovery rate.

[0171] The generation module 302 is used to generate a first dataset based on the numerical simulator and the particle swarm optimization algorithm. The first dataset is used to characterize the correspondence between input parameters, reservoir state change data and development effect indicators.

[0172] Processing module 303 is used to perform filtering and data augmentation on the first dataset to obtain the second dataset;

[0173] The processing module 303 is also used to input multiple expert trajectories into a preset inverse reinforcement learning model, which is used to characterize the dynamic process of the injected parameters acting on the reservoir and the reservoir state evolving over time. The multiple expert trajectories are input into the preset inverse reinforcement learning model respectively to obtain the update amplitude of the policy network in the inverse reinforcement learning model and the probability value of the discriminator in the inverse reinforcement learning model corresponding to the multiple expert trajectories. When the update amplitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold, the expert trajectory with the smallest update amplitude and the largest probability value is determined as the target expert trajectory.

[0174] The optimization module 304 is used to optimize the input parameters based on the target expert trajectory to obtain the optimal input parameters.

[0175] In some embodiments, the processing module 303 is further configured to normalize the input parameters to obtain normalized input parameters;

[0176] The processing module 303 is also used to perform outlier processing on the input parameters, replace outlier values ​​in the input parameters with boundary values, and supplement missing values ​​through linear interpolation and mean filling to obtain the outlier-processed input parameters;

[0177] The processing module 303 is also used to encode the input parameters into an action vector to obtain a parameter vector.

[0178] In some embodiments, the generation module 302 is further configured to initialize a plurality of particles, each particle representing a set of parameter vectors, and configure the inertia weight, learning factor, and maximum number of iterations of the particle swarm optimization algorithm.

[0179] The processing module 303 is also used to iteratively process several examples. In each iteration, the particle velocity and position are updated according to the particle's historical velocity, the individual's historical best position, and the global best position to obtain the parameter vector after iteration.

[0180] The processing module 303 is also used to perform boundary checks on the iterated parameter vector, correcting it to the lower bound value when the parameter vector is below the lower bound of the dimension, and correcting it to the upper bound value when the parameter vector is above the upper bound of the dimension.

[0181] The processing module 303 is also used to input the parameter vector into the numerical simulator to obtain the first dataset.

[0182] In some embodiments, the generation module 302 is further configured to establish an initial inverse reinforcement learning model, input historical input parameters into the policy network and discriminator of the initial inverse reinforcement learning model, and obtain the initial update magnitude and initial probability value.

[0183] The acquisition module 301 is also used to acquire multiple preset expert trajectories from the expert trajectory library, train the discriminator based on the acquired multiple preset expert trajectories and initial probability values ​​to obtain the trained discriminator, and train the policy network based on the acquired multiple preset expert trajectories and initial update magnitude to obtain the trained policy network.

[0184] In some embodiments, the processing module 303 is further configured to perform trajectory legality verification on the target expert trajectory, and to correct the legality of the target expert trajectory if the legality exceeds a preset legality range.

[0185] In some embodiments, the processing module 303 is further configured to perform filtering processing on the first dataset to obtain data whose development effect index is greater than a preset threshold and whose injection parameters meet physical and engineering constraints, thereby obtaining the first dataset after filtering processing;

[0186] The processing module 303 is also used to perform time step interpolation on the first dataset after filtering and to apply a preset amplitude perturbation to the injection parameters to generate new data, thus obtaining the second dataset.

[0187] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0188] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0189] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, such as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0190] This application also provides an apparatus, the apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it implements the method described in this application.

[0191] This application also provides a non-volatile computer-readable storage medium storing a computer program or instructions thereon, which, when executed, enables the method described in this application embodiment to be implemented.

[0192] Furthermore, in the various embodiments of the present invention, each functional module can be integrated into a processing module, or each module can exist independently, or two or more modules can be integrated into a single module.

[0193] The aforementioned storage media include, but are not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions.

[0194] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0195] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0196] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A parameter optimization method for reservoir development, characterized in that, include: The process involves acquiring input parameters during reservoir development, including oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters. These input parameters are then input into a numerical simulator to obtain an initialized numerical simulator. The numerical simulator is used to calculate reservoir state change data and development performance indicators during the oilfield development process. The reservoir state change data includes at least one of formation pressure distribution, remaining oil distribution, and saturation change. The development performance indicators include at least one of oil production, water cut, and recovery rate. A first dataset is generated based on the numerical simulator and the particle swarm optimization algorithm. The first dataset is used to characterize the correspondence between the input parameters, the reservoir state change data, and the development effect indicators. The first dataset is filtered and augmented to obtain the second dataset. The second dataset includes multiple expert trajectories, which are used to characterize the dynamic process of injection parameters acting on the reservoir and the evolution of the reservoir state over time. The multiple expert trajectories are input into a preset inverse reinforcement learning model to obtain the update magnitude of the policy network in the inverse reinforcement learning model and the probability value of the discriminator in the inverse reinforcement learning model corresponding to the multiple expert trajectories. When the update magnitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold, the expert trajectory with the smallest update magnitude and the largest probability value is determined as the target expert trajectory. The input parameters are optimized based on the target expert trajectory to obtain the optimal input parameters.

2. The method according to claim 1, characterized in that, After obtaining the input parameters during the reservoir development process, the method further includes: The input parameters are normalized to obtain normalized input parameters; The input parameters are processed for outliers by replacing outliers with boundary values ​​and by supplementing missing values ​​through linear interpolation and mean imputation, resulting in outlier-processed input parameters. The input parameters are encoded into action vectors to obtain parameter vectors.

3. The method according to claim 2, characterized in that, The generation of the first dataset based on the numerical simulator and the particle swarm optimization algorithm includes: Initialize several particles, each representing a set of parameter vectors, and configure the inertia weight, learning factor, and maximum number of iterations for the particle swarm optimization algorithm. Several examples are processed iteratively. In each iteration, the particle velocity and position are updated based on the particle's historical velocity, individual historical best position, and global best position to obtain the parameter vector after iteration. Boundary checks are performed on the iterated parameter vector, correcting it to the lower bound value when the parameter vector is below the lower bound of the dimension, and correcting it to the upper bound value when the parameter vector is above the upper bound of the dimension. The parameter vector is input into the numerical simulator to obtain the first dataset.

4. The method according to claim 1, characterized in that, Before inputting multiple expert trajectories into a preset inverse reinforcement learning model to obtain the update magnitude of the policy network in the inverse reinforcement learning model corresponding to each expert trajectory and the probability value of the discriminator in the inverse reinforcement learning model, the method further includes: An initial inverse reinforcement learning model is established by inputting historical input parameters into the policy network and discriminator of the initial inverse reinforcement learning model to obtain the initial update magnitude and initial probability value. Multiple preset expert trajectories are obtained from the expert trajectory library. The discriminator is trained based on the multiple preset expert trajectories and the initial probability value to obtain the trained discriminator. The policy network is trained based on the multiple preset expert trajectories and the initial update magnitude to obtain the trained policy network.

5. The method according to claim 1, characterized in that, The step of optimizing the input parameters based on the target expert trajectory to obtain the optimal input parameters further includes: The target expert trajectory is validated for legality. If the legality of the target expert trajectory exceeds a preset legality range, the legality of the target expert trajectory is corrected.

6. The method according to claim 1, characterized in that, The process of filtering and data augmentation on the first dataset to obtain the second dataset includes: The first dataset is filtered to obtain data whose development performance indicators are greater than a preset threshold and whose injection parameters meet physical and engineering constraints, thus obtaining the filtered first dataset. The first dataset after filtering is interpolated at time steps, and the injected parameters are perturbed by a preset amplitude to generate new data, thus obtaining the second dataset.

7. The method according to claim 1, characterized in that, The preset inverse reinforcement learning model is a dual-network static architecture based on generative adversarial imitation learning. The policy network is fixed to a 3-layer fully connected neural network structure, and the discriminator is fixed to a 2-layer fully connected neural network structure.

8. A parameter optimization device for reservoir development, characterized in that, include: The acquisition module is used to acquire input parameters during the reservoir development process. The input parameters include oilfield geological parameters, well location and operation parameters, injection parameters, and economic evaluation parameters. The input parameters are input into the numerical simulator to obtain an initialized numerical simulator. The numerical simulator is used to calculate reservoir state change data and development effect indicators during the oilfield development process. The reservoir state change data includes at least one of formation pressure distribution, remaining oil distribution, and saturation change. The development effect indicators include at least one of oil production, water cut, and recovery rate. The generation module is used to generate a first dataset based on the numerical simulator and the particle swarm optimization algorithm. The first dataset is used to characterize the correspondence between the input parameters, the reservoir state change data and the development effect indicators. The processing module is used to perform filtering and data augmentation on the first dataset to obtain the second dataset; The processing module is further configured to: input the multiple expert trajectories into a preset inverse reinforcement learning model, thereby obtaining the update magnitude of the policy network in the inverse reinforcement learning model and the probability value of the discriminator in the inverse reinforcement learning model corresponding to the multiple expert trajectories; and determine the expert trajectory with the smallest update magnitude and the largest probability value as the target expert trajectory when the update magnitude of the policy network is less than a preset threshold and the probability value of the discriminator is greater than a preset threshold. The optimization module is used to optimize the input parameters based on the target expert trajectory to obtain the optimal input parameters.

9. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images