Task-boundless continuous learning production dynamic prediction method based on playback mechanism

Abstract

The application discloses a task-boundary-free continuous learning production dynamic prediction method based on a playback mechanism and belongs to the field of oil engineering prediction, mainly constructs a double-scale attention feature extraction module containing a block embedding de-non-stationary short-term branch and a block down-sampling reserved non-stationary long-term branch to perform global time sequence feature extraction; in the case of no explicit task boundary, a double-buffer playback memory module based on a playback mechanism is designed to perform hybrid sampling training; a platform period trigger decision module is constructed based on a platform trigger mechanism to adaptively determine a stable state of model learning and trigger the playback mechanism training; a time-frequency domain fusion loss optimization module is designed to realize the task-boundary-free continuous learning. The application does not need to be retrained when new data comes, significantly reduces the catastrophic forgetting, enhances the prediction robustness and online convergence stability, and is suitable for practical production data prediction scenes.

Description

Technical Field

[0001] This invention belongs to the field of petroleum engineering technology, specifically relating to a task-boundary-free continuous learning production dynamic prediction method based on a replay mechanism. Background Technology

[0002] In the exploration and development of oil and gas fields, the results of production dynamic forecasting have a decisive impact on development benefits. Currently, due to the powerful feature extraction and nonlinear fitting capabilities of neural networks, deep learning-based industrial time series forecasting methods have made significant progress in key parameter prediction and soft measurement modeling.

[0003] As new data continuously arrives and well network structures change, traditional prediction models primarily update their training using two methods: one is to retrain the model using all available data, which significantly increases data storage and computational time costs. The other method is to fine-tune the model with new data, but this incremental update method faces the problem of catastrophic forgetting of previously learned knowledge. However, continuous learning strategies offer a solution for effectively training and updating neural networks on non-stationary industrial data. Continuous learning, also known as lifelong learning, refers to the ability of a learning system to dynamically update its model's knowledge over time without forgetting previously learned important information, thus effectively overcoming the challenge of catastrophic forgetting. Furthermore, incremental learning methods avoid the high computational costs of modeling from scratch.

[0004] Traditional continuous learning methods often rely on predefined task partitioning, meaning the model knows the start and end boundaries of each task during the learning process and adjusts parameters or manages replays when switching tasks. However, this assumption doesn't hold true in many practical applications, such as autonomous driving, service robots, medical assistance systems, and real-world production data prediction. In these scenarios, data is presented as a continuous stream, and the emergence of new knowledge often lacks clear boundaries. In such cases, traditional continuous learning methods often fail to adapt effectively due to their dependence on task information, exacerbating the catastrophic forgetting problem. However, in real-world data, data streams are constantly arriving without clear task boundaries, and it's unknown when new tasks will appear.

[0005] Therefore, there is an urgent need to propose a task-boundary-free continuous learning method to improve the accuracy of production dynamics prediction, prevent catastrophic data forgetting, and enhance the overall economic benefits of oilfield development. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a task-boundary-free continuous learning method for dynamic production prediction based on a replay mechanism. It utilizes a non-stationary time series model combined with continuous learning to solve the catastrophic forgetting problem during training. Addressing the prevalent characteristics of strong noise, numerous missing values, local mutations, and overall non-stationarity in industrial scenarios, this invention proposes a continuous learning method that requires no task boundaries, possesses long-term memory retention capabilities, and is more robust to noise and non-stationarity. Through dual-scale modeling combining block embedding and block downsampling with an attention mechanism, a dual-buffering system, time series mask reconstruction, and time-frequency domain Huber loss fusion optimization, the prediction model achieves long-term stability and continuous improvement in dynamic environments.

[0007] The technical solution of the present invention is as follows:

[0008] A task-boundary-free continuous learning production dynamic prediction method based on a replay mechanism is proposed. A replay-based task-boundary-free continuous learning production dynamic prediction model is constructed, which includes a data preprocessing module, a dual-scale attention feature extraction module, a dual-buffered replay memory module, a plateau-triggered decision module, a time-frequency domain fusion loss optimization module, and a prediction output module. The method specifically includes the following steps:

[0009] Step 1: Obtain actual production data of the reservoir, input it into the data preprocessing module to complete data preprocessing, and generate time series data;

[0010] Step 2: Input the time series data into the dual-scale attention feature extraction module to obtain global time series features, and input the global time series features into the prediction output module to obtain the predicted liquid production value;

[0011] Step 3: Construct a double-buffered replay memory module based on the replay mechanism for hybrid sampling training;

[0012] Step 4: Construct a platform period trigger decision module based on the platform trigger mechanism, and adaptively determine the model to learn the stable state and trigger the replay mechanism of step 3 for training;

[0013] Step 5: Design a time-frequency domain fusion loss optimization module to drive model parameter training and updates, and achieve continuous learning without task boundaries;

[0014] Step 6: Repeat steps 2 to 4 until the training termination condition is met, and obtain the trained production dynamic prediction model.

[0015] Step 7: Obtain the actual daily production data of the reservoir in real time, input it into the trained production dynamic prediction model, and obtain the daily liquid production of the reservoir.

[0016] Furthermore, in step 1, the data preprocessing process is as follows:

[0017] Step 1.1: Obtain three years of actual production data from reservoir water injection wells and production wells, and perform minimum-maximum normalization;

[0018] Step 1.2: Fill the normalized data with the mean to form time series data, and then divide it into training set, validation set and test set according to the proportion;

[0019] Step 1.3: The starting position of the training set remains unchanged at 0, and the ending position is 70% of the total length of the original sequence. The starting positions of the validation set and test set intervals are corrected by order offset: the starting position of the validation set is offset from the ending position of the training set by a step length equal to the length of the input sequence in the past direction of the time axis, and boundary checks are used to ensure that the starting position is not less than 0, and the ending position is 80% of the total length of the original sequence; the starting position of the test set is offset from the ending position of the validation set by a step length equal to the length of the input sequence in the past direction of the time axis, while ensuring that the starting position is not lower than the ending position of the validation set, and the ending position is 100% of the total length of the original sequence.

[0020] Furthermore, the specific process of step 2 is as follows:

[0021] Step 2.1: Process the input time series data Perform block embedding to obtain the block-based embedding representation. :

[0022] ;

[0023] ;

[0024] in, For the first One data block; This is a block operation; For linear layer embedding operations;

[0025] Step 2.2, for Perform block-by-block detrending:

[0026] ;

[0027] ;

[0028] ;

[0029] in, For the first Data blocks Trend items; It is an average pooling operation; It is a fill operation; , The first The, the A detrended data block; For the first The series after detrending of each variable It is a slice symbol;

[0030] Step 2.3: Use a short-term attention layer to capture short-term temporal dependencies within the same variable:

[0031] ;

[0032] ;

[0033] in, For the first These are intermediate variables in the output of the short-term attention layer. Representation layer normalization; For the first The original sequence of variables; For short-term attention layer; This represents a multilayer feedforward neural network; Indicates the first The feature sequence of each variable obtained through a short-term attention layer;

[0034] Step 2.4: Reconstruct the mask based on the segmented sequence:

[0035] ;

[0036] in, This is the sequence output after mask reconstruction; It is a mask matrix; It is the inverse mask matrix; The symbol for element-wise product;

[0037] Step 2.5: Perform block downsampling:

[0038] ;

[0039] in, For the first The sequence after block downsampling of each variable; For block downsampling;

[0040] Step 2.6: Apply long-term attention to each downsampled block sequence to model the cointegration relationship among all variables over the same time interval:

[0041] ;

[0042] ;

[0043] in, For the first The intermediate tensor of the long-term attention layer output at each time step; For the first The original sequence at each time step; For the first The sequence after downsampling in blocks at each time step; For long-term attention layer; It is the first The feature sequence obtained from the long-term attention layer at each time step;

[0044] Step 2.7: Concatenate the feature sequences obtained from the long and short attention layers to obtain the global temporal features. :

[0045] ;

[0046] in, This is a splicing symbol;

[0047] Step 2.8: The global temporal features are processed through a two-layer cascaded fully connected network, combined with a hybrid sampling strategy using a double-buffered playback memory module, and spliced ​​with the current batch to form a continuous learning training batch, ultimately outputting the predicted liquid production value.

[0048] Furthermore, the specific process of step 3 is as follows:

[0049] Step 3.1: Define two playback buffers in the double-buffered playback memory module: a recent buffer and a global buffer;

[0050] The sampling strategy for the recent buffer is to directly select the previous one. The sample with the minimum loss:

[0051] ;

[0052] ;

[0053] in, For recent buffer The set of samples that need to be retained; The sample with the minimum loss is selected. It is the first Sample and the Sample The loss function values ​​between; For recent buffer sets; To get from the recent buffer Further screening of the top A set consisting of samples with minimum loss; Before selection Minimum loss; , For two different samples; The value of the loss function;

[0054] The sampling strategy for the global buffer is to sample from the global buffer. Medium sampling When there are only one sample, perform a uniform random selection directly:

[0055] ;

[0056] in, For the sample The probability of being selected by sampling from the global buffer; Represents the global buffer The amount of data;

[0057] Step 3.2: During training, samples are randomly added to the recent buffer or global buffer; if the recent buffer or global buffer is not full, samples are directly added to it; when the recent buffer or global buffer is full, samples are randomly deleted from it during each subsequent sample addition process; in the... In the next update, if the sample is included in the global buffer The probability is And the replacement probability is Then the first one in the recent buffer or global buffer The probability that a sample is replaced is The survival probability is ;

[0058] No. The probability that a sample will eventually remain in the recent buffer or the global buffer is :

[0059] ;

[0060] in, For time steps;

[0061] Step 3.3: Design a continuous learning double-buffered hybrid sampling strategy: Combine the recent buffer and the global buffer. During training, samples are taken from both buffers and then concatenated with the current batch to form the final training batch, thus creating a hybrid sampling distribution. :

[0062] ;

[0063] in, It controls the weights of the two buffers; This is the set of recent buffer samples; This is the global buffer sample set.

[0064] Furthermore, the specific process of step 4 is as follows:

[0065] Step 4.1, set the time step The current batch loss is Maintain a length of Loss sliding window:

[0066] ;

[0067] in, For time steps Loss sliding window;

[0068] Step 4.2, in The mean and standard deviation of the batch loss were calculated internally:

[0069] ;

[0070] ;

[0071] in, , They are time steps The mean and standard deviation; For batch losses; For time steps The amount of data lost in the sliding window;

[0072] Step 4.3: Pre-set two thresholds: one is the mean threshold. The other is the standard deviation threshold. Only when the mean of the loss within the sliding window is less than And the standard deviation of the loss within the sliding window is less than At this point, it is determined that the model learning has stabilized; in addition, a state flag is introduced. This is used to control whether the system enters the platform's testing phase. , The two state values ​​are the state flag; when the state flag is When the platform testing phase begins, the replay mechanism training in step 3 is initiated; once the platform testing phase is complete, the status flag will be automatically set to [value]. ;

[0073] Step 4.4: Record the mean and standard deviation of the platform after the model learning stabilizes, denoted as follows: and As a reference baseline for the next round of platform testing, that is , , To update the symbols; then clear the loss sliding window, and simultaneously... Set as End this platform test; settings ;in, It is a two-dimensional nonnegative real number space;

[0074] Step 4.5: If the mean within the loss sliding window is greater than the sum of the mean and standard deviation of the plateau after the model stabilizes in the previous learning iteration, perform plateau detection again; at this time, Reset to We have entered a new platform testing phase.

[0075] Furthermore, in step 5, the temporal Huber loss, frequency domain Huber loss, and playback loss are fused to construct the total loss function;

[0076] Time-domain Huber loss The formula is:

[0077] ;

[0078] in, For threshold Huber's losses; This is a predicted value; The actual value;

[0079] Frequency domain Huber loss The formula is:

[0080] ;

[0081] in, is the real part of the complex number; It represents the imaginary part of a complex number; The spectrum of the real sequence; To predict the spectrum of the sequence;

[0082] Playback loss The formula is:

[0083] ;

[0084] Final total loss function for:

[0085] ;

[0086] in, This is a hyperparameter.

[0087] Furthermore, in step 6, the training termination condition is reaching the preset maximum number of iterations or triggering the early stopping mechanism; the condition for triggering the early stopping mechanism is: during the training iteration process, after each round of parameter updates, the total loss function is first calculated for the validation set and the test set respectively. If the loss value of the validation set is higher than the loss value of the test set in three consecutive iterations, the early stopping mechanism is triggered immediately to terminate subsequent iterations, and the optimal model parameters before the early stopping mechanism was triggered are retained.

[0088] The beneficial technical effects of this invention are as follows: Compared with existing technologies, this invention does not require retraining when new data arrives, significantly reduces catastrophic forgetting, enhances prediction robustness and online convergence stability, and is applicable to scenarios such as intelligent driving, service robots, medical assistance, and prediction of actual production data. First, this invention enhances the model's robust representation ability against noisy data, missing data, and non-stationary sequences through self-supervised tasks of block embedding and mask reconstruction, improving the model's stability and generalization. Second, by introducing a double buffer mechanism and a plateau detection strategy, representative historical samples are dynamically selected for knowledge replay without predefined task boundaries, balancing model stability and plasticity and significantly alleviating the catastrophic forgetting problem. Finally, a Huber loss function with joint constraints in the time and frequency domains is proposed, enabling the model to more accurately capture complex temporal evolution characteristics and further improve its adaptability to dynamic data distributions. The method of this invention has been validated on real oilfield datasets, significantly outperforming existing mainstream baseline methods in terms of prediction error, anti-forgetting ability, and forward and backward transfer performance. This method provides a reliable technical approach for continuous, robust, and efficient time-series forecasting in dynamic industrial environments, and has significant application value in supporting intelligent development and forecasting of oilfields. Attached Figure Description

[0089] Figure 1 This is a flowchart of the task-boundary-free continuous learning production dynamic prediction method based on a replay mechanism according to the present invention.

[0090] Figure 2 This is a comparison chart of the average error of different continuous learning methods in the experiments of this invention.

[0091] Figure 3 This is a comparison chart of forgetting values ​​for different continuous learning methods in the experiments of this invention.

[0092] Figure 4 This is a comparison diagram of backward transfer using different continuous learning methods in the experiments of this invention.

[0093] Figure 5 This is a comparison diagram of forward transfer using different continuous learning methods in the experiments of this invention.

[0094] Figure 6This is a graph showing the average error of different continuous learning methods on different tasks in the experiments of this invention.

[0095] Figure 7 This is a graph showing the forgetting values ​​of different continuous learning methods on different tasks in the experiments of this invention.

[0096] Figure 8 This is a backward transfer value graph of different continuous learning methods on different tasks in the experiments of this invention.

[0097] Figure 9 This is a forward transfer value graph of different continuous learning methods on different tasks in the experiments of this invention.

[0098] Figure 10 This is a comparison chart of the mean squared error values ​​of different continuous learning methods in the experiments of this invention.

[0099] Figure 11 This is a comparison chart of the mean absolute error values ​​of different continuous learning methods in the experiments of this invention.

[0100] Figure 12 This is a comparison chart of the average absolute percentage error values ​​of different continuous learning methods in the experiments of this invention.

[0101] Figure 13 This is a comparison chart of the root mean square error values ​​of different continuous learning methods in the experiments of this invention.

[0102] Figure 14 This is a visualization of the actual and predicted values ​​of well A after using the method of this invention in the experiment.

[0103] Figure 15 This is a visualization of the actual and predicted values ​​of well B after using the method of this invention in the experiment. Detailed Implementation

[0104] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0105] like Figure 1 As shown, a task-boundary-free continuous learning production dynamic prediction method based on a replay mechanism is proposed. A replay-based task-boundary-free continuous learning production dynamic prediction model is constructed, which includes a data preprocessing module, a dual-scale attention feature extraction module, a dual-buffered replay memory module, a plateau-triggered decision module, a time-frequency domain fusion loss optimization module, and a prediction output module. The prediction output module employs a two-layer fully connected network. The input of the model is the actual daily production data, and the output is the daily liquid production volume. The specific steps include:

[0106] Step 1: Obtain actual production data from the reservoir, input it into the data preprocessing module to complete Min-Max normalization, missing value imputation, and sliding window sampling, and generate standard time series samples; including the following steps:

[0107] Step 1.1: Obtain three years of actual production data from reservoir water injection wells and production wells, and perform Min-Max normalization;

[0108] Normalizing data can not only improve prediction accuracy, but also make the search for the optimal solution smoother for many models, making it easier to converge to the optimal solution correctly.

[0109] Step 1.2: Fill the normalized data with the mean to form time series data, and then divide it according to the ratio, where the ratio of training set, validation set and test set is set to 7:1:2 respectively;

[0110] Step 1.3: To ensure the integrity of the sequence length during sliding window sampling, the starting positions of the validation and test set intervals were corrected by sequential offset when dividing the boundaries: the starting position of the training set remained unchanged at 0, and the ending position was 70% of the original sequence length; the starting position of the validation set was offset from the ending position of the training set by a step equal to the input sequence length (seq_len) in the past direction along the time axis, and boundary checks ensured that the starting position was not less than 0, and the ending position was 80% of the original sequence length; the starting position of the test set was also offset from the ending position of the validation set by seq_len in the past direction along the time axis, while ensuring that the starting position was not lower than the ending position of the validation set to avoid data overlap, and the ending position was 100% of the original sequence length. This correction method ensures that each sliding window sample in the validation and test sets can obtain a complete historical sequence of length seq_len, avoiding issues such as index out-of-bounds errors or incomplete sequences. After the partitioning, the training set is used to optimize the trainable parameters (weights, biases, etc.) of the dual-scale attention feature extraction module and the fully connected network. The validation set is used to determine hyperparameters such as the sliding window length and buffer weights and to perform early stopping based on the validation set error. The test set is used to evaluate the model's final prediction accuracy, anti-forgetting ability, and other generalization performance metrics.

[0111] Step 2: The time series data is a non-stationary time series. The time series data is input into a dual-scale attention feature extraction module for global feature extraction. This module includes two branches: block embedding to remove non-stationary short-term branches and block downsampling to retain non-stationary long-term branches. Block embedding to remove non-stationary short-term branches captures reliable short-term time dependencies within variables through block embedding, detrending, and short-term attention calculations. Simultaneously, block downsampling to retain non-stationary long-term branches models long-term cointegration relationships across variables through block downsampling and long-term attention calculations. Finally, the two types of features from the two branches are concatenated to obtain the global time series features. Specifically, the following steps are included:

[0112] Step 2.1: Divide the input time series data into blocks and embed them to obtain the block-based embedding representation. :

[0113] (1);

[0114] (2);

[0115] in, For the first One data block; This is a block operation; It is time series data; This is a linear layer embedding operation, where the linear layer is trainable;

[0116] After segmentation, the embedded representation These represent short-term non-stationary sequences, which are easily affected by external disturbances. Directly performing attention modeling on these non-stationary sequences often results in spurious regressions, leading to invalid or false dependencies. Therefore, step 2.3 uses short-term attention to first eliminate short-term non-stationarity before performing attention calculations within each univariate. This eliminates the impact of short-term fluctuations, focuses on the time dependencies within each variable, and reduces the risk of spurious regressions.

[0117] Step 2.2: To address the issue of spurious regression that may result from modeling time correlations within the same variable, we first analyze the block-based embedding representation. Perform block-by-block detrending:

[0118] (3);

[0119] (4);

[0120] (5);

[0121] in, For the first Data blocks Trend items; It is an average pooling operation, used to extract a moving average for each data block to obtain detrended data blocks; It is a fill operation; For the first One data block; , The first The, the A detrended data block; For the first The series after detrending (removing non-stationarity) of each variable. It is a slice symbol, representing all elements in that dimension.

[0122] Step 2.3: Use the proposed short-term attention layer to capture short-term temporal dependencies within the same variable:

[0123] (6);

[0124] (7);

[0125] in, For the first These are intermediate variables in the output of the short-term attention layer. Representation layer normalization; For the first The original sequence of variables; For short-term attention layer; This represents a multilayer feedforward neural network; Indicates the first The feature sequence of each variable is obtained through a short-term attention layer.

[0126] This invention uses detrended sequences The query and key serve as the short-term attention layer; using the original sequence. This serves as the value for the short-term attention layer. Through this short-term attention layer operation, time dependencies can be effectively modeled without being affected by the short-term non-stationary nature of the sequence.

[0127] Step 2.4: Perform mask reconstruction based on the segmented sequence. In the mask reconstruction task, some time points will be randomly masked and set to zero. The input after masking is:

[0128] (8);

[0129] in, This is the sequence output after mask reconstruction; It is a mask matrix; It is the inverse mask matrix; This is the element-wise product sign; in actual calculations, corresponding elements of two vectors of the same dimension are multiplied.

[0130] Step 2.5: Before modeling cointegration among variables, increasing the amount of global information represented by each block is crucial. Block downsampling allows the model to gradually extract long-term dependency structures from local fluctuations, providing a more compact and efficient representation for subsequent cross-variable relationship modeling of long-term attention. The specific formula is:

[0131] (9);

[0132] in, For the first The sequence after block downsampling of each variable; For block downsampling, used to... middle Compressing data blocks into fewer indivual, This process is implemented by a multilayer feedforward neural network (MLP).

[0133] No. The sequence after block downsampling of each variable As a query in the long-term attention layer, the first The original sequence of variables Serving as keys and values ​​in long-term attention layers, each downsampled token dynamically integrates global contextual information during attention interactions, covering a broader temporal range. Ultimately, the model learns richer long-term dependency features while maintaining computational efficiency, effectively capturing cointegration relationships between variables that only emerge over longer time scales.

[0134] Step 2.6: This invention proposes a long-term attention layer, which introduces a long-term attention mechanism that can directly model the cointegration relationship between sequences. Specifically, long-term attention is applied to each downsampled block sequence to model the cointegration relationship between sequences over the same time interval. Cointegration relationships among all variables. For the first The original sequence at each time step, Let be a real vector space with dimension . , , For the dimension values ​​in two different directions:

[0135] (10);

[0136] (11);

[0137] in, For the first The intermediate tensor of the long-term attention layer output at each time step; For the first The original sequence at each time step; For the first The sequence after downsampling in blocks at each time step; For long-term attention layer; It is the first The feature sequence obtained by long-term attention at each time step;

[0138] Step 2.7: Concatenate the feature sequences obtained from the long and short attention layers to obtain the global temporal features:

[0139] (12);

[0140] in, This is a splicing symbol; The spliced ​​global temporal features; This represents the real vector space containing the concatenated features, with dimension 1. , , These are the dimension values ​​of the concatenated features in two different directions;

[0141] Step 2.8: The global temporal features are processed through a two-layer cascaded fully connected network, combined with a hybrid sampling strategy using a double-buffered playback memory module, and spliced ​​with the current batch to form a continuous learning training batch, ultimately outputting the predicted liquid production value.

[0142] Step 3: In the absence of a clearly defined task boundary, this invention introduces a dual-buffering system. Specifically, a hybrid sampling strategy based on a replay mechanism is designed in the dual-buffered replay memory module. Two replay buffers are defined: a recent buffer and a global buffer. The global buffer retains long-term representative samples, while the recent buffer retains samples with smaller errors. During training, samples are randomly added to either the recent buffer or the global buffer. If either the recent buffer or the global buffer is full, samples are randomly deleted. This includes the following steps:

[0143] Step 3.1: Define two playback buffers: the recent buffer and the global buffer.

[0144] The sampling strategy for the recent buffer is to directly select the previous one. The sample with the minimum loss:

[0145] (13);

[0146] (14);

[0147] in, For recent buffer The set of samples that need to be retained; The sample with the minimum loss is selected. It is the first Sample and the Sample The loss function values ​​between; For recent buffer sets; To get from the recent buffer Further screening of the top A set consisting of samples with minimum loss; Before selection Minimum loss; , For two different samples; The value of the loss function;

[0148] The purpose of the global buffer is to retain a subset of samples from the entire historical data stream with equal probability, ensuring that the model's memory is not limited to recent data but also covers the complete historical distribution. The sampling strategy of the global buffer is to... Medium sampling When there are only one sample, perform a uniform random selection directly:

[0149] (15);

[0150] in, For the sample The probability of being selected by sampling from the global buffer; Represents the global buffer The amount of data;

[0151] Step 3.2: If the recent buffer or global buffer is not full, the sample is directly added to the recent buffer or global buffer; when the recent buffer or global buffer is full, it is added to the recent buffer or global buffer in each subsequent sample. During the process, samples in the recent buffer or global buffer may be replaced. Next update If a sample is included in the global buffer The probability is And the replacement probability is Then the first one in the recent buffer or global buffer The probability that a sample is replaced is Therefore, the survival probability is This ensures that each historical sample in the final recent buffer or global buffer has an equal probability of being retained; where, This represents the current total number of samples;

[0152] The first The probability that a sample will eventually remain in the recent buffer or the global buffer is :

[0153] (16);

[0154] in, For time steps;

[0155] Step 3.3: Design a continuous learning double-buffered mixed sampling strategy: This involves combining the recent buffer and the global buffer. During training, samples are taken from both buffers and then concatenated with the current batch to form the final training batch, thus creating a mixed sampling distribution. :

[0156] (17);

[0157] in, It controls the weights of the two buffers; This is the set of recent buffer samples; This is the global buffer sample set.

[0158] Step 4: The plateau trigger decision module establishes a plateau trigger mechanism. This module maintains a loss sliding window, calculates the mean and standard deviation of the batch loss within the window, adaptively determines the model's learning of a stable state, and triggers the replay mechanism training in Step 3. First, an empty loss window is defined. Store the most recent loss value. If the mean and standard deviation of the current loss sliding window are both less than the set threshold, it indicates that the model is stable on the current platform. Then, initiate replay, randomly taking a fixed-size sample from the recent buffer or the global buffer, and calculating a replay loss. Specifically, this includes the following steps:

[0159] Step 4.1, set the time step The current batch loss is Maintain a length of Loss sliding window:

[0160] (18);

[0161] in, For time steps Loss sliding window;

[0162] Step 4.2: Calculate the mean and standard deviation of the batch loss in this window:

[0163] (19);

[0164] (20);

[0165] in, , They are time steps The mean and standard deviation; For batch losses; For time steps The amount of data lost in the sliding window;

[0166] Step 4.3: To formally determine whether a plateau has been reached, two thresholds are needed: one is the mean threshold. The other is the standard deviation threshold. Only when the mean of the loss within the sliding window is less than And the standard deviation of the loss within the sliding window is less than Only when the model's learning is considered stable is the model considered to have reached this point. Furthermore, a state flag is introduced. This is used to control whether the system enters the platform's testing phase. , For the two state values ​​of the state flag; when it is At this point, the platform testing phase begins, and the replay mechanism training in step 3 is initiated; once the platform testing phase is complete, it will be automatically set to [a new state]. ;

[0167] Step 4.4: To avoid frequent misjudgments, the mean and standard deviation of the platform after the last model learning stabilized will also be recorded, denoted as follows: and As a reference baseline for the next round of testing, i.e. , , To update the symbols. Afterwards, the loss sliding window needs to be cleared to recalculate subsequent changes, and at the same time... Set as This concludes the testing process. To enhance robustness, this condition can be required to be satisfied continuously for several steps, thereby reducing misjudgments caused by random fluctuations. ;in, It is a two-dimensional nonnegative real number space;

[0168] Step 4.5: If the mean within the loss sliding window is greater than the sum of the mean and standard deviation of the plateau after the previous model learning stabilizes (i.e., ... , If the value is less than the mean, it indicates a significant change in the data distribution, meaning new difficulties have arisen and retesting is necessary. In this case, [the data should be]... Reset to (Right now This will allow the platform to enter a new testing phase, so that it can be reassessed to determine whether it has reached a stable state.

[0169] Step 5: Design a time-frequency domain fusion loss optimization module to drive model parameter training and updates, enabling continuous learning without task boundaries, effectively balancing model stability and plasticity, and mitigating catastrophic forgetting problems. This module integrates time-domain Huber loss, frequency-domain Huber loss, and replay loss to construct the total loss function. Specifically, it includes the following steps:

[0170] Step 5.1, let the length be... The true sequence is The predicted sequence output by the model is The batch size is [number]. Using threshold Huber's losses :

[0171] (21);

[0172] in, This is the error term. The Huber loss is approximately quadratic for small errors and approximately linear for large errors, combining the smoothness of MSE (mean squared error) with the robustness of MAE (mean absolute error).

[0173] Step 5.2: In the time domain, design the time-domain Huber loss. Huber loss is used to measure the predicted value. Compared with the true value Differences between them:

[0174] (twenty two);

[0175] Step 5.3: In the frequency domain, first analyze the predicted sequence... and the real sequence Performing Fast Fourier Transform (FFT) on each component yields its complex spectral representation:

[0176] (twenty three);

[0177] (twenty four);

[0178] in, The spectrum of the real sequence; To predict the spectrum of the sequence; For Fast Fourier Transform;

[0179] Subsequently, the Huber loss is calculated on both the real and imaginary parts to obtain the frequency domain Huber loss. To constrain the consistency between the predicted signal and the true signal in terms of frequency components:

[0180] (25);

[0181] in, is the real part of the complex number; It represents the imaginary part of a complex number;

[0182] Step 5.4: Calculate a playback loss using playback samples. :

[0183] ;(26);

[0184] Step 5.5, Final Total Loss Function It is composed of a weighted fusion of the time-domain Huber loss and the frequency-domain Huber loss, plus the playback loss, and is defined as:

[0185] (27);

[0186] Among them, hyperparameters The trade-off between time-domain and frequency-domain losses. When When, the loss degenerates into a purely time-domain constraint; when At this time, the loss depends entirely on the frequency domain characteristics. In practical tasks, it can be adjusted based on the stationarity and periodicity of the data. .

[0187] Step 6: Repeat steps 2 to 4 until the preset maximum number of iterations is reached or the loss on the validation set fails to decrease after three iterations, resulting in premature stopping. This yields the trained production dynamic prediction model. During the training iteration process, after each round of parameter updates, the total prediction loss function is calculated for both the validation and test sets. If the loss value of the validation set is higher than that of the test set in three consecutive iterations, it is determined that the model has an overfitting trend. The early stopping mechanism is immediately triggered to terminate subsequent iterations, and the optimal model parameters before the early stopping mechanism is triggered are retained. After training, forward inference is performed on the complete validation set based on the optimal parameters to confirm the effectiveness of the hyperparameter combination. At the same time, prediction accuracy indicators such as MAE and MSE are calculated, the anti-forgetting value, forward and backward transfer values ​​are evaluated, and the visualization results of the baseline method are compared.

[0188] Step 7: Obtain the actual daily production data of the reservoir in real time, input it into the trained production dynamic prediction model, and obtain the daily liquid production of the reservoir.

[0189] Taking an oil reservoir in a certain study area as an example, a task-boundary-free continuous learning production dynamic prediction method based on a playback mechanism, as described above, is used to dynamically predict the production dynamics of this study area. The dataset used in this study comes from an actual oilfield production block and contains daily water injection and production monitoring data for three years. The dataset consists of time series data, daily water injection volume of multiple injection wells, and daily production volume of oil wells, completely recording the production dynamics for 1096 days from January 1, 2021 to January 1, 2024. This data is then used to perform predictions using the task-boundary-free continuous learning production dynamic prediction method based on a playback mechanism proposed in this invention.

[0190] To verify the superiority of the replay method proposed in this invention compared to traditional continuous learning methods, a comparative experiment was conducted. This invention compares the experimental results of its continuous learning method with those of seven other continuous learning methods (including two regularization methods and five replay methods). The continuous learning methods are described below:

[0191] MAS: As a baseline method, it is a continuous learning regularization method based on parameter importance. It estimates the sensitivity of the model output to each parameter and imposes strong constraints on important parameters.

[0192] SI: During training, accumulate the "importance" of each parameter to loss reduction online, and use this weighted quadratic regularization to constrain important parameters to approach previous values ​​in new tasks.

[0193] ER: In each training step, the parameters are jointly updated by splicing the current task's mini-batch with the mini-batch randomly drawn from the playback memory. The buffer adopts a hybrid strategy of FIFO as the main method and a small amount of random replacement.

[0194] DER: Stores the logits of past samples on the training trajectory in a reservoir buffer in memory, and uses Euclidean distance to constrain the current output to match these "dark knowledge" during subsequent learning.

[0195] MIR: First, perform a virtual update on the current batch, then select the most disturbed sample from the buffer according to the "loss increase" and replay it, using the minimum replay budget to repair the old knowledge that is most "disturbed" by the new knowledge.

[0196] SER: Based on experience replay, it introduces two distillation losses: "backward consistency" (aligning the logits of the old and new models with old samples in the buffer) and "forward consistency" (using the current data to imitate the prediction of the old model for future tasks). The joint constraints are used to suppress model drift, reduce forgetting and improve generalization of old tasks.

[0197] OCAR: Models online replay as a second-order joint optimization with KL stability constraints: the gradient is preconditioned with natural gradients using the Fisher information matrix, the FIM of the replay batch is used as a "stabilizer", and it is combined with Tikhonov regularization and hyperparameter scheduling.

[0198] Figures 2 to 5 The figure shows a comparison between the method of this invention and other continuous learning methods under the core metrics. The results show that the method of this invention achieves the best results in the three core metrics (mean error, forgetting value, backward transfer, and forward transfer) and the second best results in forward transfer, demonstrating a significant comprehensive performance advantage.

[0199] Figure 6 The graph shows the average error of different continuous learning methods on different tasks in the experiments of this invention. The average error index shows that the average error value increases slightly with the increase of the number of tasks. This indicates that, given a limited model capacity, the ability to memorize tends to weaken as knowledge accumulates; in addition, interference between different tasks also leads to an increase in the average error. However, the method of this invention maintains the lowest error throughout the entire learning process.

[0200] Figure 7 The diagram shows the forgetting values ​​of different continuous learning methods on different tasks in the experiment of this invention. The method of this invention does not show obvious resistance to forgetting in the initial task, but its performance is significantly improved after training in the last task.

[0201] Figure 8 and Figure 9 The images show the backward and forward transfer value maps of different continuous learning methods on different tasks in the experiments of this invention. For forward transfer, the performance differences between the various methods are relatively small, which further emphasizes that the model achieves a balance between stability and plasticity.

[0202] Figures 10 to 13 The figures show the error values ​​of different continuous learning methods in the experiments of this invention on various evaluation metrics (including mean absolute error, mean squared error, mean absolute percentage error, and root mean square error). The results show that the traditional experience replay (ER) and its variants are relatively stable in overall performance, but still have certain limitations in error control. The regularization-based methods show large fluctuations in the metrics, indicating insufficient robustness in complex time-series scenarios. In contrast, the method of this invention achieves the best results on all four metrics, significantly outperforming existing methods.

[0203] Figure 14 , Figure 15 These are visualizations of the actual and predicted values ​​of wells A and B after applying the method of this invention in the experiment. The visualization of the prediction results shows that the method of this invention can closely approximate the actual value.

[0204] In summary, the method of this invention effectively solves the challenges of catastrophic forgetting in continuous learning and the large amount of resource consumption in cumulative learning. Compared with powerful baseline models, it achieves significant performance improvements in prediction accuracy, anti-forgetting ability, and forward and backward transfer capabilities.

[0205] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A task-boundary-free continuous learning production dynamic prediction method based on a replay mechanism, characterized in that, A task-boundary-free continuous learning dynamic prediction model with a replay mechanism was constructed. This model includes a data preprocessing module, a dual-scale attention feature extraction module, a dual-buffered replay memory module, a plateau-triggered decision module, a time-frequency domain fusion loss optimization module, and a prediction output module. The specific steps of this method are as follows: Step 1: Obtain actual production data of the reservoir, input it into the data preprocessing module to complete data preprocessing, and generate time series data; Step 2: Input the time series data into the dual-scale attention feature extraction module to obtain global time series features, and input the global time series features into the prediction output module to obtain the predicted liquid production value; Step 3: Construct a double-buffered replay memory module based on the replay mechanism for hybrid sampling training; the specific process is as follows: Step 3.1: Define two playback buffers in the double-buffered playback memory module: a recent buffer and a global buffer; The sampling strategy for the recent buffer is to directly select the previous one. The sample with the minimum loss: ； ； in, For recent buffer The set of samples that need to be retained; The sample with the minimum loss is selected. It is the first Sample and the Sample The loss function values ​​between; For recent buffer sets; To get from the recent buffer Further screening of the top A set consisting of samples with minimum loss; Before selection Minimum loss; , For two different samples; The value of the loss function; The sampling strategy for the global buffer is to sample from the global buffer. Medium sampling When there are only one sample, perform a uniform random selection directly: ； in, For the sample The probability of being selected by sampling from the global buffer; Represents the global buffer The amount of data; Step 3.2: During training, samples are randomly added to the recent buffer or global buffer; if the recent buffer or global buffer is not full, samples are directly added to it; when the recent buffer or global buffer is full, samples are randomly deleted from it during each subsequent sample addition process; in the... In the next update, if the sample is included in the global buffer The probability is And the replacement probability is Then the first one in the recent buffer or global buffer The probability that a sample is replaced is The survival probability is ; No. The probability that a sample will eventually remain in the recent buffer or the global buffer is : ； in, For time steps; Step 3.3: Design a continuous learning double-buffered hybrid sampling strategy: Combine the recent buffer and the global buffer. During training, samples are taken from both buffers and then concatenated with the current batch to form the final training batch, thus creating a hybrid sampling distribution. : ； in, It controls the weights of the two buffers; This is the set of recent buffer samples; For the global buffer sample set; Step 4: Construct a platform period trigger decision module based on the platform trigger mechanism, and adaptively determine the model to learn the stable state and trigger the replay mechanism of step 3 for training; Step 5: Design a time-frequency domain fusion loss optimization module to drive model parameter training and updates, and achieve continuous learning without task boundaries; Step 6: Repeat steps 2 to 4 until the training termination condition is met, and obtain the trained production dynamic prediction model. Step 7: Obtain the actual daily production data of the reservoir in real time, input it into the trained production dynamic prediction model, and obtain the daily liquid production of the reservoir.

2. The task-boundary-free continuous learning production dynamic prediction method based on a replay mechanism according to claim 1, characterized in that, In step 1, the data preprocessing process is as follows: Step 1.1: Obtain three years of actual production data from reservoir water injection wells and production wells, and perform minimum-maximum normalization; Step 1.2: Fill the normalized data with the mean to form time series data, and then divide it into training set, validation set and test set according to the proportion; Step 1.3: The starting position of the training set remains unchanged at 0, and the ending position is 70% of the total length of the original sequence. The starting positions of the validation set and test set intervals are corrected by order offset: the starting position of the validation set is offset from the ending position of the training set by a step length equal to the length of the input sequence in the past direction of the time axis, and boundary checks are used to ensure that the starting position is not less than 0, and the ending position is 80% of the total length of the original sequence; the starting position of the test set is offset from the ending position of the validation set by a step length equal to the length of the input sequence in the past direction of the time axis, while ensuring that the starting position is not lower than the ending position of the validation set, and the ending position is 100% of the total length of the original sequence.

3. The task-boundary-free continuous learning production dynamic prediction method based on a replay mechanism according to claim 1, characterized in that, The specific process of step 2 is as follows: Step 2.1: Process the input time series data Perform block embedding to obtain the block-based embedding representation. : ； ； in, For the first One data block; This is a block operation; For linear layer embedding operations; Step 2.2, for Perform block-by-block detrending: ； ； ； in, For the first Data blocks Trend items; It is an average pooling operation; It is a fill operation; , The first The, the A detrended data block; For the first The series after detrending of each variable It is a slice symbol; Step 2.3: Use a short-term attention layer to capture short-term temporal dependencies within the same variable: ； ； in, For the first These are intermediate variables in the output of the short-term attention layer. Representation layer normalization; For the first The original sequence of variables; For short-term attention layer; This represents a multilayer feedforward neural network; Indicates the first The feature sequence of each variable obtained through a short-term attention layer; Step 2.4: Reconstruct the mask based on the segmented sequence: ； in, This is the sequence output after mask reconstruction; It is a mask matrix; It is the inverse mask matrix; The symbol for element-wise product; Step 2.5: Perform block downsampling: ； in, For the first The sequence after block downsampling of each variable; For block downsampling; Step 2.6: Apply long-term attention to each downsampled block sequence to model the cointegration relationship among all variables over the same time interval. ； ； in, For the first The intermediate tensor of the long-term attention layer output at each time step; For the first The original sequence at each time step; For the first The sequence after downsampling in blocks at each time step; For long-term attention layer; It is the first The feature sequence obtained from the long-term attention layer at each time step; Step 2.7: Concatenate the feature sequences obtained from the long and short attention layers to obtain the global temporal features. : ； in, This is a splicing symbol; Step 2.8: The global temporal features are processed through a two-layer cascaded fully connected network, combined with a hybrid sampling strategy using a double-buffered playback memory module, and spliced ​​with the current batch to form a continuous learning training batch, ultimately outputting the predicted liquid production value.

4. The task-boundary-free continuous learning production dynamic prediction method based on a replay mechanism according to claim 1, characterized in that, The specific process of step 4 is as follows: Step 4.1, set the time step The current batch loss is Maintain a length of Loss sliding window: ； in, For time step Loss sliding window; Step 4.2, in The mean and standard deviation of the batch loss were calculated internally: ； ； in, , They are time steps The mean and standard deviation; For batch losses; For time step The amount of data lost in the sliding window; Step 4.3: Pre-set two thresholds: one is the mean threshold. The other is the standard deviation threshold. Only when the mean of the loss within the sliding window is less than And the standard deviation of the loss within the sliding window is less than At this point, it is determined that the model learning has stabilized; in addition, a state flag is introduced. This is used to control whether the system enters the platform's testing phase. , These are the two state values ​​of the state flag; when the state flag is... When the platform testing phase begins, the replay mechanism training in step 3 is initiated; once the platform testing phase is complete, the status flag will be automatically set to [value]. ; Step 4.4: Record the mean and standard deviation of the platform after the model learning stabilizes, denoted as follows: and As a reference baseline for the next round of platform testing, that is , , To update the symbols; then clear the loss sliding window, and simultaneously... Set as End this platform test; settings ;in, It is a two-dimensional nonnegative real number space; Step 4.5: If the mean within the loss sliding window is greater than the sum of the mean and standard deviation of the plateau after the model stabilizes in the previous learning iteration, perform plateau detection again; at this time, Reset to We have entered a new platform testing phase.

5. The task-boundary-free continuous learning production dynamic prediction method based on a replay mechanism according to claim 1, characterized in that, In step 5, the temporal Huber loss, frequency domain Huber loss, and playback loss are fused to construct the total loss function; Time-domain Huber loss The formula is: ； in, For threshold Huber's losses; This is a predicted value; The actual value; Frequency domain Huber loss The formula is: ； in, is the real part of the complex number; It represents the imaginary part of a complex number; The spectrum of the real sequence; To predict the spectrum of the sequence; Playback loss The formula is: ； Final total loss function for: ； in, This is a hyperparameter.

6. The task-boundary-free continuous learning production dynamic prediction method based on replay mechanism according to claim 1, characterized in that, In step 6, the training ends when the preset maximum number of iterations is reached or when the early stopping mechanism is triggered. The condition for triggering the early stopping mechanism is that, during the training iteration, after each round of parameter updates, the total loss function is calculated for both the validation set and the test set. ; If the loss value of the validation set is higher than the loss value of the test set in three consecutive iterations, the early stopping mechanism is triggered immediately to terminate subsequent iterations, and the optimal model parameters before the early stopping mechanism was triggered are retained.

Images