Neural differential self-attention Transform-based oil well refracturing yield prediction method

Abstract

The invention provides an oil well refracturing yield prediction method based on a neural differential self-attention Transform. The method comprises the following steps: (S1) data collection and preparation; (S2) symbol self-attention is defined; (S3) defining neural differential self-attention; (S4) performing seasonal trend decomposition based on neural differential self-attention; and (S5) predicting the re-fracturing yield of the oil well based on the neural differential self-attention Transform. The neural difference self-attention mechanism is adopted to optimize the Transform architecture, so that the time sequence stability is mainly realized through difference to adapt to a unique seasonal or periodic mode contained in the oil well refracturing yield data, and finally, the calculation cost is reduced while the prediction precision is guaranteed. The preprocessing routine of statistical difference is broken, time variation can be captured step by step flexibly, and various time modes except the trend can be effectively captured.

Description

Technical Field

[0001] The present invention relates to the technical field of reservoir stimulation and transformation, and in particular to a method for predicting oil well refracturing production based on a neural differential self-attention Transformer. Background Art

[0002] Time series analysis refers to the research problem of collecting observations at fixed time intervals and there is correlation between consecutive observations. The production prediction of refracturing oil wells belongs to time series analysis.

[0003] Despite its success, the Transformer's potential in time series data analysis has yet to be fully realized. Current research shows that compared to traditional deep learning baseline models, the Transformer and its variants have limited performance improvements when processing time series data. This phenomenon is mainly attributed to the limitations of the existing self-attention mechanism: whether in standard form or sparse version, the self-attention mechanism essentially uses a weighted global average or k-moving average method, which is basically equivalent to a smoothed estimate of the trend pattern. This architecture has inherent defects in representing subtle, complex and diverse patterns in time series (such as outliers, seasonality and cyclical patterns): the patterns are easily diluted in the averaging process, and then the output results change little or not at all.

[0004] To address the above limitations, the present invention uses a neural differential self-attention mechanism to optimize the Transformer architecture, making time series smooth mainly through differentiation, so as to adapt to the unique seasonal or cyclical patterns contained in the oil well repeated fracturing production data, and ultimately reduce the computational cost while ensuring the prediction accuracy. Summary of the Invention

[0005] To address these issues, the present invention provides a method for predicting oil well refracturing production based on a neural differential self-attention Transformer. This method breaks away from the conventional statistical differential preprocessing, enabling flexible and gradual capture of temporal variation and effectively processing various temporal patterns beyond trends.

[0006] The present invention adopts the following technical solutions:

[0007] A method for predicting oil well refracturing production based on neural differential self-attention Transformer, including the following steps:

[0008] Step 1: Data collection and preparation;

[0009] Step 2: Define signed self-attention;

[0010] Step 3: Define neural differential self-attention;

[0011] Step 4: Seasonal trend decomposition based on neural differential self-attention;

[0012] Step 5: Oil well re-fracture production prediction based on neural differential self-attention Transformer.

[0013] Furthermore, the step 1 specifically includes the following sub-steps:

[0014] Step 101: Collecting oil well refracturing production data;

[0015] Step 102: Arrange the production data into a format that can be processed by the program.

[0016] Furthermore, the step 2 specifically includes the following sub-steps:

[0017] Step 201: Calculate the expected difference between local range random variables:

[0018]

[0019] In formula (1), L = T[a, b] represents the local range of time series data; T s and T t represents two random variables within the range of L; ρ s ,ρ t ∈[0,1] b-a+1 are the probability distributions of the source and target variables, respectively.

[0020] Step 202: Define the signed self-attention matrix:

[0021]

[0022] In formula (2), ⊙ represents the Hadamard product; Indicates a newly expanded dimension; are learnable weights. The expanded dimensions will maintain the feature dimensions after subtraction, allowing for more refined fade factors to be learned for each query-key term in the attention matrix. P is a learnable relative position embedding used to represent lags. is the standard scaling constant. and is the linear projection of query and key, the same as the classic self-attention mechanism.

[0023] Unlike traditional self-attention, which only supports learning additive attention, the signed attention matrix in Equation (2) introduces a dual affinity mechanism. The first affinity (tanh term) controls addition or subtraction. The second affinity (sigmoid term) acts as a gating operation to ignore unnecessary elements. Signed attention can flexibly adjust the operation between addition, subtraction, or ignoring addition and subtraction based on the content and lag information.

[0024] Furthermore, the step 3 specifically includes the following sub-steps:

[0025] Step 301: Normalize the signed self-attention matrix;

[0026] Since Equation (1) is naturally expressed as ∑(ρ s -ρ t )=0 as the center, (ρ s and ρ t Represent the probability distribution of source and target variables respectively), we prevent the attention from being dominated by positive or negative values by normalizing the positive and negative parts. The final normalized signed self-attention matrix for:

[0027]

[0028] In formula (3), ||·||1 is the L-1 norm, clamp(·) τ is the clamp operation matrix, which is defined as:

[0029]

[0030] In formula (4), ξ (0≤ξ≤1) is a sufficiently small constant used to exclude sufficiently small attention values (i.e., ignore them) in the calculation.

[0031] Step 302: Define the neural differential self-attention matrix:

[0032] Given value matrix The neural differential self-attention DiffAttention can be defined as:

[0033]

[0034] Furthermore, the step 4 specifically includes the following sub-steps:

[0035] Step 401: Detrending using a neural differential self-attention module;

[0036] Let the original time series data be X, and trend(X) be used to represent the trend component extracted from the time series. The process of removing the trend term using the neural differential self-attention module is as follows:

[0037] X detrend =X-trend(X) (6)

[0038] Step 402: Highlight informative anomalies and save seasonality;

[0039] Let X tRepresents the value of the time series at time t, and let θ represent the threshold for abnormal judgment. The abnormal part in the original data can be highlighted as follows:

[0040] Anomalies = {X t ||X t -DiffAttention(X t )|>θ} (7)

[0041] The seasonal component in the original data is denoted as season(X).

[0042] Furthermore, the step 5 specifically includes the following sub-steps:

[0043] Step 501: Extract local features using sliding window technology;

[0044] Based on the information-abnormal segments in step 402 and the global seasonal characteristics, the entire time series data is split into multiple overlapping window segments using a sliding window technique.

[0045] X window =[X 1：w , X 2：w+1 ,...,X n-w+1：n ] (8)

[0046] Window segments can capture local features of the data and help the model focus on details and trends within each time window.

[0047] Step 502: Transformer encoder input;

[0048] Multiple clips processed by the sliding window are input into the Transformer encoder to extract the feature representation of each clip:

[0049] H=[Encoder(X 1：w ), Encoder(X 2：w+1 ),...,Encoder(X n-w+1：1 )] (9) In formula (9), Encoder represents the encoder part of the Transformer architecture; H represents the set of encoded features of all input time segments.

[0050] Step 503: Integrate the prediction results of different time segments;

[0051] The features output by the encoder are processed and predicted through one or more fully connected layers, and then the prediction results of all segments are merged to obtain the final prediction:

[0052] Final Prediction=aggregate([FC(h1), FC(h2),..., FC(h m )]) (10) In formula (10), FC(h i ) represents the fully connected layer, which is used to extract the encoder features h i Generate predicted values; h i Represents the encoder feature vector corresponding to each window segment; aggregate is a merging function that uses an average aggregation method; Final Prediction represents the final yield prediction result, which integrates the prediction information of all time segments.

[0053] Step 504: Output the integrated prediction result.

[0054] The beneficial effects of the present invention are:

[0055] By optimizing the Transformer architecture with a neural differential self-attention mechanism, time series smoothing is achieved primarily through differencing, adapting to the unique seasonal and cyclical patterns inherent in refracturing oil well production data. This approach ultimately reduces computational costs while ensuring forecast accuracy. Breaking the conventional preprocessing of statistical differencing, this approach can gradually and flexibly capture temporal variation, effectively capturing various temporal patterns beyond trends. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present invention, but are not intended to limit the present invention.

[0057] Figure 1 This is the technical roadmap of the present invention;

[0058] Figure 2 The detrending effect of neural differential self-attention on oil well refracturing production data;

[0059] Figure 3 The effect of neural differential self-attention on highlighting the abnormal information content of oil well refracturing production data;

[0060] Figure 4 The effect of neural differential self-attention on preserving seasonality of oil well refracturing production data;

[0061] Figure 5 The prediction effect of oil well refracturing production based on neural differential self-attention. DETAILED DESCRIPTION

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0063] like Figures 1 to 5 As shown, the present invention provides a technical solution: a method for predicting oil well re-fracture production based on neural differential self-attention Transformer, comprising the following steps:

[0064] Step 1: Data collection and preparation, including the following steps:

[0065] (1) The data collected from the production statistics system of the oil production plant include wells 8315 and 8330 in the W block of Xinjiang Oilfield (pseudonym). The production data spans from 1981 to 2023, with monthly time steps (some months are closed for maintenance and no data is available). The collected basic data set is shown in Table 1:

[0066]

[0067] Table 1 Basic dataset

[0068] (2) Arrange the production data into a format that can be processed by the program, and normalize the basic data to between [-1, 1] as shown in Table 2:

[0069]

[0070]

[0071] Table 2 Processed dataset

[0072] Step 2: Define signed self-attention, which includes the following steps:

[0073] (1) Calculate the expected difference between local range random variables;

[0074] (2) Define the signed self-attention matrix.

[0075] Step 3: Define neural differential self-attention, which includes the following steps:

[0076] (1) Normalized signed self-attention matrix;

[0077] (2) Define the neural differential self-attention matrix.

[0078] Step 4: Seasonal trend decomposition based on neural differential self-attention, which includes the following steps:

[0079] (1) Use the neural differential self-attention module to remove the trend in the original data, reveal the hidden information and periodic changes in the data, and make the data more stable, such as Figure 2 As shown;

[0080] (2) Highlight the anomalies with information content and preserve seasonality. Figure 3 As shown, retaining seasonality as Figure 4 shown.

[0081] Step 5: Prediction of oil well refracture production based on neural differential self-attention Transformer, specifically including the following steps:

[0082] (1) Using sliding window technology to extract local features;

[0083] (2) Transformer encoder input;

[0084] (3) Integrate the prediction results of different time segments;

[0085] (4) Output the integrated prediction results. The prediction results of repeated fracturing production of wells 8315 and 8330 are as follows: Figure 5 shown.

[0086] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for predicting oil well re-fracture production based on neural differential self-attention Transformer, characterized by: The following steps are involved: Step 1: Data collection and preparation; Step 2: Define signed self-attention; Step 3: Define neural differential self-attention; Step 4: Seasonal trend decomposition based on neural differential self-attention; Step 5: Oil well re-fracture production prediction based on neural differential self-attention Transformer.

2. The method for predicting oil well re-fracture production based on neural differential self-attention Transformer according to claim 1 is characterized in that: The step 1 specifically includes the following sub-steps: Step 101: Collecting oil well refracturing production data; Step 102: Arrange the production data into a format that can be processed by the program.

3. The method for predicting oil well refracturing production based on neural differential self-attention Transformer according to claim 1 is characterized in that: The step 2 specifically includes the following sub-steps: Step 201: Calculate the expected difference between local range random variables; Step 202: Define a signed self-attention matrix.

4. The method for predicting oil well refracturing production based on neural differential self-attention Transformer according to claim 1 is characterized in that: The step 3 specifically includes the following sub-steps: Step 301: Normalize the signed self-attention matrix; Step 302: Define the neural differential self-attention matrix.

5. The method for predicting oil well refracturing production based on neural differential self-attention Transformer according to claim 1 is characterized in that: The step 4 specifically includes the following sub-steps: Step 401: Detrending using a neural differential self-attention module; Step 402: Highlight informative anomalies and save seasonality.

6. The method for predicting oil well refracturing production based on neural differential self-attention Transformer according to claim 1, characterized in that: The step 5 specifically includes the following sub-steps: Step 501: Extract local features using sliding window technology; Step 502: Transformer encoder input; Step 503: Integrate the prediction results of different time segments; Step 504: Output the integrated prediction result.

Images