Reservoir parameter prediction method based on improved Transform

Abstract

The invention discloses a reservoir parameter prediction method based on an improved Transform. The reservoir parameter prediction method comprises the following steps: collecting logging data and corresponding rock core lithologic data; preprocessing the logging data through a rock core homing method; selecting an input logging characteristic curve through Pearson linear correlation analysis, and carrying out standardization processing on the input logging characteristic curve; the method comprises the following steps of: improving a Transform by using a TransBlock multi-head attention mechanism and a Real-LN residual structure, and performing feature extraction on a training sample by using a TransBlock algorithm obtained after improvement to form a TransBlock feature model; and obtaining a reservoir parameter prediction result from the to-be-detected data through the TransBlock feature model. According to the method, the problems of less data and insufficient generalization ability in reservoir parameter prediction are solved by improving Transformer, the prediction performance and generalization adaptation ability of the model are improved through limited logging data and hidden correlation between the logging data, and the accuracy of reservoir parameter prediction is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of oil and gas exploration, and particularly relates to a reservoir parameter prediction method based on an improved Transformer. Background Art

[0002] The global energy industry is shifting the exploration focus to unconventional reservoirs such as shale oil and gas, tight sandstone gas, and coalbed methane. However, the geological characteristics of unconventional reservoirs usually show significant features such as strong heterogeneity of reservoir space structure and poor connectivity of fluid migration channels. The development process faces multiple technical challenges, and the breakthrough of reservoir parameter prediction technology has become the focus of industry attention.

[0003] Traditional reservoir parameter prediction methods mainly rely on rock physics theoretical models and empirical formulas with geological statistical significance. The use of these methods all has harsh application assumption conditions. Sometimes they can meet the prediction requirements, but they often perform poorly when dealing with complex geological conditions.

[0004] Currently, in the practice of oil and gas field development, the determination methods of reservoir parameters (such as porosity, permeability, etc.) are mainly divided into direct determination methods and indirect interpretation methods. The former obtains rock physics parameters through wellbore core sampling, full-diameter core experiments, etc. Although it has high measurement accuracy, its application has significant limitations. This method requires a large amount of manpower and material resources. And in actual production, due to environmental or other restrictions, it is difficult to obtain sufficient core data to meet the requirements.

[0005] Compared with traditional experimental methods, the latter shows unique advantages. This technical system constructs a non-linear mapping model between logging parameters and reservoir physical properties by integrating multi-type logging response characteristics (such as natural gamma, acoustic travel time, resistivity, etc.) and geological prior information. In recent years, various deep learning models have obtained good feedback in reservoir parameter evaluation. The Transformer algorithm is used in reservoir parameter prediction. By automatically extracting features from a large amount of multi-source data such as logging data and seismic data, and establishing a complex non-linear mapping relationship, the prediction accuracy is significantly improved.

[0006] Transformer is a sequence model based on the attention mechanism, initially proposed by a research team at Google and applied to the machine translation task. Different from traditional recurrent neural networks (RNNs) and convolutional neural networks (CNNs), Transformer only uses the self-attention mechanism to process the input sequence and output sequence, so it can perform parallel computing, greatly improving the computing efficiency.

[0007] However, deep learning algorithms such as Transformer still face some technical challenges in actual use: (1) There are few well logging data samples, resulting in insufficient model training data; (2) The machine learning model is not optimized for the time series correlation of well logging data, and the deep network is prone to the problem of gradient disappearance. Summary of the Invention

[0008] To solve the above problems, the present invention proposes a reservoir parameter prediction method based on an improved Transformer, which solves the problems of less data and insufficient generalization ability in reservoir parameter prediction by improving Transformer, and improves the prediction performance and adaptation generalization ability of the model through limited well logging data and the hidden correlation between well logging data, significantly improving the accuracy of reservoir parameter prediction.

[0009] To achieve the above object, the technical solution adopted by the present invention is: A reservoir parameter prediction method based on an improved Transformer, comprising the steps of: Collect the well logging data and the corresponding core lithology data; preprocess the well logging data through the core positioning method; Select the input well logging characteristic curves through Pearson linear correlation analysis, and perform standardization processing on the input well logging characteristic curves; Improve the Transformer using the TransBlock multi-head attention mechanism and the Real-LN residual structure, and use the improved TransBlock algorithm to extract features from the training samples to form a TransBlock feature model; Obtain the reservoir parameter prediction result by passing the data to be detected through the TransBlock feature model.

[0010] Further, the well logging data includes acoustic travel time AC, compensated neutron CNL, natural gamma GR, density DEN, caliper CAL, and deep lateral resistivity RT.

[0011] Further, Pearson linear correlation analysis is used to quantify the correlation between well logging data and reservoir parameters, and select the features related to reservoir parameters as the input.

[0012] Further, the calculation formula of the Pearson linear correlation coefficient is: ; where x i and y i are two data points for correlation analysis, and are the average values of the corresponding variables of these two data points respectively.

[0013] Furthermore, the established TransBlock model is used to extract features from the training sets in different regions, capture the long-range dependencies among well logging data through the multi-head attention mechanism, analyze the mapping relationship between reservoir parameters and input curves, and form a TransBlock feature model.

[0014] Furthermore, constructing the TransBlock multi-head attention mechanism includes the following steps: Embedding position encoding: Convert the input data into a vector form suitable for model processing and add position information so that the model can capture the sequence relationship in the data; the input well logging curve data is mapped to a high-dimensional space through a linear transformation to obtain the embedding vector E and the position encoding P, and the final input representation X = E + P is obtained; Layer normalization: Normalize all features of each sample; Establishing the multi-head attention mechanism: Enable the model to focus on the correlation between well logging data and at the same time enable the model to extract more comprehensive feature data; input the normalized input X_norm into the multi-head attention mechanism; the multi-head attention mechanism consists of multiple parallel attention heads, and each head independently calculates the attention weights; perform a linear transformation on X_norm to obtain the query Q, key K, and value V matrices, calculate the attention weights of each head, concatenate the outputs of all heads, and obtain the final attention output A_norm through a linear transformation; Layer normalization: Normalize the output of the multi-head attention mechanism; Feed-forward neural network: Further extract features through a feed-forward neural network, input the normalized output A_norm into a two-layer fully connected feed-forward neural network, and the output is F; Residual connection: Combine the output F of the feed-forward neural network with the input X through a residual connection to construct the final output O. The specific calculation formula is O = X + F.

[0015] Furthermore, for the Real-LN residual structure, based on the Post-LN model, when calculating the multi-head attention in each Transformer layer, add the Attention Scores matrix of the previous layer; a residual connection channel is added between adjacent attention modules.

[0016] Furthermore, the Relu activation function is adopted in the TransBlock feature model.

[0017] Furthermore, the Adam optimizer is adopted in the TransBlock feature model.

[0018] Beneficial effects of adopting this technical solution: By introducing the TransBlock multi-head attention mechanism and the Real-LN residual structure, the present invention enables the Transformer to solve the problems of insufficient data and poor generalization ability in reservoir parameter prediction, enabling the Transformer to enhance the prediction performance and adaptation generalization ability of the model through the hidden associations between limited well logging data, and significantly improving the accuracy of reservoir parameter prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic flow chart of a reservoir parameter prediction method based on an improved Transformer according to the present invention; Figure 2 is a preferred input well logging curve for the relevant heat map in an embodiment of the present invention; Figure 3 is a schematic diagram of the network structure of the TransBlock algorithm in an embodiment of the present invention; Figure 4 is a schematic diagram of the residual network structure in an embodiment of the present invention; Figure 5 is a preferred comparison diagram of activation functions in an embodiment of the present invention; Figure 6 is a preferred comparison diagram of optimizer functions in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings.

[0021] In this embodiment, referring to Figure 1 as shown, the present invention proposes a reservoir parameter prediction method based on an improved Transformer, including the steps of: Preprocessing the well logging data by the core positioning method for the collected well logging data and the corresponding core lithology data; Selecting the input well logging characteristic curves through Pearson linear correlation analysis and performing standardization processing on the input well logging characteristic curves so that the model can capture the sequence relationships in the data; Improving the Transformer by using the TransBlock multi-head attention mechanism and the Real-LN residual structure, and using the improved TransBlock algorithm to extract features from the training samples to form a TransBlock feature model; Obtaining the reservoir parameter prediction result by passing the data to be detected through the TransBlock feature model.

[0022] Among them, the logging data includes acoustic travel time AC, compensated neutron CNL, natural gamma GR, density DEN, caliper CAL, and deep lateral resistivity RT.

[0023] As an optimized solution of the above embodiment, Pearson linear correlation analysis is used to quantify the correlation between logging data and reservoir parameters, and the features related to reservoir parameters are selected as the input.

[0024] Specifically, the calculation formula of the Pearson linear correlation coefficient is: ; where x i and y i are two data points for correlation analysis, and are the average values of the corresponding variables of these two data points respectively.

[0025] Figure 2 It shows that when predicting the reservoir water saturation SW parameter, through the Pearson linear correlation coefficient method analysis, the finally extracted features are: AC, CNL, DEN, RT, RXO. Among them, AC is the acoustic travel time curve in logging, CNL is the compensated neutron curve, DEN is the density curve, RT is the deep lateral resistivity curve, RXO is the shallow lateral resistivity curve, GR is the natural gamma curve, PERM is the permeability curve, and POR is the porosity curve.

[0026] As an optimized solution of the above embodiment, the established TransBlock model is used to extract features from the training sets in different regions, capture the long-range dependence relationship between logging data through the multi-head attention mechanism, analyze the mapping relationship between reservoir parameters and input curves, and form a TransBlock feature model.

[0027] Figure 3 It is a schematic diagram of the TransBlock algorithm network structure. Input Embedding represents the process of converting the input words or characters into a vector representation that can be processed by a computer; TransBlock represents the introduction of the TransBlock multi-head attention mechanism; Linear Layer represents a basic layer in the Transformer, also known as the fully connected layer, and Predict represents the prediction output.

[0028] Specifically, constructing the TransBlock multi-head attention mechanism includes the steps: Embedding Position Encoding: Convert the input data into a vector form suitable for model processing and add position information so that the model can capture the sequential relationship in the data; the input well logging curve data is mapped to a high-dimensional space through a linear transformation to obtain the embedding vector E and the position encoding P, and the final input representation X = E + P; Layer Normalization: Normalize all features of each sample to make the model training more stable; Assume that the input X is layer-normalized (Layer Norm) to obtain the normalized output X_norm. The formula is: ; where μ and σ2 are the mean and variance of the input x respectively. γ is a learnable scaling and offset parameter. ε is a constant used for numerical stability. β is a translation parameter.

[0029] Establish a multi-head attention mechanism: Enable the model to focus on the correlation between well logging data and at the same time enable the model to extract more comprehensive feature data; input the normalized input X_norm into the multi-head attention mechanism; the multi-head attention mechanism consists of multiple parallel attention heads, and each head independently calculates the attention weights; perform a linear transformation on X_norm to obtain the query Q, key K, and value V matrices, calculate the attention weights of each head, concatenate the outputs of all heads, and obtain the final attention output A_norm through a linear transformation.

[0030] The formula is: ; where QKT calculates the similarity between the query and the key. is a scaling factor used to prevent gradient disappearance. The softmax function converts the similarity into a probability distribution.

[0031] Layer Normalization: Normalize the output of the multi-head attention mechanism; Feed-Forward Neural Network: Further extract features through a feed-forward neural network, input the normalized output A_norm into a two-layer fully-connected feed-forward neural network, and the output is F.

[0032] The formula is: ; where W 1 and W 2 are weight matrices, b 1 and b 2 are bias terms, and d_ff is the hidden layer dimension of the feed-forward neural network.

[0033] Residual connection: Through the residual connection, the vanishing gradient can be effectively prevented, and the training process of the model can be accelerated. Its core mechanism lies in combining the output F of the feedforward neural network with the input X to construct the final output O. The specific calculation formula is O = X + F.

[0034] This design cleverly preserves the key information of the original input while integrating the features extracted after being processed by the attention mechanism and the feedforward neural network, providing a richer and more comprehensive information representation for the model.

[0035] Through the above steps, the TransBlock structure can effectively process logging data, capture the sequential relationships and features in the data, and thus better complete the reservoir parameter prediction task. Each step ensures the stability and training efficiency of the model through layer normalization and residual connection.

[0036] Specifically, in the Real-LN residual structure, based on the Post-LN model, when calculating the multi-head attention in each Transformer layer, the Attention Scores matrix of the previous layer is added; a residual connection channel is added between adjacent attention modules.

[0037] An example diagram of the Real-LN model is as Figure 4 shown. Add & Norm represents the layer normalization module; FeedForward represents the feedforward neural network; Multi-Head Attention represents the multi-head attention mechanism; Add represents the added residual connection.

[0038] When calculating the attention scores each time, the attention scores of the previous layer are added. The formula is as follows: ; ; where, A n-1 has an initial value of 0, providing a direct path so that the last attention layer can also receive the attention values of the first layer, thus eliminating the risk of vanishing gradient.

[0039] The Real-LN as a whole maintains the structure of the Post-LN, so it has the excellent performance of the Post-LN and also integrates the advantages of the residual.

[0040] Preferably, the Relu activation function is adopted in the TransBlock feature model. Through the commonly available comparison methods in the computer field, the activation function and optimization function of the TransBlock algorithm are optimized. From the results Figure 5It can be seen that after in-depth analysis and comparison of the three activation functions of Tanh, Sigmoid, and Relu, it can be found that the Relu activation function has significant advantages in terms of computational efficiency and network performance. Compared with the Tanh and Sigmoid functions, the Relu activation function can greatly save computational resources and time costs. Its simple mathematical form and efficient computational characteristics enable the Relu function to perform excellently when dealing with large-scale data and complex models, and can significantly improve the training speed and efficiency of the model.

[0041] Preferably, the Adam optimizer is adopted in the TransBlock feature model.

[0042] Based on the actual logging dataset, for the reservoir parameter prediction models driven by the three optimizers of Adam, SGD, and AdaGrad, during the training process of the three optimizers, the convergence trend of their loss functions is as Figure 6 shown. Among them, the Adam optimizer shows the fastest convergence rate throughout the training process, and the final loss function value reached is the smallest, which means that under the same training conditions, the Adam optimizer can make the model approach the optimal solution faster and significantly improve the training efficiency and performance of the model.

[0043] The TransBlock feature model is verified through the test dataset. In order to more clearly determine the performance of the TransBlock feature model, by introducing the popular RF, XGB, LSTM, and Transformer algorithms for comparative analysis, the prediction effect is evaluated using four indicators: the coefficient of determination (R2), the root mean square error (RMSE), and the mean absolute error (MAE); If the prediction effect of the TransBlock feature model is better than that of the RF, XGB, LSTM, and Transformer algorithms, it can be considered that the improved Transformer algorithm has improved the accuracy of reservoir parameter prediction, effectively solved the problem of insufficient generalization ability of the model due to the small amount of reservoir parameter data, and the improved Transformer algorithm (TransBlock algorithm) can be popularized and applied; Table 1 is a statistical comparison table of algorithm recognition for Well A1 in a certain area of the Sichuan Basin. It can be seen from the table that in the prediction tasks of porosity (POR), permeability (PERM), and water saturation (SW), the improved Transformer algorithm (TransBlock algorithm) performs better than the prediction effects of RF, XGB, LSTM, and the Transformer algorithm. The improved Transformer algorithm (TransBlock algorithm) can efficiently and sensitively capture the sequence relationship in the data through positional encoding, and its prediction results in the vertical direction of logging are better than those of other comparison algorithms. This further proves that the improved Transformer algorithm improves the accuracy of reservoir parameter prediction and effectively solves the problem of insufficient model generalization ability caused by the lack of reservoir parameter data. Therefore, the improved Transformer algorithm (TransBlock algorithm) can be popularized and applied.

[0044] Table 1 Statistical Comparison Table of Algorithm Recognition

[0045] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification only illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A reservoir parameter prediction method based on improved Transformer, characterized in that: Includes steps: The collected logging data and the corresponding core lithology data are preprocessed by core tracing method; The input logging characteristic curve is selected through Pearson linear correlation analysis, and the input logging characteristic curve is standardized; The Transformer is improved by using the TransBlock multi-head attention mechanism and the Real-LN residual structure. The improved TransBlock algorithm is used to extract features from training samples to form a TransBlock feature model. The data to be tested are passed through the TransBlock characteristic model to obtain the reservoir parameter prediction results.

2. The reservoir parameter prediction method based on improved Transformer according to claim 1, characterized in that: The logging data include acoustic wave time difference AC, compensated neutron CNL, natural gamma GR, density DEN, wellbore CAL and deep lateral resistivity RT.

3. The reservoir parameter prediction method based on improved Transformer according to claim 1, characterized in that: Pearson linear correlation analysis was used to quantify the correlation between logging data and reservoir parameters, and features related to reservoir parameters were selected as input.

4. The reservoir parameter prediction method based on improved Transformer according to claim 3, characterized in that: The calculation formula of the Pearson linear correlation coefficient is: ； Among them, x i and i For the two data points for correlation analysis, and are the average values ​​of the variables corresponding to these two data points.

5. The reservoir parameter prediction method based on improved Transformer according to claim 1, characterized in that: The established TransBlock model is used to extract features from training sets in different regions. The long-range dependencies between logging data are captured through a multi-head attention mechanism. The mapping relationship between reservoir parameters and input curves is analyzed to form a TransBlock feature model.

6. A reservoir parameter prediction method based on improved Transformer according to claim 5, characterized in that: Building the TransBlock multi-head attention mechanism includes the following steps: Embedded position encoding: Convert the input data into a vector form suitable for model processing and add position information so that the model can capture the sequence relationship in the data; the input logging curve data is mapped to a high-dimensional space through linear transformation to obtain the embedded vector E and position encoding P, and the final input representation X = E + P is obtained; Layer normalization: normalize all features of each sample; Establish a multi-head attention mechanism: make the model pay attention to the correlation between logging data, and at the same time enable the model to extract more comprehensive feature data; input the normalized input X_norm into the multi-head attention mechanism; the multi-head attention mechanism consists of multiple parallel attention heads, and each head independently calculates the attention weight; perform a linear transformation on X_norm to obtain the query Q, key K and value V matrices, and calculate the attention weight of each head, concatenate the outputs of all heads, and obtain the final attention output A_norm through a linear transformation; Layer normalization: normalize the output of the multi-head attention mechanism; Feedforward neural network: The features are further extracted through the feedforward neural network, and the normalized output A_norm is input into a two-layer fully connected feedforward neural network, and the output is F; Residual connection: Through the residual connection, the output F of the feedforward neural network is combined with the input X to construct the final output O. The specific calculation formula is O=X+F.

7. The reservoir parameter prediction method based on improved Transformer according to claim 5, characterized in that: The Real-LN residual structure, based on the Post-LN model, adds the Attention Scores matrix of the previous layer when calculating multi-head attention in each Transformer layer; and adds a residual connection channel between adjacent attention modules.

8. The reservoir parameter prediction method based on improved Transformer according to claim 5, characterized in that: The Relu activation function is used in the TransBlock feature model.

9. The reservoir parameter prediction method based on improved Transformer according to claim 5, characterized in that: The Adam optimizer is used in the TransBlock feature model.

Images