Method for realizing well logging lithology identification based on convolutional recurrent deep learning network of self-attention mechanism

Abstract

The application provides a method for realizing well logging lithology identification based on a convolutional recurrent deep learning network of a self-attention mechanism, mainly comprising: (1) data set construction, that is, using drilling core lithology description results and well logging data of a coring well section to group samples; (2) model construction, that is, determining a sample input mode and a basic structure of the model; (3) model training, that is, removing null values and abnormal values, normalizing sample characteristic values, encoding label values, and dividing the samples into a training set, a verification set and a test set; (4) model evaluation, that is, realizing model construction, parameter optimization, model training and verification based on PyTorch. The application has reasonable conception, and realizes well logging lithology identification by constructing a core-well logging mapping relationship.

Description

Technical Field

[0001] This invention relates to the field of oil and gas exploration and development technology, specifically to a method for well logging lithology identification based on a convolutional recurrent deep learning network with a self-attention mechanism. Background Technology

[0002] Core sampling provides a wealth of crucial information related to sedimentation, geochemistry, and rock physics, making it essential for oil and gas field exploration and development. Clearly defining the vertical and lateral lithofacies distribution is a prerequisite for extending this information from the core scale to the oil and gas block or basin scale. While core sampling offers high precision, cost constraints result in relatively scarce data. Cuttings logging has lower resolution and higher uncertainty, typically used as reference information. Geophysical logging, although an indirect source of lithofacies data, offers significant scale advantages. In conventional workflows, lithofacies and lithofacies assemblages are summarized through core description, and then the logging response of the cored well section is combined to map lithofacies to logging facies. Scale extension is gradually achieved by integrating data from single wells, interconnected wells, and even seismic data. This demonstrates that fully utilizing core-logging data and establishing a core-logging data mapping relationship is an indispensable and crucial step.

[0003] Methods for constructing core-logging mappings are all based on rock physical properties to distinguish lithofacies types. These methods can generally be subdivided into conventional approaches (qualitative descriptions, cross-plots, and empirical formulas, etc.), traditional machine learning (support vector machines, K-nearest neighbor clustering, decision tree-based methods, and backpropagation neural networks, etc.), and deep learning methods. Conventional methods rely heavily on subjective experience in data and parameter selection, resulting in low efficiency and high uncertainty. Traditional machine learning methods often overly rely on feature engineering to simplify data complexity (or dimensionality), making them suitable for tasks with relatively low complexity but prone to overfitting and having relatively weak generalization ability. The controlling factors of rock physical properties (such as minerals, porosity, fluids, and borehole environment) require complex, high-dimensional feature combinations. Deep learning's network structure excels at learning high-dimensional features from input data, thus reducing excessive reliance on feature engineering and geological experts. With the significant advancements in algorithms and computing power, and the availability of massive amounts of data, implementing deep learning to construct core-logging mappings has greater potential.

[0004] In conclusion, it is necessary to further improve and innovate existing technologies. Summary of the Invention

[0005] To address the technical problems existing in the background art, this invention proposes a method for well logging lithology identification based on a convolutional recurrent deep learning network with a self-attention mechanism. Its concept is reasonable and can achieve high accuracy, especially in the identification of fine-grained thin-layer sand bodies, which far exceeds the ability of machine learning models. This model has strong practical application value and can avoid the high uncertainty and low efficiency caused by subjective selection of data and parameters.

[0006] To address the aforementioned technical problems, this invention provides a method for well logging lithology identification based on a convolutional recurrent deep learning network with a self-attention mechanism, which mainly includes the following steps:

[0007] (1) Dataset Construction

[0008] A sample was constructed using the lithological description results from drilling core samples and the logging data from the cored well sections;

[0009] (2) Model building

[0010] The sample input method and basic model structure are determined as follows: the input of sample feature vectors and dimensionality enhancement are achieved through one-dimensional convolution, and the number of repetitions of one-dimensional convolution is determined according to the number of lithology categories; then the results of one-dimensional convolution are combined and input into a long short-term memory recurrent neural network, and a self-attention mechanism is added to achieve weighted output to obtain a high-dimensional feature vector, and then dimensionality reduction is achieved through a fully connected layer;

[0011] (3) Model Training

[0012] Remove null and outlier values, normalize sample feature values, encode label values, and divide samples into training, validation, and test sets;

[0013] (4) Model Evaluation

[0014] This implementation uses PyTorch to build models, optimize parameters, train models, and validate them.

[0015] The method for well logging lithology identification using a convolutional recurrent deep learning network based on self-attention mechanism, wherein the well logging data in step (1) are sample feature values ​​and the lithology description results are sample labels.

[0016] The method for well logging lithology identification based on the self-attention mechanism convolutional recurrent deep learning network is described in step (2). The basic structure of the model is mainly classified into four layers: convolutional network layer, long short-term memory recurrent neural network layer, attention mechanism layer and fully connected network layer. The sample input method is as follows: the 6-dimensional feature vector is copied into 7 copies of the lithology type, and 7 high-dimensional feature vectors are generated by independent convolutional network layers.

[0017] The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism includes: the convolutional network layer comprising 7 independent structures, which output 7 high-dimensional feature vectors, which are then concatenated and dimensionality transformed before being input into the long short-term memory recurrent neural network layer.

[0018] The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism comprises: a long short-term memory recurrent neural network layer consisting of 7 sequentially arranged structural units with shared weights, each structural unit outputting a high-dimensional vector, for a total of 7 high-dimensional vectors; the output of the last hidden layer, having traversed the 7 structural units and possessing common features, serves as the search vector for the attention mechanism layer; the 7 high-dimensional vectors output from the long short-term memory recurrent neural network layer serve as key-value pairs for the attention mechanism layer; after weighting by the attention mechanism layer, a high-dimensional vector with individual features is output and enters the fully connected network layer.

[0019] The method for identifying well logging lithology using a convolutional recurrent deep learning network based on a self-attention mechanism includes: the fully connected network layer comprising an input layer, two hidden layers, and an output layer, reducing the high-dimensional vector of the attention mechanism layer to a 7-dimensional vector corresponding to the lithofacies type.

[0020] The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism, wherein each of the independent structures comprises five basic units in sequence, namely: a one-dimensional convolutional unit, a one-dimensional convolutional unit, a max pooling layer, a one-dimensional convolutional unit, and a max pooling layer.

[0021] The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism is described above. In this method, the activation functions of the convolutional network layers and the fully connected network layers are selected as hyperbolic tangent functions. The model learning strategy is set as follows: learning rate 2e-5, decay rate 0.1. When the validation loss decreases by less than 0.008 over 15 consecutive cycles, the learning rate will be decayed.

[0022] The method for identifying well logging lithology using a convolutional recurrent deep learning network based on a self-attention mechanism, wherein the ratio of the training set to the validation set in step (3) is 8:2.

[0023] The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism, wherein step (3) specifically comprises the following steps:

[0024] (3.1) The standard deviation standardization method in the scikit-learn module is used to convert the input feature data into a range with a mean of 0 and a unit variance. The label encoding method is used to encode the 7 lithofacies types using integers in the range [0,6].

[0025] (3.2) Before being loaded into the hybrid model, the dataset was divided into batches of 20 samples. The hybrid model was optimized by gradient descent using the cross-entropy loss method and model optimization algorithm. At the same time, weighted cross-entropy loss was considered to handle the imbalanced dataset. The weight values ​​were obtained by the proportion of each lithofacies type.

[0026] The method for well logging lithology identification based on the self-attention mechanism convolutional recurrent deep learning network includes the following steps: In step (4), the evaluation indexes for model evaluation are selected as precision, recall, balanced F-score and accuracy to evaluate the model, and the results are calculated using the results of five-fold cross-validation.

[0027] By adopting the above technical solution, the present invention has the following beneficial effects:

[0028] This invention presents a well-conceived method for identifying well logging lithology using a convolutional recurrent deep learning network based on a self-attention mechanism. It achieves lithology identification by constructing a core-logging mapping relationship. Compared to machine learning methods, this invention achieves higher accuracy, particularly in identifying fine-grained thin-layer sand bodies, far exceeding the capabilities of machine learning models. Furthermore, the use of weighted loss further enhances lithology identification capabilities with limited sample sizes.

[0029] Compared to other models, this invention proposes to construct a high-dimensional vector at the input end, which is equal to the number of lithology classifications. This vector is then passed through a long short-term memory recurrent neural network containing seven units to form an evolutionary sequence. Finally, the attention mechanism is used to select the optimal high-dimensional vector features to represent various lithologies, thus addressing the problem of overlapping logging responses among different lithologies. Evaluation and testing results show that the accuracy obtained by this invention is high, especially in the identification of fine-grained thin-layer sand bodies, which far exceeds that of machine learning models. This model has strong practical application value and can avoid the high uncertainty and low efficiency caused by subjective selection of data and parameters. Attached Figure Description

[0030] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0031] Figure 1This is a model structure diagram of the method for well logging lithology identification based on a self-attention mechanism convolutional recurrent deep learning network according to the present invention.

[0032] Figure 2 This document presents the model test results of the method for well logging lithology identification based on a self-attention mechanism convolutional recurrent deep learning network according to the present invention, and the first set of comparison results with AutoGluon.

[0033] Figure 3 This document presents the model test results of the method for well logging lithology identification based on a self-attention mechanism convolutional recurrent deep learning network according to the present invention, and the second set of comparison results with AutoGluon. Detailed Implementation

[0034] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] The present invention will be further explained below with reference to specific embodiments.

[0036] The method for well logging lithology identification based on a convolutional recurrent deep learning model with a self-attention mechanism provided in this embodiment includes the following steps:

[0037] (1) Dataset Construction

[0038] A sample was constructed using logging data from core samples, based on lithofacies type and cored well sections. The logging data served as sample characteristic values, and the lithofacies type results were used as sample labels. Due to the specific circumstances of the example data, the logging curve characteristic values ​​were 6 types of logging curves (…). Figure 1 This includes density logging (DEN), natural gamma logging (GR), deep and shallow dual lateral resistivity logging (RD and RS), sonic logging (AC), and compensated neutron logging (CNL). The label values ​​represent seven lithofacies types: conglomerate (GS), sandstone (SGS), coarse sandstone (CSS), medium sandstone (MSS), fine sandstone (FSS), siltstone (SSS), and mudstone (MDS).

[0039] (2) Model building

[0040] Determine the sample input method and basic model structure: The model structure is as follows Figure 1 As shown, the main structure consists of four layers: Convolutional Neural Network (CNN), Long Short-Term Memory Recurrent Neural Network (LSTM), Attention Mechanism (ATT), and Fully Connected Network (FC), built on the PyTorch deep learning framework.

[0041] The sample input method is as follows: the 6-dimensional feature vector is copied 7 times (representing the number of lithofacies types), and each copy is used to generate 7 high-dimensional feature vectors through an independent convolutional network (CNN) layer. Therefore, the CNN layer consists of 7 independent structures, each containing 5 basic operational units: a 1D convolutional unit (CNNV1D), a max pooling layer (MAXPOOL1D), and another 1D convolutional unit (CNNV1D). The 7 independent structures of the CNN layer output a total of 7 high-dimensional feature vectors, which are then concatenated and transformed before being input into a Long Short-Term Memory (LSTM) recurrent neural network layer. The LSTM layer consists of 7 sequentially arranged cells with shared weights, each cell outputting a high-dimensional vector, for a total of 7 high-dimensional vectors. The last hidden layer outputs (c... 7 The process involves seven structural units (cells) sharing common characteristics, serving as the search vector for the Attention Detection and Treatment (ATT) layer. The seven high-dimensional vector outputs of the Long Short-Term Memory (LSTM) recurrent neural network layer serve as key-value pairs for the ATT layer. After weighting by the ATT layer, a high-dimensional vector with unique characteristics is output and enters the fully connected network (FC) layer. The FC layer consists of one input layer, two hidden layers, and one output layer, reducing the high-dimensional vector from the ATT layer to a seven-dimensional vector corresponding to the lithofacies type.

[0042] (3) Model Training

[0043] Null and outlier values ​​were removed, sample feature values ​​were normalized, and label values ​​were encoded. Continuous cored well sections were reserved as the test dataset, and the remaining samples were divided into training and validation sets in an 8:2 ratio. The standard deviation standardization method (StandardScaler) from the scikit-learn module was used to convert the input feature data into a range with a mean of 0 and unit variance. Label encoding (LabelEncoder) was used to encode the seven lithofacies types using integers in the range [0,6]. Before loading into the hybrid model, the dataset was divided into batches of 20 samples each. Gradient descent optimization of the hybrid model was performed using cross-entropy loss and the Adam model optimization algorithm, while weighted cross-entropy loss was considered to handle imbalanced datasets. Weights were obtained based on the proportion of each lithofacies type. Weight decay regularization (L2) was used to address model overfitting. In addition, the hyperbolic tangent function (Tanh) is selected as the activation function for the convolutional network layer (CNN) and the fully connected network layer (FC); the learning strategy of the hybrid model is set as follows: learning rate 2e-5, decay rate 0.1, and learning rate decay will be performed when the validation loss decreases by less than 0.008 for 15 consecutive epochs.

[0044] (4) Model Evaluation

[0045] Evaluation metrics included precision, recall, balanced F1-score, and accuracy, all calculated using five-fold cross-validation. Amazon's Autogluon automated machine learning framework, proven effective in testing through research and competitions, was used as a comparison model to evaluate the proposed model. The validation set results show that the patented model achieved the same validation precision (0.83) as Autogluon without weighted loss. After applying weighted loss, the precision decreased to 0.78, but the recall for sandstone (small sample size) significantly improved (Table 1).

[0046] Table 1. Validation and evaluation results of the hybrid model and AutoGluon

[0047]

[0048]

[0049] The test set (two wells with continuous coring) also shows that Autogluon has a weak ability to identify thin-layered, fine-grained lithology, especially... Figure 3The rock was almost entirely identified as mudstone. However, the patented model exhibits significant advantages in thin-layered, fine-grained lithology, especially after applying weighted loss, which significantly improves the identification results. Figure 3 Therefore, although the weighted loss function has a negative impact on the accuracy of the hybrid model, the increased lithological recall with a small sample size and the test results from the two wells both demonstrate the improvement of the patent model by the weighted loss function. Furthermore, the decrease in accuracy is related to the decrease in mudstone recall.

[0050] The design of this invention fully considers (1) the requirements of rock physical properties for high-dimensional feature combination; (2) the common feature of rock physical properties as the basis for lithology identification; and (3) the individual feature of different lithologies having different rock physical advantage responses. That is, the one-dimensional convolutional neural network layer (1DCNN) converts the input features into high-dimensional feature vectors, and uses the parameter sharing of the long short-term memory recurrent neural network layer (LSTM) and the weighting ability of the attention mechanism layer (ATT) to sequentially realize the common and individual extraction of high-dimensional feature vectors, and outputs them through dimensionality reduction by the fully connected network layer (FC).

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for well logging lithology identification based on a convolutional recurrent deep learning network with a self-attention mechanism, characterized in that, Includes the following steps: (1) Dataset construction A sample was constructed using the lithological description results from drilling core samples and the logging data from the cored well sections; (2) Model building The sample input method and basic model structure are determined as follows: the input of sample feature vectors and dimensionality enhancement are achieved through one-dimensional convolution, and the number of repetitions of one-dimensional convolution is determined according to the number of lithology categories; then the results of one-dimensional convolution are combined and input into a long short-term memory recurrent neural network, and a self-attention mechanism is added to achieve weighted output to obtain a high-dimensional vector, and then dimensionality reduction is achieved through a fully connected layer; (3) Model training Remove null and outlier values, normalize sample feature values, encode label values, and divide samples into training, validation, and test sets; (4) Model evaluation Implement model building, parameter optimization, model training, and validation using PyTorch; The basic structure of the model in step (2) is classified into four layers: convolutional network layer, long short-term memory recurrent neural network layer, attention mechanism layer and fully connected network layer. The sample input method is as follows: the 6-dimensional feature vector is copied 7 times for lithofacies types, and 7 high-dimensional feature vectors are generated by passing each of the 6-dimensional feature vectors through independent convolutional network layers; The convolutional network layer includes 7 independent structures, which output 7 high-dimensional feature vectors. These vectors are then concatenated and transformed into dimensions before being input into the long short-term memory recurrent neural network layer. The long short-term memory recurrent neural network layer includes 7 sequentially arranged structural units with shared weights. Each structural unit outputs a high-dimensional vector, for a total of 7 high-dimensional vectors. The last hidden layer output goes through 7 structural units and has common features. It serves as the search vector of the attention mechanism layer. The 7 high-dimensional vector outputs of the long short-term memory recurrent neural network layer serve as key-value pairs of the attention mechanism layer. After being weighted by the attention mechanism layer, a high-dimensional vector with individual features is output and enters the fully connected network layer.

2. The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism as described in claim 1, characterized in that, The well logging data in step (1) are sample feature values, and the lithological description results are used as sample labels.

3. The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism as described in claim 1, characterized in that: The fully connected network layer consists of one input layer, two hidden layers, and one output layer, which reduces the high-dimensional vector of the attention mechanism layer to a 7-dimensional vector corresponding to the lithofacies type.

4. The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism as described in claim 1, characterized in that, Each of the independent structures comprises five basic units in sequence, namely: a one-dimensional convolutional unit, a one-dimensional convolutional unit, a max pooling layer, a one-dimensional convolutional unit, and a max pooling layer.

5. The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism as described in claim 1, characterized in that, The activation function for the convolutional network layer and the fully connected network layer is the hyperbolic tangent function. The model learning strategy is set as follows: learning rate 2e-5, decay rate 0.

1. When the validation loss decreases by less than 0.008 for 15 consecutive cycles, the learning rate will be decayed.

6. The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism as described in claim 1, characterized in that: The ratio of the training set to the validation set in step (3) is 8:

2.

7. The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism as described in claim 1, characterized in that, The specific steps of step (3) are as follows: (3.1) The standard deviation standardization method in the scikit-learn module is used to convert the input feature data into a range with a mean of 0 and a unit variance. The label encoding method is used to encode the 7 lithofacies types using integers in the range [0,6]. (3.2) Before loading into the hybrid model, the dataset is divided into batches of 20 samples. The hybrid model is optimized by gradient descent using the cross-entropy loss method and model optimization algorithm. At the same time, weighted cross-entropy loss is considered to handle the imbalanced dataset. The weight values ​​are obtained by the proportion of each lithofacies type.

8. The method for well logging lithology identification using a convolutional recurrent deep learning network based on a self-attention mechanism as described in claim 1, characterized in that: In step (4), the evaluation indicators for model evaluation are precision, recall, balanced F-score and accuracy, and the results are calculated using the results of five-fold cross-validation.

Images