Lithology Identification Method Based on Improved Tabnet Model

By using an improved Tabnet model, combined with a Transformer encoder and regularization coefficients, the error problem of nonlinear mapping relationships in lithology identification was solved, enabling accurate identification of complex reservoirs and balanced processing of lithology samples, thus improving the accuracy and stability of lithology identification.

CN121030445BActive Publication Date: 2026-11-17SOUTHWEST PETROLEUM UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511035972.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-07-26
Publication Date
2026-11-17
Estimated Expiration
2045-07-26

AI Technical Summary

Technical Problem

Existing lithology identification methods have significant errors when dealing with nonlinear mapping relationships, making it difficult to accurately identify the lithology of complex reservoirs. Furthermore, imbalanced lithology samples lead to a decline in model performance.

Method used

An improved Tabnet model is adopted, which adds a Transformer encoder module before the attention converter module at each step, combines regularization coefficients and Bsmote algorithm for data equalization, uses Bayesian optimization to tune hyperparameters, and introduces ReLU activation function and Adam optimizer to improve the model’s feature selection ability and recognition accuracy.

Benefits of technology

It effectively processes complex well logging data, improves the accuracy of lithology identification, realizes global feature extraction and interpretability of lithology samples, solves complex nonlinear problems in lithology identification, and enhances the model's identification accuracy and stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121030445B_ABST
    Figure CN121030445B_ABST
Patent Text Reader

Abstract

This invention relates to a lithology identification method based on an improved Tabnet model, comprising the following steps: constructing an improved Tabnet model, including adding a Transformer encoder module before the attention converter module at each step of the Tabnet network to form a composite attention converter module; training the improved Tabnet model; and using the trained improved Tabnet model for lithology identification. This application not only performs distributed decision-making and progressively outputs complex feature representations between parameters, but also synchronously learns the sequential and long-distance dependencies between features, achieving global feature extraction. This effectively processes complex well logging data and improves the accuracy of lithology identification. For the case of imbalanced lithology sample data in the study area, the Bsmote algorithm is used to obtain balanced lithology samples for lithology prediction. Finally, this deep learning algorithm solves the complex nonlinear problem of lithology identification in conglomerate reservoirs, achieving accurate lithology identification in the study area.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of oil and gas exploration technology, and in particular to a lithology identification method based on an improved Tabnet model. Background Technology

[0002] In oil and gas reservoir exploration and development, lithology identification is a crucial component of reservoir logging evaluation. Identifying the lithology of deep, complex areas helps obtain geological information that cannot be obtained through sampling. It is also fundamental for solving reservoir geological parameters and reservoir description. Lithology identification analysis not only enhances the understanding of subsurface geological conditions but also effectively guides various decisions during the development process. Conventional lithology identification methods typically assume a linear relationship between reservoir logging parameters, which can lead to significant errors in lithology judgment results. Due to the strong heterogeneity among reservoirs in the study area, complex feature mapping relationships often exist between different lithologies and logging data. Machine learning methods have advantages in handling nonlinear mapping relationships and solving complex problems. They can reduce manual costs and accelerate manual interpretation. Therefore, artificial intelligence algorithms have significant application value in lithology identification based on logging curves.

[0003] TabNet is an innovative model based on an attention mechanism that adaptively selects global and local features using an internal sequential multi-step framework. Each step approximates the decision node of a neural network. During training at each step, the input data is first processed by a feature segmentation module. This means that in each layer, only a subset of features are selected for output computation, ensuring feature sparsity and suppressing overfitting. Typically, the TabNet model takes B×D inputs and outputs, where B is the batch size and D represents the dimension of the input features. Each step of TabNet primarily consists of an attention converter and a feature converter. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a lithology identification method based on an improved Tabnet model.

[0005] This invention is achieved through the following technical solution:

[0006] The lithology identification method based on the improved Tabnet model includes the following steps:

[0007] Construct an improved Tabnet model: This involves adding a Transformer encoder module before the attention converter module at each step of the Tabnet network to form a composite attention converter module;

[0008] Train the improved Tabnet model;

[0009] Lithology identification was performed using a trained, improved Tabnet model.

[0010] Furthermore, TabNet introduces regularization coefficients, which are used to control the distribution of the learned mask values ​​as close to 0 and 1 as possible.

[0011] Preferably, natural gamma, acoustic transit time, neutron, density, resistivity, flushing zone resistivity, spontaneous potential, wellbore, wellbore difference, and depth are used as inputs to the model.

[0012] Preferably, large and medium gravel, small and medium gravel, boulders, mudstone, sandstone, coarse gravel, and fine gravel are used as the output of the model.

[0013] Furthermore, before training, the Bsmote algorithm is used to perform data equalization processing on the sample dataset for lithology identification.

[0014] Furthermore, after equalization, the data is normalized using Z-score.

[0015] Preferably, the Bayesian optimization method is used to tune the hyperparameters.

[0016] Furthermore, model training includes the following steps:

[0017] Based on well logging data and well logging data of the study area, depth information, well logging parameters, and lithology label data were selected as the lithology identification dataset. The original depth and well logging data were dimensionless and normalized. Then, the normalized dataset was divided into training set and test set for model training and verification of identification accuracy.

[0018] In the improved Tabnet model, hyperparameters are selected with lithology identification accuracy as the objective. Based on the various hyperparameters obtained during the optimization process, a prior dataset is obtained and a Gaussian model is fitted. After calculating the posterior probability distribution through the Gaussian model, the covariance and expected mean of the objective function of each group of hyperparameters are used as indicators to determine the hyperparameters corresponding to the maximum values ​​of the objective function. After completing multiple iterations of training, the optimal combination of hyperparameters for lithology identification is determined.

[0019] The optimized hyperparameter combination is applied to the improved Tabnet model, and the lithology identification dataset is imported into the model for intelligent identification of lithology samples.

[0020] Preferably, the improved Tabnet model selects the ReLU activation function as the activation function for the hidden layer neurons.

[0021] Preferably, batch size, learning rate, and number of iterations are optimized using cross-validation.

[0022] Compared with the prior art, this application has at least the following beneficial effects:

[0023] 1. The improved Tabnet neural network model for lithology identification proposed in this application, by adopting a data segmentation method, can not only make distributed decisions and gradually output complex feature representations between parameters, but also simultaneously learn the sequential and long-distance dependencies between features to achieve global feature extraction. This enables it to effectively process complex well logging data and improve the accuracy of lithology identification. Furthermore, it visualizes the global importance of each sample and the differences in feature importance between features, thus giving the deep learning model interpretability.

[0024] 2. To address the imbalance of lithological sample data in the study area, the Bsmote algorithm is used to obtain balanced lithological samples for lithology prediction. Ultimately, this deep learning algorithm is used to solve the complex nonlinear problem of lithology identification in conglomerate reservoirs, achieving accurate lithology identification in the study area. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a diagram of the improved Tabnet single-Step structure in the embodiment;

[0027] Figure 2 This is a structural diagram of the Tabnet feature converter in the embodiment;

[0028] Figure 3 This is a structural diagram of the Transformer encoder in the embodiment;

[0029] Figure 4 This is a structural diagram of the composite attention converter in the embodiment;

[0030] Figure 5 This is a lithology distribution map before and after sample equalization in the example;

[0031] Figure 6 This is a flowchart of the improved Tabnet neural network lithology identification process in the embodiment;

[0032] Figure 7 This is a graph showing the results of cross-plot analysis between parameters in the example;

[0033] Figure 8 This is the FMI lithological classification standard diagram used in the examples;

[0034] Figure 9 This is a diagram showing the lithological identification results of well M using FMI in the example.

[0035] Figure 10 This is a comparison table of the classification confusion matrix results after sample balancing for each model in the embodiments;

[0036] Figure 11 The diagram shows the iterative graph of the improved Tabnet loss function in the embodiment.

[0037] Figure 12 The figure shows the ROC curve and AUC experimental results of the improved Tabnet model in the example.

[0038] Figure 13 The graph shows the PR curve and AUC experimental results of the improved Tabnet model in the example.

[0039] Figure 14 This is a feature importance graph from the embodiments;

[0040] Figure 15 This is a feature heatmap of the first 100 samples in the example;

[0041] Figure 16 The improved Tabnet model is used to identify the entire well section of well A in this example.

[0042] Figure 17 The example shows the improved Tabnet model's identification results for the entire well section of well B. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0044] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined with each other. It should also be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0045] 1. The principle of the lithology identification method based on the improved Tabnet model provided in this embodiment

[0046] The geological conditions in the study area of ​​this embodiment are complex, and the sandstone and conglomerate oil reservoirs exhibit high heterogeneity. The Tabnet network structure maintains the end-to-end feature learning capability of neural networks while possessing the good interpretability of decision tree models, thus effectively processing multidimensional data to study the complex geological information in the region. The Transformer Enceder module effectively captures long-range dependencies in the input data, aiding in the extraction of global features. In this embodiment, the attention converter at each step of the Tabnet encoder is combined with the Transformer encoder module to form a composite attention converter module. This allows Tabnet to improve its global feature extraction capability during feature selection, enhance the connections between various logging parameter features, and fully explore the importance of features. The overall structure diagram of the improved Tabnet model is shown below. Figure 1 As shown.

[0047] Tabnet's feature transformers consist of a series of learnable linear transformations and non-linear activation functions, designed to extract more useful information from the raw input features. For example... Figure 2 As shown, the feature converter's internal structure consists of two parts: a cross-decision-step shared module in the first half and a single-step independent decision module in the second half. In the first half, the parameters are shared across all decision steps and are trained simultaneously on all decision steps. In the second half, the single-step independent decision module trains the parameters independently only on the corresponding decision step. By adopting this structure, TabNet can learn the common features of all previous decision steps and effectively extract the characteristics of the corresponding features in the current decision step. This helps the model learn the patterns and regularities that are prevalent in the input data, and dynamically adjusts the weights of each feature according to the specific needs of the current task in each decision step. This composite structure can effectively improve the model's computational efficiency and enhance its interpretability.

[0048] Both parts of the feature transformer consist of fully connected (FC) layers, batch normalization (BN) layers, and gated linear unit (GLU) layers. The FC fully connected layer is responsible for extracting useful features from the original input. It maps the input features to a new feature space through a set of learnable weight matrices, enabling the model to learn and capture the complex relationships between input features and transform these features into a form suitable for subsequent processing.

[0049] Batch Normalization (BN) layers are an important component of many convolutional neural network architectures. This module standardizes each small batch of input data. During model training, the BN layer records the mean and standard deviation of each batch. After all data has been trained, the mean and standard deviation of all batches are calculated again, thereby constraining the distribution of data in the output data of the BN layer.

[0050] The GLU layer is a CNN network architecture capable of processing sequential data in parallel. It controls the transmission of information through a gating mechanism. GLU combines two parallel linear transformations, one of which is activated using the sigmoid activation function. The result of this activation is multiplied by the other linear transformation, thus selectively emphasizing certain features. Dividing by the square root of 0.5 during residual connections at each layer of the feature transformer is to control the gradient during backpropagation. This process effectively prevents large variance due to excessively large gradients, ensuring the stability of the training process.

[0051] The attention converter calculates the learnable mask value M[i] for the current execution step based on the calculation result of the previous execution step, as shown in equation (1). By calculating the mask value, the model can dynamically select the most relevant features. This achieves sparsity in input feature selection, thereby reducing unnecessary computational overhead and improving the model's generalization performance.

[0052] M[i] = Sparsemax(P[i-1]·h i (a[i-1])) (1)

[0053]

[0054] In the above formula, α[i-1] represents the features segmented by the feature segmentation module in the previous decision step; h i This represents the operation of the FC+BN layer in the current decision step; it also represents the feature importance of a certain feature in the previous execution step, and its calculation result is related to the mask value σ, i.e.: γ is an adjustable parameter used to adjust the weight ratio of a certain feature in the current execution step, so as to adaptively reduce its weight. The closer γ is to 1, the more limited the probability of the feature being applied if the feature has been used multiple times in previous decision steps. Appropriately reducing γ can make the probability of the feature being used in the current decision step higher, and the feature may be selected multiple times, that is, the constraint condition is more "soft". i is the number of execution steps corresponding to the ongoing execution step in the model prediction process, and j is any number of execution steps between the first execution step and the currently ongoing execution step.

[0055] ViT (Vision Transformer) is a deep neural network with an attention mechanism. Compared to traditional convolutional neural networks, ViT is better at analyzing large-scale data. In the ViT model, the input image is first segmented into fixed-size image patches, and these patches are then converted into a vector sequence that the model can process. Essentially, ViT is still processing sequential data. The Transformer encoder is the core component of Vision Transformer, responsible for feature extraction from the acquired sequential data. This module processes the data through a multi-head attention mechanism, a multilayer perceptron, and batch normalization. The Transformer encoder module structure is as follows: Figure 3 As shown.

[0056] like Figure 4 As shown, a composite attention converter module is formed by adding the Transformer encoder module from the ViT structure before the attention converter module in each step of Tabnet. By adding this module, the model can enhance the relationship between features and more effectively explore the importance of features when performing local feature analysis after feature extraction in each step.

[0057] Furthermore, to further enhance the model's ability to select sparse features, Tabnet's attention converter module introduces an additional regularization coefficient L. sparse As shown in equation (2):

[0058]

[0059] In the above formula, M b,j [i] represents the contribution of the j-th feature in the b-th sample to the overall decision; N steps Let M[i] be the number of decision steps currently executed, B be the batch size, D be the dimension of the input features, i be the number of steps corresponding to the ongoing execution step in the model prediction process, and ε be a perturbation coefficient. This regularization term aims to control the distribution of M[i] as close to 0 and 1 as possible, i.e., L... sparse The smaller the sample size, the sparser M[i]. The attention transformer first performs a linear transformation on the original features to generate a new set of feature representations, and then calculates the importance score of each feature using a learnable matrix to reflect the importance of each feature to the current decision. After sparse selection using the calculated attention scores, the selected important features are re-aggregated and used for subsequent prediction tasks.

[0060] The regularization coefficient L was introduced. sparse The specific process can be summarized as follows:

[0061] (1) The original input features are first processed by a shared feature converter consisting of a fully connected layer + BN + GLU to generate a new set of feature representations.

[0062] (2) Generate a new set of feature representations by calculating the importance score M of each feature through a learnable matrix and a sparse activation function (such as Sparsemax) in the attention converter. b,j [i], M b,j [i] represents the degree of contribution of the j-th feature in the i-th decision step to the current decision.

[0063] (3) The regularization coefficient is the importance score M introduced into the attention converter for each feature. b,j In the calculation of [i], formula (2) directly applies to the attention weight M. b,j [i].

[0064] M b,j [i] is a probability distribution (satisfying ∑jMb,j[i]=1). The sparse regularization term forces this distribution to approach a one-hot form (i.e., most feature weights are close to 0, and a few are 1). This is used to optimize M. b,j The sparsity of [i].

[0065] (4) This regularization term is added to the total loss function and jointly optimized with the classification loss (such as cross-entropy). The total loss function is usually calculated as shown in the following formula:

[0066] L total =L task +λL sparse

[0067] In the above formula, L total It is the total loss function, L task This is the cross-entropy of the lithology identification and classification task, L. sparse λ is the regularization coefficient, and λ is the weighting coefficient of the sparse regularization term, which controls the strength of sparsity.

[0068] A regularization term is added to the total loss function, which is then incorporated into the model and affects the calculation of attention weights in the attention converter. During training, the gradient simultaneously updates the learnable parameters of the attention converter and the weights of the feature converter, causing the model to automatically tend towards sparsity in feature selection, thereby further enhancing the model's ability to select sparse features.

[0069] 2. Performance analysis of the lithology identification method provided in this embodiment

[0070] 2.1 Data Sources and Preprocessing

[0071] The lithological identification data used in this embodiment comes from well logging data of 8 wells in Block M of the Mahu area in the Junggar Basin. The selected logging curves for lithological identification include: natural gamma (GR), sonic transit time (AC), neutron density (CNL), density (DEN), resistivity (RT), RXO (flushed zone resistivity), spontaneous potential (SP), diameter (CALC), and diameter difference (CALI). Simultaneously, due to the depth sequence of the selected logging parameters, the depth label is also used as a feature parameter for lithological identification. The logging curves and their functions are shown in Table 1.

[0072] Table 1: Various logging curves and their functions

[0073]

[0074]

[0075] Based on previous lithological classifications and labeling methods, lithological identification targets are divided into the following seven categories: 0-large to medium gravel, 1-small to medium gravel, 2-giant gravel, 3-mudstone, 4-sandstone, 5-coarse gravel, and 6-fine gravel. The numbers 0-6 are used as lithological labels. A total of 9529 data points from four wells were selected for intelligent identification of lithological samples, and the training and test sets were divided in a 7:3 ratio.

[0076] Since the improved Tabnetnet model used in the lithology identification sampling of this embodiment can adaptively select features, no manual feature selection is required before inputting the dataset into the model. Therefore, the nine selected well logging curves (GR, AC, CNL, DEN, RT, RXO, SP, CALC, CALI) and the depth label, totaling 10 feature variables, are used as input variables for model training. Some data are shown in Table 2.

[0077] Table 2: Partial Well Logging Data

[0078] Depth GR AC CNL DEN RT RXO SP CALC CALI Lithology 1980 63.94 72.766 15.8 2.426 9.032 3.29 17.33 0.137 9.236 3 1980.125 64.02 71.056 15.8 2.441 10.201 3.83 16.99 0.148 9.283 3 1980.25 66.46 69.505 16.1 2.455 12.28 5.408 16.87 0.126 9.339 3 1980.375 71.46 69.753 18.9 2.468 11.931 5.223 15.79 0.085 9.262 3 1980.5 71.45 71.433 18.1 2.447 10.248 5.122 15.79 0.016 9.258 3 …… …… …… …… …… …… …… …… …… …… …… 3069.5 80.28 72.037 23.3 2.54 9.065 9.334 55.50 0.132 8.955 7 3069.625 79.59 72.325 23.3 2.57 8.114 9.603 55.50 0.071 8.783 7

[0079] Because lithology identification datasets often exhibit imbalances across different lithology classes, models tend to focus more on the majority class samples during training, neglecting information about minority class samples. This significantly impacts the performance of classification models. Data balancing, a widely used oversampling technique in machine learning and deep learning, provides a balanced dataset, allowing models to perform better. This embodiment employs the Bsmote algorithm for data balancing of the lithology identification sample dataset. This method only oversamples minority class samples located at boundaries. Specifically, if a minority class sample is surrounded by a majority class sample, it is oversampled. This method significantly improves computational efficiency and reduces memory usage compared to the Smote algorithm.

[0080] Figure 5 The distribution characteristics of various lithological categories before and after sample equalization are shown. It can be seen that in the original dataset, fine conglomerate samples were the most numerous, with 2254 samples, accounting for 23.5% of all samples, while sandstone samples were the fewest, with only 668 samples, accounting for 7% of all samples. That is, the number of fine conglomerate samples, which had the largest proportion in the original data, was 3.1 times that of sandstone samples, which had the smallest proportion, indicating an uneven distribution of the overall data. After BSMOTE sample equalization, the number of samples from minority categories such as mudstone, sandstone, and coarse conglomerate significantly increased, accounting for 14%, 13%, and 12% of the entire dataset, respectively, while the most numerous fine conglomerate samples accounted for 16%. The BSMOTE oversampling algorithm has, to a limited extent, improved the uneven distribution of lithological categories in the original dataset.

[0081] Due to limitations in logging technology and the environment during the data acquisition process, noise points and outliers may exist in the data points. To avoid the impact of systematic errors and different units of measurement on the classification results, Z-score normalization is performed on the data.

[0082]

[0083] In the above formula, k = 1, 2, 3, ..., n, x k For the Kth sample before normalization, x' k Let μ be the normalized Kth sample, μ be the mean of the dataset, and σ be the standard deviation of the dataset.

[0084] 2.2 Hyperparameter Optimization Method

[0085] Hyperparameter tuning is a fundamental step in machine learning model prediction. It involves determining the specific values ​​of these parameters with the aim of achieving optimal performance. Common hyperparameter tuning methods include grid search, random search, and Bayesian optimization. However, as the number of hyperparameters increases, the time consumption of grid search and random search also increases. Bayesian optimization utilizes approximation and incorporates previous parameter information to continuously update the prior, building a surrogate function prediction model based on past objective function evaluation results. This helps find the minimum objective function value. Compared to grid search and random search algorithms, Bayesian optimization refers to previous evaluation results when attempting to optimize the next set of hyperparameters, saving significant computation time and resources and reducing time complexity. Therefore, this embodiment uses Bayesian optimization for hyperparameter selection.

[0086] 2.3 Performance Indicators

[0087] When using intelligent algorithms for lithology identification and prediction, the performance of the model is evaluated using metrics such as confusion matrix, accuracy, precision, recall, F1 score, ROC, AUC value, and PR curve.

[0088] (1) Accuracy

[0089] Accuracy measures the proportion of samples that make correct predictions out of the total sample. It is calculated as follows:

[0090]

[0091] In the above formula, the explanations of TP, TN, FP, and FN are as follows: The first letters T and F represent the prediction situation, that is, T means the prediction result is correct and F means the prediction result is wrong; the second letter represents whether the prediction result is a positive sample or a negative sample.

[0092] (2) Precision, recall and F1 score

[0093] Precision (Pre) refers to the probability that a data point is correctly predicted when the predicted outcome is positive.

[0094]

[0095] Recall is relative to the original sample; it represents the probability that a sample that was actually positive will be predicted as positive.

[0096]

[0097] The F1 score is a metric that comprehensively evaluates precision and recall; the higher the F1 score, the higher the model quality.

[0098]

[0099] (3) Confusion matrix

[0100] In classification problems, the confusion matrix uses precision, recall, and F1 score as metrics to evaluate the model's classification performance. Taking binary classification as an example, the confusion matrix reflects the difference between the label and the predicted result, and there are four combinations: true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). The sum of the number of each combination represents the number of samples. Common binary classification matrices are shown in Table 3.

[0101] Table 3: Binary Confusion Matrix

[0102]

[0103] (4) ROC curve, PR curve and AUC value

[0104] ROC curves are a graphical method for demonstrating the performance of a classification model at different thresholds. In an ROC curve, the horizontal axis represents the "true positive rate" and the vertical axis represents the "false positive rate". The "true positive rate" refers to the proportion of positive examples out of all positive samples, and the formula for the true positive rate is shown in (8). The "false positive rate" refers to the proportion of negative examples out of all negative samples, as shown in formula (9). The closer the ROC curve is to the upper left corner, the stronger the model's discriminative ability.

[0105]

[0106] The recall curve (PR curve) is plotted on the horizontal axis and precision on the vertical axis. Model performance is typically assessed using the cross-sectional area (BEP) of the PR curve (larger BEP indicates better classification performance), the balance point between recall and precision, and the F1 score. Generally, a higher BEP or F1 score is considered to indicate better learner performance.

[0107] The AUC value is used to intuitively evaluate the performance of a classification model. The AUC value is typically represented by the area enclosed by the horizontal axis and the line x=1 in the ROC curve, or the area enclosed by the x and y axes in the PR curve. Since the ROC and PR curves are usually located above y=±x, the AUC value generally ranges between 0.5 and 1. A general criterion for judging the AUC value is: AUC=0.5-0.7 indicates low performance; AUC=0.7-0.85 indicates average performance; AUC=0.85-0.95 indicates very good performance; and AUC=0.95-1 indicates excellent performance.

[0108] 2.4 Model Training Process

[0109] 2.4.1 Model Training Process

[0110] The operation flow of the Bayesian-optimized intelligent logging lithology identification model is as follows:

[0111] (1) Based on the obtained well logging and well logging data of the study area, depth information, well logging parameters, and lithology label data were selected as the lithology identification dataset. To address the imbalance in the number of samples for different types of lithology, the Bsmote oversampling algorithm was used for sample equalization. The equalized depth, well logging, and well logging data were then normalized to avoid the impact of different data dimensions on subsequent model training. The normalized dataset was divided into training and testing sets for model training and verification of identification accuracy.

[0112] (2) The lithology identification dataset was imported into the improved Tabnet lithology identification model for intelligent identification of lithology samples. Bayesian optimization was performed. In the improved Tabnet model, the range of hyperparameters was set. The lithology identification accuracy was selected. To optimize the hyperparameters for the objective, the Bayesian optimization algorithm is shown in formula (10). The range of each hyperparameter is determined, and based on the various hyperparameters obtained during the optimization process, a prior dataset is acquired and a Gaussian model is fitted. After calculating the posterior probability distribution using the Gaussian model, the hyperparameters corresponding to the maximum values ​​of the objective function are determined using the covariance and expected mean of the objective function for each group of hyperparameters as indicators. After multiple iterations of training, the optimal combination of hyperparameters suitable for lithology identification in this embodiment is finally determined.

[0113]

[0114] In the above formula, γ represents the solution obtained by the Bayesian optimization algorithm; x represents any combination of parameters in the lithology identification dataset; and X represents the search space of hyperparameters, which is the set of all possible combinations of hyperparameters. The lithology identification accuracy is represented by different combinations of hyperparameters.

[0115] The optimized hyperparameter combination is applied to the improved Tabnet lithology identification model; the lithology identification dataset is imported into the model for intelligent identification of lithology samples.

[0116] (3) The model's predictive performance was evaluated using metrics such as accuracy, precision, recall, and F1 score. Traditional methods (cross-plotting, FMI) and various machine learning algorithms were applied to lithology identification in the same work area to evaluate the predictive performance of each method. The specific lithology identification process is as follows: Figure 6 As shown.

[0117] 2.4.2 Parameter Tuning

[0118] By introducing activation functions during the training of neural networks, the network can effectively fit various nonlinear functions. Furthermore, by adjusting the region of individual input values ​​for neurons, the computational efficiency of the neural network can be improved, thereby significantly enhancing the model's ability to learn nonlinear relationships between features. Common activation functions include the sigmoid function, the Tanh function, and the ReLU function.

[0119] The Tanh function has a derivative of 0 in the regions of positive and negative infinity, leading to the vanishing gradient problem. If the initial weights of the sigmoid function are in the range [0,1], the gradient will continuously decay and approach 0 during multiple iterations of the backpropagation algorithm's derivative calculation. If the initial weights are in the range [1,+∞), the exploding gradient problem will occur. The ReLU function, however, is more computationally efficient and achieves network sparsity to reduce interference from noisy data. Therefore, the improved Tabnet model chooses the ReLU activation function as the activation function for the hidden layer neurons.

[0120] To ensure the improved Tabnet model achieves good classification performance, other hyperparameters need to be optimized during model training to find the optimal combination. For hyperparameters affecting classification performance, such as batch size, learning rate, and number of iterations, cross-validation is used for optimization.

[0121] Batch size: This refers to the number of samples fed into the neural network for training each time. Setting a reasonable batch size can accelerate model convergence and save computer memory resources. The value of this parameter is related to the computer hardware performance and is generally an integer multiple of 2.

[0122] Learning rate: The learning rate controls the range of steps in which weights are updated during a single iteration in a neural network. Setting this parameter too high will result in an excessively large range of weight updates, leading to unstable training; setting it too low will cause the network to converge too slowly and easily get trapped in local minima.

[0123] Number of iterations: This parameter determines the adequacy of model training. Setting too few iterations will result in insufficient model training; setting too many iterations may lead to overfitting.

[0124] In addition, the Adam optimizer is used to adaptively adjust the learning rate, efficiently handling sparse gradients and noise to improve model performance. Table 4 shows the model parameter values ​​obtained using Bayesian optimization.

[0125] Table 4: Model Parameter Optimization Results

[0126]

[0127] 2.5 Analysis of Experimental Results

[0128] 2.5.1 Analysis of the Recognition Effect of Traditional Methods

[0129] This embodiment employs the traditional cross-plot method and the FMI method to identify the lithology of the study block. The cross-plot method is a commonly used lithology identification method. The distribution of data points on a plane is observed through the cross-plot to classify lithologies. Well logging response analysis of different lithologies shows that resistivity, sonic transit time, neutron density, and density are sensitive to lithological responses. Based on these parameters and combined with well logging data, cross-plot analysis is performed. The results of the cross-plot analysis between various parameters are as follows: Figure 7 As shown.

[0130] FMI (Full-bore Microresistivity Scanning Imaging) method measures the microresistivity changes around the wellbore using a button electrode, generating a two-dimensional image of the wellbore using high-resolution scan data to visually reflect the formation structure and structural features. Lithological classification is then performed based on quantitative or semi-quantitative parameter interpretation. Based on previous experience and the formation depth of the corresponding blocks, lithological identification standards based on imaging logging have been established, such as... Figure 8 As shown.

[0131] Lithological calibration was performed based on well logging data from the study area, combined with... Figure 8 The lithology identification standard was established, and the conglomerate lithology in the study area was identified using the FMI method. The lithology identification effect was evaluated by generating FMI dynamic and static electrical imaging logging maps. Figure 9 This is a lithological identification diagram based on previous studies of a well in the study area at a depth of 3175-3177 meters, divided according to lithological standards. It can be seen that the logging lithology is deposited according to gravel size. FMI logging interpretation of the selected layers, from top to bottom, shows: coarse conglomerate, fine conglomerate, and coarse conglomerate, which is basically consistent with the logging data.

[0132] The FMI method is only applicable to well sections with specific logging data, and its identification range is limited. Furthermore, for reservoirs with complex lithological distributions, the logging curves usually also have complex properties, which makes it difficult for researchers to carry out lithological identification work.

[0133] 2.5.2 Analysis of Data Balance Effect

[0134] Compared to traditional lithology identification methods, artificial intelligence methods can uncover the complex mapping relationship between well logging curves and lithology categories, and fuse multi-dimensional attribute features. This allows for lithology identification results without needing to visualize well logging curve attributes, making it highly valuable for application. However, due to the randomness inherent in dataset partitioning and model training, the classification prediction results may be subject to chance. To verify the effectiveness of the model's classification performance, a 5-fold cross-validation method was used to train the lithology identification model. The dataset was divided into five subsets of approximately equal size. Five experiments were conducted, each time selecting one fold as the validation set and the remaining four folds as the training set, for a total of 10 validations. The model's performance on the validation set was recorded during each validation. The lithology sample dataset in this embodiment, before and after Bsmote sample equalization, and after training with the improved Tabnet model using 5-fold cross-validation, yields the validation results shown in Tables 5-8.

[0135] Table 5: Results of Cross-Validation Before Five Folds of Training Set Sample Balance

[0136]

[0137]

[0138] Table 6: Results of the First Five-Fold Cross-Validation of Test Set Sample Balance

[0139]

[0140] Table 7: Results of Five-Fold Cross-Validation After Training Set Sample Balance

[0141]

[0142] Table 8: Results of Five-Fold Cross-Validation After Sample Balance on the Test Set

[0143]

[0144]

[0145] It can be seen that after the five-fold cross-validation experiment, the test set recognition accuracy of the dataset before and after sample balancing is about 0.9 and 0.94, respectively. This indicates that after Bsmote sample balancing, the lithology classification effect of the improved Tabnet model is further improved, and the lithology prediction model has high stability.

[0146] To verify the superior performance and prediction accuracy of the rock recognition model in this embodiment, after sample equalization processing of the dataset, the prediction results of the improved Tabnet model are compared with the prediction results of the Tabnet model, RF, SVM, Resnet18, and stacked S-BiLSTM-CNN+ATTENTION (a fusion deep learning model). Figure 10 (a)-(f) are the confusion matrix results of each model on the lithology identification test set after Bsmote sample equalization. Table 9 compares the classification accuracy of each model before and after sample equalization.

[0147] Table 9: Accuracy Comparison Before and After Sample Equilibrium

[0148] Model Accuracy before sample equalization Accuracy after sample equalization Improvement rate RF 69% 84% 15% SVM 72% 77% 5% Resnet18 76% 88% 12% S-litm-cnn+Attention 79% 86% 7% Tabnet 89% 92% 2% Improved Tabnet 92% 95% 4%

[0149] Comparison Table 9 Figure 10The results show that the prediction accuracy of each model was improved after sample equalization. Specifically, the prediction accuracy of Random Forest improved by 15%, Support Vector Machine by 5%, and ResNet18 by 12%. S-Bilstm-CNN-attention improved the recognition accuracy by 7%. In the Random Forest model's predictions, the error rate for identifying large and medium conglomerate was 12%, for small and medium conglomerate it was 33%, for giant conglomerate it was 29%, and for mudstone it was 7%. The error rates for coarse and fine conglomerate were 25% and 6%, respectively. In contrast, the Support Vector Machine model had an error rate of 23% for large and medium conglomerate, 46% for small and medium conglomerate, 34% for giant conglomerate, 6% for mudstone, 43% for coarse conglomerate, and 6% for fine conglomerate. In the ResNet18 model, the error rates for identifying large / medium conglomerate, small / medium conglomerate, and megaconite were 11%, 27%, and 15%, respectively; the error rate for mudstone was 7%; and the error rates for coarse and fine conglomerate were 17% and 5%, respectively. In the S-Bilstm-CNN-attention model, the error rates for large / medium conglomerate, small / medium conglomerate, and megaconite were 13%, 31%, and 17%, respectively; the error rate for mudstone was 7%; and the error rates for coarse and fine conglomerate were 17% and 6%, respectively. For the Tabnet model, the improved error rate for large / medium conglomerate was 12%, a decrease of 3% compared to the original model. The error rate for small / medium conglomerate was 6%, a decrease of 2% compared to the original model. The error rate for megaconite was 6%, a decrease of 1% compared to the original model. The error rate for mudstone was 1%, a decrease of 4% compared to the original model. The accuracy rate for coarse conglomerate identification was 12%, an increase of 2% compared to the original model. The error rate for fine conglomerate identification was 5%, a decrease of 1% compared to the original model. Overall, the improved Tabnet model exhibited the lowest error rate and highest accuracy in identifying complex lithology in the study area compared to other models, demonstrating its ability to effectively divide the core samples from the study area.

[0150] 2.5.3 Analysis of Recognition Results of the Improved Tabnet Model Algorithm

[0151] To further examine the improvement of the Tabnet model in classifying various lithologies, the classification indices of seven lithologies in different models were compared, as shown in Table 10.

[0152] Table 10: Comparison of Classification Results Indicators for Each Model

[0153]

[0154]

[0155] The table above shows that the optimal F1 scores for each lithology category are 0.929, 0.952, 0.928, 0.975, 0.987, 0.927, and 0.967, respectively. Except for sandstone, whose optimal F1 value comes from the Tabnet model, the other lithology categories all come from the improved Tabnet classification model. Overall, the improved Tabnet model exhibits the best classification performance.

[0156] During neural network training, the loss function primarily quantifies the difference between the network model's predictions and the actual values. A smaller loss function indicates higher model stability. Cross-entropy is commonly used as the loss function to measure the distance between the probability distribution of the model's predicted labels and the true label distribution, thereby evaluating model performance. Figure 11 The image shows the loss function curve of the improved Tabnet model. It can be seen that no overfitting occurred during training, and the model exhibits high stability.

[0157] The performance of the improved Tabnet classification model was validated using ROC curves, PR curves, and AUC values. Specifically, the ROC curves, PR curves, and corresponding AUC values ​​of the improved Tabnet model for seven lithologies were obtained on the test set. Figure 12 , Figure 13 As shown.

[0158] Comparing the experimental results of the ROC and PR curves, it can be seen that the improved Tabnet model achieved ideal classification results for all types of lithology. For a few sandstone samples, the identification accuracy was significantly improved after data equalization using Bsmote. The model could completely predict sandstone samples with an AUC value of 1. Furthermore, the AUC values ​​for all types of lithology were above 0.98, indicating that the model performed excellently in handling the lithology identification problem in Block M. This also reflects a relatively ideal balance between different categories of data, high feature effectiveness, and high model accuracy and stability, thus effectively distinguishing various types of lithology from other types.

[0159] Furthermore, in lithology identification tasks, besides focusing on model accuracy and stability, the improved Tabnet model places greater emphasis on interpretability compared to other deep learning models. This model can visualize the global importance of different characteristics across different samples based on the comprehensive decision results of each execution step. By combining the global importance of different features across all samples, it helps researchers understand which features are most critical for distinguishing different rock types, thus guiding subsequent data processing and feature engineering. Figure 14 , Figure 15 These are feature importance maps for all samples and feature importance heatmaps for the top 100 samples, respectively. Figure 15In the diagram, the vertical axis represents the number of samples, and the horizontal axis represents the feature length, with 5 units representing one feature. Figure 14 In the middle, Depth, GR, AC, CNL, DEN, RT, RXO, SP, CALC, and CALI correspond to respectively Figure 15 Features 1-10.

[0160] Depend on Figure 14 , Figure 15 It can be seen that in the rock identification dataset of this embodiment, Depth, AC, CNL, and SP have high feature importance and global importance, that is, these parameters are important parameters for distinguishing the lithological categories of block M.

[0161] 2.5.4 Analysis of Verification Results

[0162] To verify the practical application effect of the improved Tabnet model, the model was applied to the remaining verification wells in the work area. Figure 16 The results of applying the model to a section of well A are shown. Well A contains 921 lithological data points between 3088m and 3203m. 892 lithological points were correctly identified, while 49 were incorrectly identified. The overall accuracy rate for this section is 0.943.

[0163] Figure 17 The results of the improved Tabnet model for identifying lithological data points in well B (3177m-3688m) are presented. A total of 3936 lithological data points were identified, with 3795 correctly identified and 141 incorrectly identified, achieving an accuracy of 0.964. This demonstrates that the improved Tabnet model exhibits good predictive accuracy and stability in the validation well section.

[0164] As shown above, the classification model built on Tabnet exhibits the best performance in lithology identification tasks compared to various machine learning algorithms (SVM, RF, MLP, KNN, S-Bilstm-cnn+attention). Concatenating the Transformer coder module with the attention converter module in Tabnet demonstrates that the improved Tabnet model can further enhance lithology prediction, raising the lithology prediction accuracy to 94%. Furthermore, when applied to verification wells in the same work area, the lithology identification accuracy reached 91%, indicating that the model has good generalization ability.

[0165] The improved Tabnet model provided in this application can achieve limited sparsity of global output features. Key well logging parameter features are automatically extracted from the lithology identification sample dataset for model training. Furthermore, it can select different features for different samples, demonstrating a significant advantage in the interpretability of deep learning models. In the future, the model structure should be further optimized to maintain low computational complexity and high operational performance during training. This model can be applied to different work areas for training based on lithology samples to enhance its generalization and robustness.

[0166] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lithology identification method based on an improved Tabnet model, characterized in that, Includes the following steps: Construct an improved Tabnet model: This involves adding the Transformer encoder module from the ViT structure before the attention converter module at each step of the Tabnet network to form a composite attention converter module. Train the improved Tabnet model; Lithology identification was performed using a trained, improved Tabnet model. Natural gamma, sonic transit time, neutron, density, resistivity, flushing zone resistivity, spontaneous potential, wellbore, wellbore difference, and depth are used as input parameters for the model. Large and medium gravel, small and medium gravel, boulders, mudstone, sandstone, coarse gravel, and fine gravel are used as output parameters of the model.

2. The lithology identification method based on the improved Tabnet model according to claim 1, characterized in that, Tabnet introduces regularization coefficients, which are used to control the distribution of the learned mask values ​​between 0 and 1.

3. The lithology identification method based on the improved Tabnet model according to claim 1, characterized in that, Before training, the Bsmote algorithm was used to perform data equalization on the sample dataset for lithology identification.

4. The lithology identification method based on the improved Tabnet model according to claim 3, characterized in that, After equalization, the data is normalized using Z-score.

5. The method of lithology identification based on the improved Tabnet model according to claim 1, characterized in that, The Bayesian optimization method was used to tune the hyperparameters.

6. The method of lithology identification based on the improved Tabnet model according to claim 1, characterized in that, Model training includes the following steps: Based on well logging data and well logging data of the study area, depth information, well logging parameters, and lithology label data were selected as the lithology identification dataset. The original depth and well logging data were dimensionless and normalized. Then, the normalized dataset was divided into training set and test set for model training and verification of identification accuracy. In the improved Tabnet model, hyperparameters are selected with lithology identification accuracy as the objective. Based on the various hyperparameters obtained during the optimization process, a prior dataset is obtained and a Gaussian model is fitted. After calculating the posterior probability distribution through the Gaussian model, the covariance and expected mean of the objective function of each group of hyperparameters are used as indicators to determine the hyperparameters corresponding to the maximum values ​​of the objective function. After completing multiple iterations of training, the optimal combination of hyperparameters for lithology identification is determined. The optimized hyperparameter combination is applied to the improved Tabnet model, and the lithology identification dataset is imported into the model for intelligent identification of lithology samples.

7. The lithology identification method based on the improved Tabnet model according to claim 1 or 6, characterized in that, The improved Tabnet model selects the ReLU activation function as the activation function for the hidden layer neurons.

8. The lithology identification method based on the improved Tabnet model according to claim 1 or 6, characterized in that, Batch size, learning rate, and number of iterations are optimized using cross-validation.

Citation Information

Patent Citations

  • WLAN indoor positioning method of TebNet neural network model based on deep learning

    CN112653991A

  • Lithology identification method based on stacked hybrid neural network model

    CN120046655A