Method for predicting toc based on kan neural network

Abstract

The application discloses a method for predicting TOC based on a KAN neural network, and belongs to the technical field of computer models, and comprises the following steps: determining M influence factors related to the TOC content; performing correlation analysis or feature importance analysis on the influence factors to find key features; constructing a data set D based on the key features; constructing an improved KAN neural network, including introducing a gating coefficient and a second-order difference constraint; training the improved KAN neural network with the data set D to obtain a first prediction model; pruning and setting a symbol function to obtain a second prediction model; and training the second prediction model again to obtain a TOC prediction model and a fitting formula of a TOC prediction value, which are used for TOC prediction of a to-be-predicted sampling point. The application improves the accuracy of TOC prediction through influence factor screening, an innovative gating mechanism, an optimized loss function and a training strategy, and has the characteristics of light weight, high precision and strong interpretability, and is suitable for embedded devices or real-time station applications.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of shale gas development, and particularly relates to a method for predicting TOC based on a KAN neural network. BACKGROUND

[0002] At present, shale gas resources have become one of the most important energies in the world. In the process of shale gas exploration and development, one of the key indicators for finding sweet spots is the total organic carbon (TOC) content. The traditional method for obtaining TOC relies on rock pyrolysis experiments of formation cores, which is not only time-consuming and laborious, but also can only obtain discrete data points and is difficult to provide continuous TOC profile information. Therefore, TOC prediction using logging data has become a widely used means in the industry.

[0003] At present, the traditional TOC evaluation methods mainly include geochemical testing and geophysical logging evaluation. However, these methods have obvious defects. 1. The disadvantages of traditional geochemical testing are: (1) a large number of drill holes and core samples are required, which is heavy and laborious; (2) the testing cost and labor cost are high; (3) the obtained TOC data is discontinuous, resulting in poor prediction accuracy. 2. The traditional geophysical logging evaluation method (such as the DeltaLogR method) has the following disadvantages: (1) less parameter information is used; (2) it cannot be used for spatio-temporal prediction in the whole work area, which also leads to low overall prediction accuracy.

[0004] In recent years, with the development of deep learning technology, neural networks have been introduced into the field of TOC prediction. For example, the prediction method based on BP neural network and support vector machine (SVM) has improved the prediction accuracy to a certain extent. However, these methods still have significant shortcomings: 1. BP neural network: although the use of resistivity, acoustic wave, natural gamma, density and other logging data improves the prediction accuracy, its generalization ability is poor, and the trained model is only suitable for a small range of areas, which cannot meet the demand of accurate and general TOC prediction. 2. Support vector machine: poor response to outliers, especially in the prediction of high TOC shale, which also cannot meet the actual application requirements.

[0005] In addition, with the development and progress of neural network technology, there are also related researches on total organic carbon prediction using neural network technology, such as prediction technology based on BP neural network and prediction technology based on support vector machine. The prediction method based on BP neural network mainly uses resistivity, acoustic wave, natural gamma, density logging to predict total organic carbon. Although this method has improved the accuracy of prediction compared with the traditional method, this method does not have universality, and the trained BP neural network can only be applied to a very small range, and does not meet the accurate and universal total organic carbon prediction demand. The prediction method based on support vector machine mainly uses resistivity, acoustic wave, natural gamma, density logging, and uses support vector machine technology to predict total organic carbon. Since this method has poor response to abnormal values, such as poor prediction effect for shale with high total organic carbon, it also does not meet the accurate and universal total organic carbon prediction demand.

[0006] In addition, the scheme of publication number: CN113837501A uses a deep graph neural network to input and analyze multiple logging curves as a multi-dimensional dynamic graph data structure with correlation, and can obtain an accurate relationship between logging curves and TOC. However, the model used in this scheme is complex, has high computational cost, redundant model parameters and is not easy to change, and is prone to overfitting, and does not have the conditions for wide application. Moreover, this scheme cannot perform spatiotemporal prediction in the whole work area, and can only ensure local prediction accuracy, and the overall prediction accuracy still needs to be improved. SUMMARY

[0007] The purpose of the present application is to provide a method for predicting TOC based on KAN neural network, which can be widely applied in the whole work area, has low computational cost, high prediction accuracy and better interpretability, and solves the above problems.

[0008] To achieve the above purpose, the technical scheme adopted by the present application is as follows: a method for predicting TOC based on KAN neural network, comprising the following steps:

[0009] S1, determining M influence factors related to TOC content and marking the first to the Mth in turn;

[0010] S2, determining N sampling points in the shale research area, obtaining the values of the M influence factors and the true value of TOC at each sampling point, and after preprocessing, performing correlation analysis or feature importance analysis on the influence factors to find the 4 influence factors most related or most important to the true value of TOC as key features;

[0011] S3, constructing a data set D;

[0012] Each sampling point constructs a sample, and all samples constitute a data set D, wherein the sample corresponding to the kth sampling point is sk , , wherein x1-x4 are values of the 1st key feature-4th key feature at the kth sampling point, y k is a true value of TOC at the kth sampling point, 1≤k≤N;

[0013] S4, constructing an improved KAN neural network;

[0014] A KAN neural network is obtained, including an input layer, a hidden layer and an output layer, the input layer including four first neurons for inputting 4 key features of a sample, the output layer being one neuron for outputting a TOC predicted value of the sample, the hidden layer having 9 neurons, a value of the qth neuron being obtained according to the following formula;

[0015] ,

[0016] ,

[0017] wherein 1≤q≤9, g p is a gating coefficient of the pth key feature x p , 1≤p≤4, is a value of x p obtained through an inner function, is an inner function of x p corresponding to the qth neuron, is a first intermediate value of x p to the qth neuron;

[0018] A TOC predicted value of the output layer neuron is obtained according to the following formula;

[0019] ,

[0020] ,

[0021] wherein g q is a gating coefficient of , is an outer function corresponding to the qth neuron, is a value of obtained through is a second intermediate value of the qth neuron, the inner function and the outer function are all activation functions, and a cubic B-spline function is adopted;

[0022] S5, training the improved KAN neural network with a data set D to convergence to obtain a first prediction model;

[0023] ​​​S6, pruning the first prediction model, setting the activation function as a sign function, to obtain a second prediction model;

[0024] S7, training the second prediction model again to convergence, to obtain a TOC prediction model, and a fitting formula of the TOC prediction value;

[0025] S8, for a to-be-predicted sampling point s in the research area, obtaining four key feature values at s, and obtaining a TOC prediction value through the TOC prediction model or outputting the TOC prediction value through the fitting formula.

[0026] As preferred: the influence factors include well depth DEPTH, caliper log CALI, acoustic travel time log AC, natural gamma GR, potassium K, uranium-free gamma KTH, photoelectric absorption index log PE, neutron porosity log PHIN, density DEN, thorium TH, uranium U, and resistivity logarithm RT_log.

[0027] As preferred: in S2, the preprocessing is that all sampling points form a first matrix A1 of N x (M+1), and then the missing values are interpolated, and each column of data is normalized to obtain a second matrix A2; the element a nm in the nth row and mth column of A1 is the value of the mth influence factor of the nth sampling point, 1≤n≤N, and the element a n(M+1) in the nth row and (M+1)th column of A1 is the TOC true value of the nth sampling point.

[0028] As preferred: in S2, the correlation analysis is mutual information analysis, and the mutual information of the last column of data in each column of data A2 is calculated, and the influence factors corresponding to the four columns of data with the largest mutual information value are selected as the key features.

[0029] The feature importance analysis is random forest evaluation, A2 is input into a random forest model to obtain the importance score of each influence factor, and the four influence factors with the highest scores are selected as the key features.

[0030] As preferred: when training the KAN neural network, the optimizer is LBFGS, and the total loss function is L total ;

[0031] L total =L RMSE +λL moothness ,

[0032] L moothness = torch.mean(torch.diff(act_params, n=2,dim=1)^2),

[0033] In the formula, L RMSE is the root mean square error loss, L moothness is the smoothness loss, and λ is the Lmoothness a weight hyper-parameter, 0 < λ ≤ 0.3, torch.mean is a torch.mean function, torch.diff is a Torch.diff function, act_params is a parameter of an activation function, n is a number of times of recursive differential calculation, and dim is a dimension of differential calculation.

[0034] As preferred: S7 obtains a fitting formula, and accuracy of the fitting formula is verified based on mean square error.

[0035] Compared with the prior art, the application has the advantages that:

[0036] (1) The application has the characteristics of high precision and strong interpretability. First, the application has better scaling characteristics: compared with a traditional MLP (multi-layer perceptron), KAN has a smaller network size and fewer parameter quantities, while following a better error scaling law, and still maintaining high precision under limited data. Second, the application has strong interpretability: through visualizing network calculation processes and activation function changes, combined with a symbolic regression technique, an analyzable mathematical formula is finally output, so that the prediction result has geological significance.

[0037] (2) The application has an innovative gating mechanism, including dynamic feature interaction and feature importance quantification. Dynamic feature interaction: the application introduces a gating coefficient in the KAN network to dynamically adjust feature flow, solving the limitations of static feature combination of a traditional KAN; feature importance quantification: the gating coefficient can explicitly show the contribution of different logging parameters (such as GR, AC, DEN, etc.) in prediction, improving the credibility of the model.

[0038] (3) The application has an optimized loss function and training strategy. First, the shape of the activation function is constrained on the basis of the original loss function L RMSE of the KAN, a second-order difference constraint is added, specifically a smoothness loss L moothness is added to avoid excessive oscillation of the activation function (cubic B-spline function), and ensure prediction stability. Second, an LBFGS optimizer is used for parameter update, LBFGS is a quasi-Newton optimization algorithm, which can efficiently handle high-dimensional nonlinear optimization problems, significantly improve training efficiency, and is particularly suitable for network structures such as KAN with complex function forms. Finally, a regularization mechanism is introduced, and the weight hyper-parameter λ is used to balance the model complexity and overfitting risk, and λ is usually set to 0.1.

[0039] (4) The application selects key features based on correlation analysis or feature importance analysis, reduces noise interference, and in data preprocessing, missing values are imputed and normalized to form a standardized input matrix, improving the robustness of the model.

[0040] (5) The application has a lightweight deployment capability. The application greatly compresses the model volume through model pruning and symbolization, and is suitable for embedded devices or real-time station applications. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a flowchart of the application;

[0042] Figure 2 is an improved KAN neural network architecture diagram;

[0043] Figure 3 is a schematic diagram of a gate unit. DETAILED DESCRIPTION

[0044] The application will be further described below in conjunction with examples and drawings.

[0045] Example 1: see Figures 1 to 3 A method for predicting TOC based on a KAN neural network, comprising the following steps:

[0046] S1, determining M influence factors related to TOC content and marking the first to the Mth in turn;

[0047] S2, determining N sampling points in the shale research area, obtaining the values of the M influence factors and the TOC true value at each sampling point, and after preprocessing, performing correlation analysis or feature importance analysis on the influence factors to find the 4 influence factors most related or most important to the TOC true value as key features;

[0048] S3, constructing a data set D;

[0049] Each sampling point constructs a sample, and all samples constitute the data set D, wherein the sample corresponding to the kth sampling point is s k , , wherein x1-x4 are the values of the 1st key feature to the 4th key feature at the kth sampling point, y k is the TOC true value of the kth sampling point, and 1≤k≤N;

[0050] S4, constructing an improved KAN neural network;

[0051] Obtaining a KAN neural network, including an input layer, a hidden layer and an output layer, the input layer including four first neurons for inputting the 4 key features of the sample, the output layer being one neuron for outputting the TOC prediction value of the sample, and the hidden layer neurons being 9, the value of the qth neuron being obtained according to the following formula;

[0052] ,

[0053] ,

[0054] wherein, 1≤q≤9, g p is the gating coefficient of the pth key feature x p , 1≤p≤4, is the value obtained by x p through the inner function , is the inner function corresponding to the qth neuron of x p , is the first intermediate value of x p to the qth neuron;

[0055] The TOC prediction value of the output layer neuron is obtained according to the following formula;

[0056] ,

[0057] ,

[0058] wherein, g q is the gating coefficient of , is the outer function corresponding to the qth neuron, is the value obtained by through , is the second intermediate value of the qth neuron, and the inner function and the outer function are all activation functions and adopt cubic B-spline functions;

[0059] S5, training the improved KAN neural network with the data set D to convergence to obtain a first prediction model;

[0060] S6, pruning the first prediction model, setting a sign function for the activation function, and obtaining a second prediction model;

[0061] S7, training the second prediction model again to convergence to obtain a TOC prediction model and a fitting formula of the TOC prediction value;

[0062] S8, obtaining four key feature values at the sampling point s to be predicted in the research area, predicting a TOC prediction value through the TOC prediction model, or outputting the TOC prediction value through the fitting formula.

[0063] In the embodiment, the influence factors include well depth DEPTH, caliper log CALI, acoustic travel time log AC, natural gamma ray GR, potassium K, uranium-free gamma ray KTH, photoelectric absorption index log PE, neutron porosity log PHIN, density DEN, thorium TH, uranium U, and resistivity logarithm RT_log.

[0064] In S2, the preprocessing is to form a first matrix A1 of N×(M+1) with all sampling points, then interpolate the missing values, normalize each column of data, and obtain a second matrix A2; the element a in the nth row and mth column of A1 is nm is the value of the mth influencing factor at the nth sampling point, 1≤n≤N, and the element a in the nth row and M+1 column is n(M+1) is the true TOC value of the nth sampling point.

[0065] In S2, the correlation analysis is a mutual information analysis, which calculates the mutual information of each column of data in A2 and the last column of data in A2, and takes the impact factors corresponding to the four columns of data with the largest mutual information values ​​as key features;

[0066] The feature importance analysis is a random forest evaluation. A2 is input into the random forest model to obtain the importance score of each influencing factor, and the four influencing factors with the highest scores are selected as key features.

[0067] When training the KAN neural network, the optimizer is LBFGS and the total loss function is L total ;

[0068] L total =L RMSE +λL moothness ,

[0069] L moothness = torch.mean(torch.diff(act_params, n=2,dim=1)^2),

[0070] Where, L RMSE is the root mean square error loss, L moothness is the smoothness loss, λ is L moothness The weight hyperparameter is 0<λ≤0.3, is the torch.mean function, is the Torch.diff function, act_params is the parameters of the activation function, n is the number of times the difference is recursively calculated, and dim is the dimension of the difference.

[0071] After S7 obtains the fitting formula, the accuracy of the fitting formula is verified based on the mean square error.

[0072] Example 2: See Figures 1 to 3 ,A method for predicting TOC based on KAN neural network comprises the following steps;

[0073] S1, identify 12 influencing factors related to TOC content, namely well depth DEPTH, caliper logging CALI, acoustic transit time logging AC, natural gamma GR, potassium K, uranium-free gamma KTH, photoelectric absorption index logging PE, neutron porosity logging PHIN, density DEN, thorium TH, uranium U, and resistivity logarithm RT_log; marked as the 1st to the 13th;

[0074] S2, same as step S2 in Example 1, performs sampling, preprocessing, correlation analysis or feature importance analysis. In this embodiment, feature importance analysis is used to obtain four key features, namely RT_log, DEN, AC, and GR;

[0075] S3, constructing a data set D, similar to step S3 in Example 1. After obtaining the data set D, it is divided into a training set and a validation set at a ratio of 8:2;

[0076] S4, constructing an improved KAN neural network, which is the same as step S4 in Example 1. For the working principle of the improved KAN neural network, see Figure 2 、 Figure 3 , Figure 3 The curve in represents the inner function, which is a cubic B-spline function, that is, , in this network;

[0077] The input layer includes four first neurons, which correspond to the input of four key features x1~x4;

[0078] The hidden layer has 9 neurons, and the values ​​of q neurons are obtained according to x1~x4 The method is: for the pth key feature x p In the existing KAN network, an inner function is used to calculate , but the present invention introduces a lightweight gating unit to provide a gating coefficient g p , obtained by the inner function and the original x p Fusion, gating coefficient g p Used to control the ratio of the two to obtain the first intermediate value ; Then according to The calculation formula is to sum the 9 first intermediate values ​​to get the value of the qth neuron ;

[0079] The output layer has one neuron. In the prior art, the values ​​of the nine neurons are calculated using the outer layer function. ~ The sum can be directly calculated, but the present invention still introduces a gating unit to provide a gating coefficient g q ,right and fusion, g q for controlling the proportion of both, thereby obtaining a first intermediate value , and ~ summed.

[0080] S5~S8, same as example 1 step S5~S8.

[0081] Example 3: Selecting real sampling data in a mountainous area in Sichuan and part of the simulation data to form an initial data set D1, wherein each initial sample in the initial data set contains a TOC real value and 12 influence factor values, and 4 key features are selected from the influence factor through random forest importance analysis, which are density DEN, natural gamma GR, resistivity logarithm RT_log, and acoustic travel time log AC. Then, only the 4 key features and the TOC real value are retained from the initial sample to obtain the final sample, and the set of these final samples is used to form the data set D2.

[0082] Selecting 5 kinds of models to conduct comparative experiments with data set D2, one of which is the present application, and the other four existing models are: random forest, support vector machine, TabNet, and TabTransformer, and selecting objective comparison indicators precision and recall for experiments. The objective comparison indicators of the experimental results of each model are shown in Table 1.

[0083] Table 1. Comparison table of multi-model experimental results

[0084] Model Precision Recall Random Forest 82.43% 76.34% Support Vector Machine 79.06% 73.57% TabNet 86.07% 80.84% TabTransformer 84.18% 78.36% The invention 90.61% 88.38%

[0085] From Table 1, regarding recall, the present application > TabNet > TabTransformer > random forest > support vector machine, it can be known that after combining the model quantity requirement, training, updating cost, test time required and hardware of the present application, and after saving in many aspects, compared with other several models, the highest recall rate can be guaranteed, and almost consistent precision rate can still be achieved. While other several methods are difficult to improve the recall rate, or in the case of guaranteeing the recall rate, more precision rate will be sacrificed.

[0086] The above only describes the preferred embodiments of the present application and does not limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for predicting TOC based on a KAN neural network, characterized by: The following steps are included: S1, determine M influencing factors related to TOC content and mark the first to the Mth in sequence; S2: N sampling points are determined within the shale research area. At each sampling point, the values ​​of M influencing factors and the true TOC value are obtained. After preprocessing, correlation analysis or feature importance analysis is performed on the influencing factors to find the four most correlated or important influencing factors with the true TOC value as key features. S3, construct dataset D; Each sampling point constructs a sample, and all samples constitute the data set D, where the sample corresponding to the k-th sampling point is s k , , where x1~x4 are the values ​​of the 1st~4th key features at the kth sampling point, and y k is the true TOC value of the k-th sampling point, 1≤k≤N; S4, construct an improved KAN neural network; Get a KAN neural network, including input layer, hidden layer and output layer. The input layer includes four first neurons, which are used to input the four key features of the sample. The output layer is 1 neuron, which is used to output the TOC prediction value of the sample. The hidden layer has 9 neurons, and the value of the qth neuron is According to the following formula: ， ， Where, 1≤q≤9, g p is the pth key feature x p The gating coefficient is 1≤p≤4, is x p Through the inner function The value obtained, is x p The inner function corresponding to the qth neuron, is x p The first intermediate value of the qth neuron; TOC prediction value of output layer neurons According to the following formula: ， ， Where g q for The gating coefficient, is the outer layer function corresponding to the qth neuron, for through The value obtained, is the second intermediate value of the qth neuron, the inner layer function and the outer layer function are both activation functions, and a cubic B-spline function is used; S5, using the dataset D to train the improved KAN neural network until convergence, and obtain the first prediction model; S6, pruning the first prediction model, setting a sign function for the activation function, and obtaining a second prediction model; S7, training the second prediction model again until convergence, obtaining a TOC prediction model and a fitting formula for TOC prediction value; S8, for the sampling point s to be predicted in the study area, obtain the four key characteristic values ​​at s, and obtain the TOC prediction value through the TOC prediction model, or output the TOC prediction value through the fitting formula.

2. The method for predicting TOC based on a KAN neural network according to claim 1, characterized in that: The influencing factors include well depth DEPTH, caliper logging CALI, acoustic transit time logging AC, natural gamma GR, potassium K, uranium-free gamma KTH, photoelectric absorption index logging PE, neutron porosity logging PHIN, density DEN, thorium TH, uranium U, and resistivity logarithm RT_log.

3. The method for predicting TOC based on a KAN neural network according to claim 1, characterized in that: In S2, the preprocessing is to form a first matrix A1 of N×(M+1) with all sampling points, then interpolate the missing values, normalize each column of data, and obtain a second matrix A2; the element a in the nth row and mth column of A1 is nm is the value of the mth influencing factor at the nth sampling point, 1≤n≤N, and the element a in the nth row and M+1 column is n(M+1) is the true TOC value of the nth sampling point.

4. The method for predicting TOC based on a KAN neural network according to claim 1, characterized in that: In S2, the correlation analysis is a mutual information analysis, which calculates the mutual information of each column of data in A2 and the last column of data in A2, and takes the impact factors corresponding to the four columns of data with the largest mutual information values ​​as key features; The feature importance analysis is a random forest evaluation. A2 is input into the random forest model to obtain the importance score of each influencing factor, and the four influencing factors with the highest scores are selected as key features.

5. The method for predicting TOC based on a KAN neural network according to claim 1, characterized in that: When training the KAN neural network, the optimizer is LBFGS and the total loss function is L total ; THE total =L RMSE +λL moothness , L moothness = torch.mean(torch.diff(act_params, n=2,dim=1)^2), Where, L RMSE is the root mean square error loss, L moothness is the smoothness loss, λ is L moothness The weight hyperparameter is 0<λ≤0.3, is the torch.mean function, is the Torch.diff function, act_params is the parameters of the activation function, n is the number of times the difference is recursively calculated, and dim is the dimension of the difference.

6. The method for predicting TOC based on a KAN neural network according to claim 1, characterized in that: After S7 obtains the fitting formula, the accuracy of the fitting formula is verified based on the mean square error.

Images