Metallization prediction method based on BO-PCA-XGBoost model

Abstract

The invention discloses a metallogenic prediction method based on a BO-PCA-XGBoost model, and belongs to the technical field of metallogenic prediction. The invention discloses a metallogenic prediction method based on a BO-PCA-XGBoost model, and the method comprises the following steps: S1, collecting geochemical and remote comprehensive data of a non-ore deposit and a known ore deposit, and processing the missing value of a metallogenic prediction variable; s2, an optimized PCA (principal component analysis) method is adopted to extract mineral sample main features, and feature dimensions are reduced and noise is eliminated while main information of data is maintained; s3, an initial XGBoost model is constructed, a hyper-parameter search space is defined, a target optimization function, the number of iterations and an initial point are set, and a Bayesian optimization algorithm is initialized; s4, automatically searching an optimal hyper-parameter combination of the XGBoost model by using a Bayesian optimization algorithm; and S5, performing final model training on the training set by using the XGBoost model after BO optimization. By the adoption of the metallogenic prediction method based on the BO-PCA-XGBoost model, high-dimensional data can be better processed, the prediction effect of the model is improved, and accurate prediction of the metallogenic potential area is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of mineralization prediction, and in particular to a mineralization prediction method based on a BO-PCA-XGBoost model. Background Art

[0002] Mineral resource exploration and development has always been a key research direction in the field of geoscience. With the continuous development of minerals, people have to continue to explore deep underground, which also means that the difficulty of exploration is increasing.

[0003] In the early days, traditional mineral prediction methods mainly relied on the experience and judgment of geological experts, which had problems such as strong subjectivity and low prediction accuracy. With the continuous advancement of exploration technology, the available geological data has increased exponentially, providing a large amount of raw data for mineral prediction. This makes it very difficult to mine deep information from these data using traditional methods.

[0004] In recent years, artificial intelligence technologies such as machine learning have been widely used in the field of geology, and can use massive geological data to make objective and efficient mineralization predictions. Most of the existing mineralization prediction models based on machine learning rely only on geological data itself, and most of them only focus on the analysis of a single data source, lacking the full utilization of multi-source geological knowledge. Therefore, how to better capture the potential structure of the data and improve model performance is also the key to research. Summary of the invention

[0005] The purpose of the present invention is to provide a mineralization prediction method based on the BO-PCA-XGBoost model, which selects features that have an important impact on mineralization prediction from multiple features through the principal component analysis method, removes redundancy and noise, and improves the calculation efficiency and prediction performance of the model. By introducing the Bayesian optimization algorithm, the optimal hyperparameter combination can be automatically searched and adjusted, manual intervention can be reduced, and the training efficiency and performance of the prediction model can be effectively improved.

[0006] To achieve the above object, the present invention provides a mineralization prediction method based on the BO-PCA-XGBoost model, comprising the following steps:

[0007] S1. Collect geophysical, chemical and remote sensing integrated data of non-ore points and known ore points as raw data, clean the raw data and process the missing values ​​of mineralization prediction variables;

[0008] S2. Using the optimized principal component analysis (PCA) method, after standardizing the eigenvalues ​​in the total sample data set, the covariance between the features is calculated, the eigenvalues ​​of the covariance matrix are decomposed, and the Bayesian optimization algorithm is combined to automatically determine the optimal number of principal components, select the main eigenvectors according to the contribution rate of the data variance, and project the original data onto the selected principal component direction;

[0009] S3. Build the XGBoost model, define the hyperparameter search space, set the target optimization function, number of iterations and initial point, and initialize the Bayesian optimization algorithm.

[0010] S4, using the Bayesian optimization algorithm, assisted by cross-validation, to automatically search for the best hyperparameter combination for the XGBoost model;

[0011] S5. Use the BO-optimized XGBoost model to perform final model training on the training set; use the final model to predict the unlabeled data.

[0012] Preferably, in S1, the mineralization prediction variables are obtained by extracting drilling data and calculating inverse distance interpolation; A, B, C, ... are mineralization prediction variables, S = {A, B, C, ...}, S is a set of mineralization prediction variables; Y is a label set of the data taken, and the total sample data set D = S∪Y;

[0013] For missing values ​​in the total sample data set, the mean imputation method is used to handle them.

[0014] Preferably, in S2, the eigenvalues ​​are standardized by calculating each eigenvalue x j The mean value u j and standard deviation σ j , for each feature x j Perform standardization, the standardization formula is:

[0015]

[0016] Preferably, in S2, the covariance matrix C of the standardized data set is:

[0017]

[0018] Among them, x' i represents the standardized value of the i-th sample; u' is the mean vector of the feature; m is the number of samples.

[0019] Preferably, in S2, the eigenvalue of the covariance matrix C is decomposed to obtain the eigenvalue λ j and the corresponding eigenvector v j :

[0020] Cv j =λ j v j (3)

[0021] Eigenvalue λ j Indicates that the data corresponds to the eigenvector v j The size of the variance in direction;

[0022] Arrange all eigenvalues ​​in descending order and calculate the cumulative variance contribution rate:

[0023]

[0024] Among them, k represents the number of principal components selected, n represents the total number of features, and the k value that reaches the cumulative contribution rate of 95% is calculated.

[0025] Preferably, in S2, the target optimization function is set, the Bayesian optimizer is initialized, the initial XGBoost model is used, and 5-fold cross validation is introduced to assist in Bayesian optimization. The optimal number of principal components k' obtained through Bayesian optimization is used to select principal component features; the feature vectors v1, v2, ..., v corresponding to the selected k' principal components are used. k' Construct the projection matrix P and perform dimensionality reduction transformation on the standardized data:

[0026] X 降维 =X'P (5)

[0027] Among them, X 降维 is the dataset after dimensionality reduction, and X' is the dataset after standardization.

[0028] Preferably, S3 specifically includes: defining the value range of the hyperparameters that need to be optimized in the XGBoost model, the target optimization function is the target that needs to be minimized or maximized in the Bayesian optimization process, and introducing 5-fold cross validation to obtain stable performance indicators.

[0029] Preferably, the optimization hyperparameters are learning rate, maximum depth, number of basic models, splitting conditions of the control tree, subsample ratio, and column sampling ratio.

[0030] Preferably, in S4, an initial hyperparameter combination is selected and the corresponding model performance index is evaluated. Based on the observed performance data set, a probability model is constructed using a Gaussian process to approximate the objective function; the next hyperparameter combination to be evaluated is determined by optimizing the acquisition function; the model performance corresponding to the new hyperparameter combination is evaluated by 5-fold cross validation and added to the observed data set; the Gaussian process model is updated; and the above steps are iterated until the maximum number of iterations.

[0031] Preferably, S5 specifically includes: using the best hyperparameter configuration after BO optimization to perform final training in the XGBoost model to obtain a final model; and using the final model to predict unlabeled data.

[0032] The advantages and positive effects of the mineralization prediction method based on the BO-PCA-XGBoost model described in the present invention are:

[0033] 1. The present invention uses an improved principal component analysis method to effectively identify and extract features that play a key role in mineralization prediction, remove redundant information and noise data, and significantly improve the computational efficiency and prediction performance of the model when dealing with complex and high-dimensional data.

[0034] 2. By introducing the Bayesian optimization algorithm, it can automatically search and adjust the optimal hyperparameter combination, significantly reduce manual intervention and improve training efficiency.

[0035] 3. Through cross-validation and system parameter optimization, the present invention improves the generalization ability of the model on unknown data, ensures the stability and reliability of the model in practical applications, and meets the prediction needs in different environments.

[0036] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The present invention is a flowchart of a mineralization prediction method based on the BO-PCA-XGBoost model in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0039] Example

[0040] In this embodiment, the manganese ore resources in the northeastern Guizhou Province are studied, and mineralization prediction is performed on the collected multi-source data such as geophysical, chemical, and remote sensing.

[0041] like Figure 1 A mineralization prediction method based on the BO-PCA-XGBoost model includes the following steps:

[0042] S1. Collect 640 sets of geophysical, chemical and remote sensing data of non-mineral points and 646 sets of known mineral points as raw data, clean the raw data and process the missing values ​​of mineralization prediction variables.

[0043] The mineralization prediction variables are obtained by extracting the drilling data and calculating the inverse distance interpolation. If A, B, C, ... are mineralization prediction variables and S is the set of mineralization prediction variables, then S = {A, B, C, ...}. Y is the label set of the data taken, and the total sample data set D = S∪Y.

[0044] For missing values ​​in the total sample data set, the mean imputation method is used to handle them.

[0045] S2. Using the optimized principal component analysis (PCA) method, after standardizing the eigenvalues ​​in the total sample data set, the covariance between the features is calculated, the eigenvalues ​​of the covariance matrix are decomposed, and combined with the Bayesian optimization algorithm, the optimal number of principal components is automatically determined. The main eigenvectors are selected according to the contribution rate of the data variance, and the original data is projected onto the selected principal component direction.

[0046] For the mining point sample data, assuming there are m samples (actually 1286), each sample contains n eigenvalues ​​(actually 23), then the matrix can be expressed as:

[0047]

[0048] x ij represents the j-th eigenvalue of the i-th sample.

[0049] (1) Standardization

[0050] Normalizing the eigenvalues ​​is done by calculating for each feature x j The mean value u j and standard deviation σ j , for each eigenvalue x ij After normalization, a standardized matrix X' is finally obtained.

[0051] u j Represents the mean of the j-th column (i.e., the j-th feature):

[0052]

[0053] Among them, σ j is the standard deviation of the j-th column feature:

[0054]

[0055] The normalization formula is:

[0056]

[0057] (2) Calculate the covariance matrix C of the standardized data set

[0058] The covariance between features is calculated by calculating the covariance matrix C of the standardized data set to understand the correlation between different features and determine the direction to be retained during dimensionality reduction. The formula is:

[0059]

[0060] Among them, x' i represents the standardized value of the i-th sample; u' is the mean vector of the feature; m is the number of samples.

[0061] (3) Decomposition of the eigenvalues ​​of the covariance matrix

[0062] Eigenvalue λ j and the corresponding eigenvector v j The following conditions are met:

[0063] Cv j =λ j v j (6)

[0064] Eigenvalue λ j Indicates that the data corresponds to the eigenvector v j The variance in the direction. By solving this formula, we can get the eigenvalue λ j and the corresponding eigenvector v j A larger eigenvalue means that the data has a larger change in that direction, so that direction is more important, so all eigenvalues ​​are sorted in descending order.

[0065] Calculate the cumulative variance contribution rate (i.e. the proportion of data variance explained by the principal component), the formula is:

[0066]

[0067] Among them, k represents the number of principal components selected, and n represents the total number of features. The k value that can reach 95% of the cumulative contribution rate is calculated to ensure that the data retains sufficient information after dimensionality reduction. Finally, the k value is 10.

[0068] (4) Combined with the Bayesian optimization algorithm, the optimal number of principal components k' is automatically determined

[0069] Define the range of the optimal number of principal components k' in the PCA method [10,23]. Set the target optimization function to accuracy and initialize the Bayesian optimizer. Use the initial XGBoost model, set the number of iterations, and introduce 5-fold cross validation for assistance. After Bayesian optimization, use the best value k' of 10 to select the first k' principal component features.

[0070] (5) Project the original data onto the selected principal component direction

[0071] Use the eigenvectors v1, v2, …, v corresponding to the selected k' principal components k' Construct the projection matrix P, and then perform dimensionality reduction transformation on the standardized data:

[0072] X 降维 =X'P (8)

[0073] Among them, X 降维 is the dataset after dimensionality reduction, and X' is the dataset after standardization.

[0074] (6) Performance testing

[0075] Comparing the original data and the modified PCA data in the initial XGBoost model

[0076] Table 1 Comparison of original data and improved PCA data

[0077] Models Accuracy Precision Recall F1score Kappa AUC XGB 0.888 0.868 0.93 0.89 0.773 0.94 PCA-XGB 0.896 0.875 0.94 0.906 0.789 0.947

[0078] According to the data, the model performance has been improved after optimization using the improved PCA.

[0079] S3. Define the hyperparameter search space and set the target optimization function, number of iterations, and initial point.

[0080] (1) Define the hyperparameter search space

[0081] Define the range of hyperparameters that need to be optimized in the XGBoost model. Appropriate hyperparameter values ​​can achieve the best performance of the model, reduce the complexity of the tree, and prevent overfitting.

[0082] Table 2 Hyperparameter ranges

[0083] parameter Parameter name Value range learning_rate Learning Rate [0.01,0.3] max_depth Maximum Depth [3,10] n_estimators Number of base models [50,200] gamma Controlling the splitting conditions of the tree [0,0.5] subsample Subsample proportion [0.6,1.0] colsample_bytree Column sampling ratio [0.6,1.0]

[0084] (2) Initialize Bayesian optimization.

[0085] Set the target optimization function to accuracy as the optimization target. Introduce 5-fold cross validation to obtain stable performance indicators. Based on experience, set the initial point or design the acquisition function to further improve the optimization performance. Set the number of iterations to 16. Select the Gaussian process model to fit the performance of the target function.

[0086] S4. Automatically search for the optimal hyperparameter combination of the XGBoost model through the Bayesian optimization algorithm. The specific operations are:

[0087] (1) Select several initial hyperparameter combinations and evaluate the corresponding model performance indicators. Based on the observed performance data, a probabilistic model is constructed using Gaussian processes to approximate the objective function (i.e., model performance indicator).

[0088] (2) The next hyperparameter combination to be evaluated is determined by optimizing the acquisition function. The acquisition function uses Expected Improvement (EI). The EI function attempts to find a hyperparameter combination that is both likely to bring improvements and has great potential. It will give priority to points that are expected to improve the current optimal solution, while also considering exploring potential points in unknown areas.

[0089] (3) Evaluate the model performance corresponding to the new hyperparameter combination through 5-fold cross validation and add it to the observed data set. Then update the Gaussian process model.

[0090] (4) Iterate the above steps until the maximum number of iterations is reached.

[0091] The following are the hyperparameters that the model optimizes in each iteration of this example:

[0092] Table 3 Bayesian optimization process of PCA-XGB model

[0093] Step acc max_depth learning_rate colsample_bytree gamma subsample n_estimators 1 0.897 5 0.20 0.76 0.363 0.76 151 2 0.901 10 0.02 0.93 0.441 0.62 180 3 0.89 6 0.01 0.77 0.459 0.78 78 4 0.905 9 0.07 0.92 0.085 0.63 128 5 0.906 8 0.05 0.91 0.219 0.88 186 6 0.899 4 0.01 0.89 0.469 0.75 170 7 0.904 9 0.03 0.84 0.387 0.83 139 8 0.905 9 0.05 0.81 0.460 0.62 99 9 0.912 6 0.19 0.98 0.350 0.68 107 10 0.901 5 0.12 0.60 0.408 0.71 137 11 0.902 4 0.29 1 0.076 0.76 141 12 0.910 7 0.09 1 0.081 0.6 198 13 0.903 10 0.28 0.60 0.377 0.87 116 14 0.895 3 0.03 0.60 0.309 0.95 70 15 0.895 7 0.30 1 0.210 0.85 168 16 0.901 3 0.01 0.60 0.226 0.97 200 Best 6 0.19 0.98 0.22 0.68 107

[0094] S5. Use the best hyperparameters after BO optimization and configure them into the XGBoost model for final training to obtain the final model. Use the RF model for comparison and use the Bayesian optimization algorithm to adjust the parameters of the RF model. Use accuracy, precision, recall, F1 score, Kappa coefficient and AUC value to test the performance of this method.

[0095] The following is a comparison of the performance indicators of the four models:

[0096] Table 4 Performance comparison of four methods

[0097] Models Accuracy Precision Recall F1score Kappa AUC RF 0.880 0.86 0.91 0.89 0.757 0.933 BO-RF 0.900 0.87 0.955 0.911 0.797 0.95 PCA-XGB 0.896 0.875 0.94 0.906 0.789 0.947 BO-PCA-XGB 0.924 0.897 0.97 0.932 0.846 0.961

[0098] According to the data, the BO-PCA-XGBoost model has certain improvements in various indicators, which also shows that the mineralization prediction method based on the BO-PCA-XGBoost model has great potential in mineralization prediction invention.

[0099] Therefore, the mineralization prediction method based on the BO-PCA-XGBoost model described in the present invention is adopted, and the features that have an important impact on mineralization prediction are selected from multiple features through the principal component analysis method, redundancy and noise are removed, and the calculation efficiency and prediction performance of the model are improved. By introducing the Bayesian optimization algorithm, the optimal hyperparameter combination can be automatically searched and adjusted, manual intervention is reduced, and the training efficiency and performance of the prediction model are effectively improved.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A mineralization prediction method based on BO-PCA-XGBoost model, characterized in that: The following steps are involved: S1. Collect geophysical, chemical and remote sensing integrated data of non-ore points and known ore points as raw data, clean the raw data and process the missing values ​​of mineralization prediction variables; S2. Using the optimized principal component analysis (PCA) method, after standardizing the eigenvalues ​​in the total sample data set, the covariance between the features is calculated, the eigenvalues ​​of the covariance matrix are decomposed, and the Bayesian optimization algorithm is combined to automatically determine the optimal number of principal components, select the main eigenvectors according to the contribution rate of the data variance, and project the original data onto the selected principal component direction; S3. Build the XGBoost model, define the hyperparameter search space, set the target optimization function, number of iterations and initial point, and initialize the Bayesian optimization algorithm. S4, using the Bayesian optimization algorithm, assisted by cross-validation, to automatically search for the best hyperparameter combination for the XGBoost model; S5. Use the BO-optimized XGBoost model to perform final model training on the training set; use the final model to predict the unlabeled data.

2. The mineralization prediction method based on the BO-PCA-XGBoost model according to claim 1, characterized in that: In S1, the mineralization prediction variables are obtained by extracting drilling data and calculating inverse distance interpolation; A, B, C, ... are mineralization prediction variables, S = {A, B, C, ...}, S is the set of mineralization prediction variables; Y is the label set of the data taken, and the total sample data set D = S∪Y; For missing values ​​in the total sample data set, the mean imputation method is used to handle them.

3. The mineralization prediction method based on the BO-PCA-XGBoost model according to claim 2, characterized in that: In S2, the eigenvalues ​​are normalized by calculating for each feature x j The mean value u j and standard deviation σ j , for each feature x j Perform standardization, the standardization formula is:

4. The mineralization prediction method based on the BO-PCA-XGBoost model according to claim 3, characterized in that: In S2, the covariance matrix C of the standardized data set is: Among them, x' i represents the standardized value of the i-th sample; u' is the mean vector of the feature; m is the number of samples.

5. The mineralization prediction method based on the BO-PCA-XGBoost model according to claim 4, characterized in that: In S2, the eigenvalue of the covariance matrix C is decomposed to obtain the eigenvalue λ j and the corresponding eigenvector v j : Cv j =λ j v j Eigenvalue λ j Indicates that the data corresponds to the eigenvector v j The size of the variance in direction; Arrange all eigenvalues ​​in descending order and calculate the cumulative variance contribution rate: Among them, k represents the number of principal components selected, n represents the total number of features, and the k value that reaches the cumulative contribution rate of 95% is calculated.

6. The mineralization prediction method based on the BO-PCA-XGBoost model according to claim 5, characterized in that: In S2, the target optimization function is set, the Bayesian optimizer is initialized, the initial XGBoost model is used, and 5-fold cross validation is introduced to assist in Bayesian optimization; the optimal number of principal components k' obtained by Bayesian optimization is used to select principal component features; the feature vectors v1, v2, ..., v corresponding to the selected k' principal components are used k' Construct the projection matrix P and perform dimensionality reduction transformation on the standardized data: X 降维 =X’P Among them, X 降维 is the dataset after dimensionality reduction, and X' is the dataset after standardization.

7. The mineralization prediction method based on the BO-PCA-XGBoost model according to claim 6, characterized in that: The S3 is specifically as follows: defining the value range of the hyperparameters that need to be optimized in the XGBoost model, the target optimization function is the target that needs to be minimized or maximized in the Bayesian optimization process, and introducing 5-fold cross validation to obtain stable performance indicators.

8. The mineralization prediction method based on the BO-PCA-XGBoost model according to claim 7, characterized in that: The optimization hyperparameters are learning rate, maximum depth, number of basic models, splitting conditions of control trees, subsample ratio, and column sampling ratio.

9. The mineralization prediction method based on the BO-PCA-XGBoost model according to claim 8, characterized in that: In S4, an initial hyperparameter combination is selected and the corresponding model performance index is evaluated. Based on the observed performance data set, a probability model is constructed using a Gaussian process to approximate the objective function; the next hyperparameter combination to be evaluated is determined by optimizing the acquisition function; the model performance corresponding to the new hyperparameter combination is evaluated by 5-fold cross validation and added to the observed data set; the Gaussian process model is updated; and the above steps are iterated until the maximum number of iterations.

10. The mineralization prediction method based on the BO-PCA-XGBoost model according to claim 9, characterized in that: The S5 is specifically as follows: using the best hyperparameters after BO optimization to configure the XGBoost model for final training to obtain the final model; and using the final model to predict the unlabeled data.

Images