Method for constructing three-dimensional metallogenic prediction balance data set based on KMEDS algorithm

Abstract

The invention provides a method for constructing a three-dimensional metallogenic prediction balance data set based on a KMEDS algorithm, and belongs to the technical field of three-dimensional metallogenic prediction, and the method comprises the following steps: S1, collecting geochemical remote data, carrying out the preprocessing, and constructing a three-dimensional attribute data set; s2, clustering all the samples in the three-dimensional attribute data set by using a K-Means clustering algorithm, dividing all the samples into different groups, and selecting the group of which the proportion of minority classes of samples meets the requirement; s3, applying an improved SMOTE oversampling algorithm to all the selected groups to generate new samples, and adding the new samples into the three-dimensional attribute data set in the step S1 to form a new data set; and S4, inputting the new data set into the LightGBM model, and carrying out three-dimensional metallogenic prediction. According to the method, the problems of mineralization prediction sample data missing and sample data imbalance in the prior art can be solved, so that the three-dimensional mineralization prediction accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional mineralization prediction, and in particular to a method for constructing a three-dimensional mineralization prediction equilibrium data set based on a KMEDS algorithm. Background Art

[0002] With the development of computer technology, the application of machine learning in three-dimensional mineralization prediction has achieved remarkable results. For example, in 2015, Carranza and Laborte used the random forest algorithm to conduct mineralization prediction for gold mines in Baguio, Philippines. In 2021, Fu Guangming et al. obtained five groups of features based on three-dimensional geological modeling and gravity and magnetic inversion, and then constructed a mineralization prediction model for the study area based on the random forest algorithm. The prediction results have important guiding significance for the exploration of characteristic mineral resources of Zhuxi tungsten mine in northeast Jiangxi.

[0003] However, in actual engineering, the datasets for 3D mineralization prediction still face the dilemma of data category imbalance. In addition, due to the difficulty of exploration, it is inevitable that there are a large number of missing values ​​in geological datasets. Using such datasets to train classifiers leads to a decline in classifier performance. At present, oversampling or undersampling algorithms are usually used to solve such problems in other fields. In the field of 3D mineralization prediction, some studies have generated negative samples by random undersampling to balance the ratio of positive and negative samples, and some studies have given higher weights to minority samples to make the classifier pay more attention to them. However, these technologies either only consider the balance of sample quantity and do not pay attention to the quality of the generated new samples, or only take the original sample data for prediction, which does not fundamentally solve the problem of data imbalance. With the deepening of the application of machine learning technology in the field of 3D mineralization prediction, the imbalanced distribution of geological dataset categories has also become a problem that needs to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to provide a method for constructing a three-dimensional mineralization prediction balanced data set based on the KMEDS algorithm, which can solve the problems of missing sample data and unbalanced sample data in the prior art for mineralization prediction, thereby improving the accuracy of three-dimensional mineralization prediction.

[0005] To achieve the above object, the present invention provides a method for constructing a three-dimensional mineralization prediction equilibrium data set based on the KMEDS algorithm, comprising the following steps:

[0006] Step S1, collect and pre-process the remote sensing data of geophysical objects to construct a three-dimensional attribute data set;

[0007] Step S2, clustering all samples in the three-dimensional attribute data set using the K-Means clustering algorithm, dividing all samples into different groups, and selecting the group whose ratio of the number of minority class samples to the total number of samples in the group is higher than n, where the value range of n is 0.3 to 1;

[0008] Step S3, applying the improved SMOTE oversampling algorithm to all the groups selected in step S2 to generate new samples, and adding the new samples to the three-dimensional attribute data set in step S1 to form a new data set;

[0009] The improved SMOTE oversampling algorithm is as follows:

[0010] Any group selected in step S2 consists of N minority class samples and W features. The minority class samples in this group are used to generate a Euclidean matrix M, where the dimension of the matrix M is represented by [N, W].

[0011] Assuming that the mean of each dimension of sample set X is m and the standard deviation is s, the standardized variable of sample set X is:

[0012]

[0013] Among them, X * is the sample set after standardization;

[0014] Calculate the distance between any two sample points in the standardized sample set in sequence to obtain a [N,N]-dimensional matrix, denoted as matrix A;

[0015] The distance calculation formula between any two sample points is as follows:

[0016]

[0017] Among them, x i ,y i represents two sample points in the sample set; i represents the i-th dimension of the sample point; ED represents the Euclidean distance between two sample points;

[0018] Calculate the probability weight P for each sample:

[0019]

[0020] Among them, ES is the sum of the elements of each row in matrix A, and the calculation formula is as follows:

[0021] ES=[ΣED 1i ,ΣED 2i ,ΣED 3i ,……,ΣED Ni ] 1×N

[0022] Among them, ΣED Ni is the sum of the Euclidean distances of the N rows of elements in matrix A;

[0023] Assuming that the total number of minority class samples to be generated is S, the number of oversampling times required for each sample is:

[0024]

[0025] Among them, T is the number of oversampling times required for each sample; S is the total number of minority class samples; U is the number of groups of minority class samples that need to be oversampled, that is, the number of groups selected after cluster analysis;

[0026] Prioritize samples in descending order of T value, and oversample each sample in the group to generate new samples;

[0027] Step S4: Input the new data set into the LightGBM model to perform three-dimensional mineralization prediction.

[0028] Preferably, in step S1, geophysical data, geochemical data and remote sensing data include basic geological data, geophysical data, geochemical data and remote sensing data.

[0029] Preferably, in step S1, the preprocessing includes grid generation, normalization processing and inverse distance space interpolation.

[0030] Based on the above-mentioned method for constructing a three-dimensional mineralization prediction balanced data set based on the KMEDS algorithm, the present invention also provides a system for balancing a three-dimensional mineralization prediction data set based on the KMEDS algorithm, including a data set construction module, a clustering module, an oversampling module and a prediction module;

[0031] Among them, the data set construction module is used to collect and preprocess the geophysical and chemical remote data to construct a three-dimensional attribute data set;

[0032] Clustering module, used to use K-means algorithm to screen groups that meet the requirements to generate new samples;

[0033] An oversampling module, used for applying an improved SMOTE oversampling algorithm to all selected groups to generate new samples, and adding the new samples to the three-dimensional attribute data set of step S1 to form a new data set;

[0034] The prediction module is used to input new data sets into the LightGBM model for three-dimensional mineralization prediction.

[0035] Therefore, the present invention adopts the above-mentioned method of constructing a three-dimensional mineralization prediction equilibrium data set based on the KMEDS algorithm, and the beneficial technical effects are as follows:

[0036] (1) The data set is grouped through a clustering algorithm, and only the group with a high proportion of minority class samples is selected, so that oversampling is performed only in the safe area, avoiding the generation of noise;

[0037] (2) By calculating the Euclidean distance between each sample point in the group and other sample points, the sampling value of the sample point is weighted to reduce the possibility of overlapping between the newly generated samples and other samples, while ensuring that the total amount of new samples generated by oversampling remains unchanged;

[0038] (3) The LightGBM model was selected for mineralization prediction, which achieved efficient processing of three-dimensional attribute data sets, reduced memory consumption and improved training speed.

[0039] In summary, the present invention is not only low in complexity and easy to implement, but also generates samples based on cluster distribution, which well preserves the statistical characteristics of most geological dimensions in the original data, focuses on solving the imbalance between categories and the imbalance within categories, improves the quality of geological three-dimensional attribute data sets, overcomes the uncertainty of model prediction results, and improves training efficiency and model accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flow chart of a method for constructing a three-dimensional mineralization prediction equilibrium data set based on the KMEDS algorithm of the present invention;

[0041] Figure 2 This is a structural block diagram of a system for balancing three-dimensional mineralization prediction data sets based on the KMEDS algorithm. DETAILED DESCRIPTION

[0042] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0043] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0044] Embodiment 1

[0045] like Figure 1 As shown, the embodiment of the present invention discloses a method for constructing a three-dimensional mineralization prediction equilibrium data set based on the KMEDS algorithm, which specifically includes the following operations:

[0046] Step S1, collect and pre-process the remote sensing data of geophysical objects to construct a three-dimensional attribute data set;

[0047] Collect multi-source attribute data such as geological basic data, geophysics, geochemistry, remote sensing, etc., and construct a three-dimensional attribute data set through operations such as gridding, normalization, and inverse distance space interpolation. In the field of three-dimensional mineralization prediction, positive samples mean the presence of minerals, and negative samples mean the absence of minerals.

[0048] Step S2, clustering all samples in the three-dimensional attribute data set using the K-Means clustering algorithm, dividing all samples into different groups, and selecting the group whose ratio of the number of minority class samples to the total number of samples in the group is higher than n, where the value range of n is 0.3 to 1;

[0049] The value of n can be adjusted according to the sample ratio, where the minority sample refers to the category with fewer samples in the positive and negative samples. For example, if the ratio of positive and negative samples is 100:1, the negative sample is the minority sample, and vice versa, the positive sample is the minority sample;

[0050] Step S3, applying the improved SMOTE oversampling algorithm to all the groups selected in step S2 to generate new samples, and adding the new samples to the three-dimensional attribute data set in step S1 to form a new data set;

[0051] The improved SMOTE oversampling algorithm is as follows:

[0052] Any group selected in step S2 consists of N minority class samples and W features. The group of minority class samples is used to generate a Euclidean matrix M, and the dimension of the matrix M is represented by [N, W].

[0053] Assuming that the mean of each dimension of sample set X is m and the standard deviation is s, the standardized variable of sample set X is:

[0054]

[0055] Among them, X * is the sample set after standardization;

[0056] Calculate the distance between any two sample points in the standardized sample set in sequence to obtain a [N,N]-dimensional matrix, denoted as matrix A;

[0057] The distance calculation formula between any two sample points is as follows:

[0058]

[0059] Among them, x i ,y i represents two sample points in the sample set; i represents the i-th dimension of the sample point; ED represents the Euclidean distance between two sample points;

[0060] Calculate the probability weight P of each sample, the range of P is (0, 1), which is used to determine the importance of each sample:

[0061]

[0062] Among them, ES is the sum of the elements of each row in matrix A, and the calculation formula is as follows:

[0063] ES=[∑ED 1i ,∑ED 2i ,∑ED 3i ,………,ΣED Ni ] 1×N

[0064] Among them, ΣED Ni is the sum of the Euclidean distances of the N rows of elements in matrix A;

[0065] Assuming that the total number of minority class samples to be generated is S, the number of oversampling times required for each sample is:

[0066]

[0067] Among them, T is the number of oversampling times required for each sample; S is the total number of minority class samples; U is the number of groups of minority class samples that need to be oversampled, that is, the number of groups selected after cluster analysis;

[0068] Prioritize samples in descending order of T value, and oversample each sample in the group to generate new samples;

[0069] Step S4: Input the new data set into the LightGBM model to perform three-dimensional mineralization prediction.

[0070] The present invention will be further described below by means of specific examples.

[0071] Taking a certain area as the research area, based on the collected geological basic data, geophysics, geochemistry, remote sensing and other multi-source attribute data, negative samples are generated to carry out three-dimensional mineralization prediction.

[0072] The collected raw data are preprocessed, including inverse distance space interpolation, dividing quantitative data and categorical data, normalizing quantitative data, and performing one-hot encoding on categorical data.

[0073] The quantitative data and the categorical data are divided as shown in Table 1.

[0074] Table 1 Data division

[0075] Quantitative data Nominal type data F Layer HP Name I Fault P Density Magnetism Resistivity

[0076] In Table 1, F, HP, I, and P are geochemical data, which are fluorine, acid insoluble matter, iodine, and phosphorus, respectively; Density, Magnetism, and Resistivity are geophysical data, which are density, magnetic susceptibility, and resistivity, respectively; Layer is stratum data, Fault is fault data, and Name is lithology data.

[0077] The quantitative data were normalized and are shown in Table 2.

[0078] Table 2 Quantitative data

[0079] F HP I P Density Magnetism Resistivity 2.36 0.0 0.003 28.487499 2.78 24 2018 2.254 0.0 0.00308 15.1237 2.68 34 3214 2.17571 0.0 0.0028 24.1087 2.78 24 2018 2.0375 0.0 0.00265 23.79001 2.68 34 3214 1.935 0.0 0.0021 8.40875 2.02 56 643 1.47333 0.0 2.00785 19.205 2.78 24 2018

[0080] The classified data is processed by one-hot encoding, as shown in Table 3.

[0081] Table 3 Classification data

[0082] Layer_K Layer_Nh1c Layer_Nh2f ··· Fault_F81 Fault_F82 Fault_F9 0 0 0 ··· 0 0 0 0 0 0 ··· 0 0 0 0 0 1 ··· 0 0 0 0 0 1 ··· 0 0 0 0 0 1 ··· 0 0 0

[0083] The dataset is constructed based on known labeled positive and negative samples. The dataset contains 30,063 positive samples and 5,450 negative samples.

[0084] First, the K-means clustering algorithm is applied to the samples to divide the samples into K groups. All groups with a minority class sample ratio greater than or equal to n are selected. The setting of the K value affects the effect of the K-means clustering algorithm. Secondly, the sample ratio threshold n determines the sampled group. In this embodiment, the sample threshold is set to 2 times the positive and negative sample ratio, and the K value is set to an integer between [2,6] to explore the sampling effect of the KMED-SMOTE algorithm on the unbalanced mineralization data set. The improved SMOTE oversampling algorithm is applied to these selected groups, and the generated new samples are added to the original data set to form a new data set. The positive and negative sample ratio of the new data set is 1:1. The data set is randomly divided into a training set and a test set at a ratio of 8:2, and a LightGBM model is constructed (parameters objective = binary, max_depth = -1, boosting_type = goss, learning_rate = 0.03). The model is trained and predicted, and the mineralization prediction accuracy is compared with and without the data set constructed using the embodiment of the present invention, as shown in Table 4.

[0085] Table 4 Comparison of experimental results

[0086] Oversampling Algorithm Prediction accuracy Not adopted 0.7845 Conventional Smote 0.8237 KMED-SMOTE 0.8563

[0087] Embodiment 2

[0088] like Figure 2 As shown, the present invention also provides a system for balancing three-dimensional mineralization prediction data sets based on the KMEDS algorithm, including a data set construction module, a clustering module, an oversampling module and a prediction module;

[0089] Among them, the data set construction module is used to collect and preprocess the geophysical and chemical remote data to construct a three-dimensional attribute data set;

[0090] Clustering module, used to use K-means algorithm to screen groups that meet the requirements to generate new samples;

[0091] An oversampling module, used for applying an improved SMOTE oversampling algorithm to all selected groups to generate new samples, and adding the new samples to the three-dimensional attribute data set of step S1 to form a new data set;

[0092] The prediction module is used to input new data sets into the LightGBM model for three-dimensional mineralization prediction.

[0093] The system described in this embodiment can implement the three-dimensional mineralization prediction method based on the LightGBM model mentioned above, which will not be elaborated here.

[0094] It is worth noting that the contents not elaborated in detail in the present invention are all prior art and are well known to those skilled in the art.

[0095] Therefore, the present invention adopts the above-mentioned method of constructing a three-dimensional mineralization prediction balanced data set based on the KMEDS algorithm, which can solve the problems of missing sample data and unbalanced sample data in the prior art for mineralization prediction, thereby improving the accuracy of three-dimensional mineralization prediction.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for constructing a three-dimensional mineralization prediction equilibrium data set based on the KMEDS algorithm, characterized in that: The following steps are involved: Step S1, collect and pre-process the remote sensing data of geophysical objects to construct a three-dimensional attribute data set; Step S2, clustering all samples in the three-dimensional attribute data set using the K-Means clustering algorithm, dividing all samples into different groups, and selecting the group whose ratio of the number of minority class samples to the total number of samples in the group is higher than n, where the value range of n is 0.3 to 1; Step S3, applying the improved SMOTE oversampling algorithm to all the groups selected in step S2 to generate new samples, and adding the new samples to the three-dimensional attribute data set in step S1 to form a new data set; The improved SMOTE oversampling algorithm is as follows: Any group selected in step S2 consists of N minority class samples and W features. The minority class samples in this group are used to generate a Euclidean matrix M, where the dimension of the matrix M is represented by [N, W]. Assuming that the mean of each dimension of sample set X is m and the standard deviation is s, the standardized variable of sample set X is: Among them, X * is the sample set after standardization; Calculate the distance between any two sample points in the standardized sample set in sequence to obtain a [N,N]-dimensional matrix, denoted as matrix A; The distance calculation formula between any two sample points is as follows: Among them, x i ,y i represents two sample points in the sample set; i represents the i-th dimension of the sample point; ED represents the Euclidean distance between two sample points; Calculate the probability weight P for each sample: Among them, ES is the sum of the elements in each row of matrix A, and the calculation formula is as follows: Among them, ΣED Ni is the sum of the Euclidean distances of the N rows of elements in matrix A; Assuming that the total number of minority class samples to be generated is S, the number of oversampling times required for each sample is: Where T is the number of oversampling times required for each sample; S is the total number of minority class samples; U is the number of groups of minority class samples that need to be oversampled; Prioritize samples in descending order of T value, and oversample each sample in the group to generate new samples; Step S4: Input the new data set into the LightGBM model to perform three-dimensional mineralization prediction.

2. A method for constructing a three-dimensional mineralization prediction equilibrium data set based on the KMEDS algorithm according to claim 1, characterized in that: In step S1, geophysical data, geochemical data and remote sensing data include basic geological data, geophysical data, geochemical data and remote sensing data.

3. A method for constructing a three-dimensional mineralization prediction equilibrium data set based on the KMEDS algorithm according to claim 2, characterized in that: In step S1, preprocessing includes grid generation, normalization processing and inverse distance space interpolation.

Images