Machine learning model training method, computing device and computer readable storage medium

Abstract

The invention discloses a machine learning model training method, computing equipment and a computer readable storage medium, and the method comprises the steps: generating a merged set according to a target test sample and a training sample in a preset training set; a target subset with the minimum regularization covariance determinant in the merged set is determined, the mahalanobis distance from the target test sample to the target subset is smaller than a robust distance cut-off value, and the number of samples contained in the target subset is larger than a preset number threshold value; determining a target training set according to the target training samples with mahalanobis distances to the target subset smaller than a robust distance cutoff value in the combined set and identification results correspondingly marked by the target training samples in a preset training set; and training a machine learning model based on the target training set to predict the target test sample according to the trained machine learning model. Therefore, the prediction accuracy of the machine learning model can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and more particularly to a machine learning model training method, computing device, and computer-readable storage medium. Background Art

[0002] In data analysis based on machine learning models, the distribution differences between data bring huge challenges to data analysis and have a negative impact on the performance and generalization ability of machine learning models. For example, when the distribution of sample data in the training set is significantly different from the distribution of unknown data (also called test samples) in actual application scenarios, the machine learning model constructed by the training set data often finds it difficult to accurately capture the potential characteristics of the unknown data, resulting in a significant decrease in the prediction accuracy of the machine learning model when facing unknown data. However, how to improve the prediction accuracy of machine learning models has been under research. Summary of the invention

[0003] The purpose of this application is to provide a machine learning model training method, computing device and computer-readable storage medium to at least solve the problems in the related art.

[0004] To achieve the above objectives:

[0005] In a first aspect, an embodiment of the present application provides a method for training a machine learning model, the method comprising:

[0006] Step S1, generating a merged set according to the target test sample and the training samples in the preset training set;

[0007] Step S2, determining a target subset in the merged set with a minimum regularized covariance determinant, the Mahalanobis distance from the target test sample to the target subset is less than a robust distance cutoff value, and the number of samples included in the target subset is greater than a preset number threshold;

[0008] Step S3, determining a target training set according to target training samples whose Mahalanobis distance to the target subset is less than the robust distance cutoff value in the merged set and recognition results corresponding to the annotations of the target training samples in the preset training set;

[0009] Step S4: train a machine learning model based on the target training set to predict the target test sample according to the trained machine learning model.

[0010] In one embodiment, before step S2, the following steps are included:

[0011] Calculate the Q of the merged set n Estimation;

[0012] Q n The estimator is truncated to obtain a diagonal matrix;

[0013] Normalize the combined set according to the diagonal matrix.

[0014] In one embodiment, step S2 comprises:

[0015] Step S10, determining the number of samples h that the subset should contain in the current iteration;

[0016] Step S11, based on the number of samples h that the subset should contain in the current iteration, searching for a subset with a minimum regularized covariance determinant from the merged set, and obtaining a first subset with a minimum regularized covariance determinant in the merged set determined in the current iteration;

[0017] Step S12, determining a robust distance cutoff value according to the Mahalanobis distance from the merged set to the first subset;

[0018] Step S13, taking the samples whose Mahalanobis distance in the first subset is less than the robust distance cutoff value and are merged into the first subset as the second subset;

[0019] Step S14, detecting whether the second subset contains the target test sample and whether the number of samples contained in the second subset is greater than a preset number threshold, if not, executing step S15, otherwise executing step S16;

[0020] Step S15, perform operation h=h+1, and return to step S10;

[0021] Step S16: determine the second subset as the target subset in the merged set having the minimum regularized covariance determinant.

[0022] In one implementation, the step S11 includes:

[0023] Step S110, calculating the mean and regularized covariance matrix of the current subset of the minimum regularized covariance determinant; the initial subset of the minimum regularized covariance determinant is set as the target test sample;

[0024] Step S111, detecting whether the determinant of the regularized covariance matrix of the current subset is the same as the determinant of the regularized covariance matrix of the previous subset, if not, executing step S112, otherwise executing step S114;

[0025] Step S112, calculating the Mahalanobis distances from all samples in the merged set to the current subset according to the mean and regularized covariance matrix of the current subset;

[0026] Step S113, sorting all samples in the merged set according to the size of the Mahalanobis distance, and taking the h samples with the smallest Mahalanobis distance in the merged set as the updated current subset of the minimum regularized covariance determinant, and returning to step S110;

[0027] Step S114: determine the current subset as the first subset in the merged set having the minimum regularized covariance determinant.

[0028] In one embodiment, the number h of samples that the subset should include in the first iteration is set to 2.

[0029] In one embodiment, a machine learning model is used for quantitative analysis and / or classification prediction.

[0030] In a second aspect, an embodiment of the present application provides a computing device, comprising: a processor and a memory storing a computer program, wherein when the processor runs the computer program, the machine learning model training method described in the first aspect above is implemented.

[0031] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the machine learning model training method described in the first aspect is implemented.

[0032] In the machine learning model training method, computing device and computer-readable storage medium provided in the embodiments of the present application, by determining the target subset in the merged set with the minimum regularized covariance determinant, the target training samples whose Mahalanobis distance from the merged set to the target subset is less than the robust distance cutoff value are obtained, that is, the training samples with the same distribution as the target test samples are obtained, and then the machine learning model is trained using the training samples with the same distribution as the target test samples, so as to predict the target test samples according to the trained machine learning model, thereby effectively improving the prediction accuracy of the machine learning model and enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flowchart of a machine learning model training method provided in an embodiment of the present application;

[0034] Figure 2 A schematic diagram of a process for determining a target subset having a minimum regularized covariance determinant in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of a process of finding a subset having a minimum regularized covariance determinant from a merged set in an embodiment of the present application;

[0036] Figure 4 A schematic diagram of the specific process of the machine learning model training method provided in the embodiment of the present application;

[0037] Figure 5 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0039] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0040] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of..." or "when..." or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, “A, B, or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0041] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0042] It should be noted that in this article, step codes such as S1 and S2 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S2 first and then S1, etc., but these should all be within the scope of protection of this application.

[0043] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0045] See also Figure 1 , is a machine learning model training method provided in an embodiment of the present application. The machine learning model training method can be performed by a machine learning model training device provided in an embodiment of the present application. The machine learning model training device can be implemented in software and / or hardware, for example, it can be a computing device such as a computer or a server. The machine learning model training method provided in this embodiment includes:

[0046] Step S1: Generate a merged set based on the target test sample and the training samples in the preset training set.

[0047] The target test sample is any test sample to be predicted, and its corresponding recognition result is unknown. The preset training set is a set constructed based on samples with known recognition results, including multiple training samples and the corresponding annotated recognition results of each training sample. It can be understood that for quantitative analysis prediction, the recognition result can be a certain value, while for classification prediction, the recognition result can be a label value. Here, according to the target test sample and the training samples in the preset training set, a merged set is generated, which can be the target test sample and the training sample forming a merged set, and the number of samples contained in the merged set is the number of training samples plus 1. For example, the test set Xte The i-th test sample in is defined as the target test sample x te , and the target test sample x te Add training sample set X to get merged set X m =[X; x te ], if the number of training samples in the training sample set X is n, then the number of samples contained in the merged set is n+1.

[0048] Step S2, determining a target subset in the merged set with a minimum regularized covariance determinant, the Mahalanobis distance from the target test sample to the target subset is less than the robust distance cutoff value, and the number of samples included in the target subset is greater than a preset number threshold.

[0049] The preset quantity threshold can be set according to actual needs, for example, it can be set in combination with the type of sample data, etc. In this embodiment, the preset quantity threshold can be set to 80 or 100, etc. The robust distance cutoff value can be set according to actual needs, or it can be obtained based on iterative calculation.

[0050] In one embodiment, before step S2, the following steps are included:

[0051] Calculate the Q of the merged set n Estimation;

[0052] Q n The estimator is truncated to obtain a diagonal matrix;

[0053] The merged set is normalized according to the diagonal matrix to obtain a normalized merged set.

[0054] For example, according to the formula Q n =d{x i -x j |,i<j} (k) Calculate the merged set X m Q n estimator, d is the consistency factor, n is the merged set X m The number of samples, x i and x j Respectively represent the merged set X m The i-th sample and the j-th sample in Q n The estimator is truncated so that the minimum value is not less than 0.001, and a diagonal matrix D containing the scale information of each variable is obtained. x Then, using the formula The merged set X m Standardize and obtain the standardized merged set U m , where the vector v xFor the combined set X m The vector consisting of the median of each variable in the standardized merged set X m The target test sample in is denoted as u te .

[0055] In one embodiment, see Figure 2 , step S2, comprising:

[0056] Step S10, determining the number of samples h that the subset should contain in the current iteration;

[0057] Step S11, based on the number of samples h that the subset should contain in the current iteration, searching for a subset with a minimum regularized covariance determinant from the merged set, and obtaining a first subset with a minimum regularized covariance determinant in the merged set determined in the current iteration;

[0058] Step S12, determining a robust distance cutoff value according to the Mahalanobis distance from the merged set to the first subset;

[0059] Step S13, taking the samples whose Mahalanobis distance in the first subset is less than the robust distance cutoff value and are merged into the first subset as the second subset;

[0060] Step S14, detecting whether the second subset includes the target test sample and whether the number of samples included in the second subset is greater than a preset number threshold, if not, executing operation h=h+1 and returning to step S10, otherwise executing step S15;

[0061] Step S15: determine the second subset as the target subset in the merged set having the minimum regularized covariance determinant.

[0062] In this embodiment, the number of samples h that the subset should include in the first iteration can be set to 2, and of course other values ​​are also possible, which are not specifically limited here. For each iteration operation, first, after determining the number of samples h that the subset should include in the current iteration, by finding the subset with the minimum regularized covariance determinant from the merged set, the first subset with the minimum regularized covariance determinant in the merged set determined in the current iteration is obtained; then, the Mahalanobis distance from the merged set to the first subset is calculated, and the robust distance cutoff value is determined based on the Mahalanobis distance; then, the samples whose Mahalanobis distance from the merged set to the first subset is less than the robust distance cutoff value are taken as the second subset; then, it is detected whether the second subset contains the target The test sample and whether the number of samples contained in the second subset is greater than the preset number threshold. If the second subset does not contain the target test sample and / or the number of samples contained in the second subset is less than the preset number threshold, the operation h=h+1 is performed, that is, the number of samples h that the subset should contain in the current iteration is added by 1 as the number of samples that the subset should contain in the next iteration. If the second subset contains the target test sample and the number of samples contained in the second subset is greater than the preset number threshold, the second subset is determined as the target subset with the minimum regularized covariance determinant in the merged set.

[0063] In one embodiment, see Figure 3 , step S11, comprising:

[0064] Step S110, calculating the mean and regularized covariance matrix of the current subset of the minimum regularized covariance determinant; the initial subset of the minimum regularized covariance determinant is set as the target test sample;

[0065] Step S111, detecting whether the determinant of the regularized covariance matrix of the current subset is the same as the determinant of the regularized covariance matrix of the previous subset, if not, executing step S112, otherwise executing step S114;

[0066] Step S112, calculating the Mahalanobis distances from all samples in the merged set to the current subset according to the mean and regularized covariance matrix of the current subset;

[0067] Step S113, sorting all samples in the merged set according to the size of the Mahalanobis distance, and taking the h samples with the smallest Mahalanobis distance in the merged set as the updated current subset of the minimum regularized covariance determinant, and returning to step S110;

[0068] Step S114: determine the current subset as the first subset in the merged set having the minimum regularized covariance determinant.

[0069] Among them, for the current subset with the minimum regularized covariance determinant, first calculate the mean and regularized covariance matrix of the current subset with the minimum regularized covariance determinant, and then detect whether the determinant of the regularized covariance matrix of the current subset is the same as the determinant of the regularized covariance matrix of the previous subset. If the determinant of the regularized covariance matrix of the current subset is the same as the determinant of the regularized covariance matrix of the previous subset, then the current subset is determined as the first subset with the minimum regularized covariance determinant in the merged set. If the row of the regularized covariance matrix of the current subset is If the column form is different from the determinant of the regularized covariance matrix of the previous subset, the Mahalanobis distances of all samples in the merged set to the current subset are calculated based on the mean and regularized covariance matrix of the current subset, and then all samples in the merged set are sorted according to the size of the Mahalanobis distance, and the h samples with the smallest Mahalanobis distance in the merged set are used as the updated current subset with the minimum regularized covariance determinant, and return to step S110, that is, the mean and regularized covariance matrix of the updated current subset with the minimum regularized covariance determinant are calculated, and this cycle is repeated.

[0070] Step S3: determine the target training set according to the target training samples whose Mahalanobis distance from the merged set to the target subset is less than the robust distance cutoff value, and the recognition results of the corresponding annotations of the target training samples in the preset training set.

[0071] Among them, we can first determine the target training samples whose Mahalanobis distance from the merged set to the target subset is less than the robust distance cutoff value, and then combine the recognition results of the corresponding annotations of the target training samples in the preset training set to determine the target training set, that is, the target training set includes the target training samples and the recognition results of the corresponding annotations of the target training samples.

[0072] Step S4: train a machine learning model based on the target training set to predict the target test sample according to the trained machine learning model.

[0073] Among them, the machine learning model in this embodiment can be used for quantitative analysis (i.e., regression analysis) and / or classification prediction, that is, the machine learning model training method provided in this embodiment can be used not only for quantitative analysis, but also for classification prediction. The machine learning model can be constructed based on machine learning algorithms such as partial least squares regression, random forest, support vector machine, etc., which are not specifically limited here. Exemplarily, when it is necessary to perform quantitative analysis on the target test sample, the machine learning model can be constructed based on machine learning algorithms such as partial least squares, least squares support vector machine, etc. that can be used for quantitative analysis; when it is necessary to perform classification prediction on the target test sample, the machine learning model can be constructed based on machine learning algorithms such as least squares support vector machine, neural network, etc. that can be used for classification prediction. After obtaining the constructed machine learning model, the constructed machine learning model can be trained based on the target training set to predict the target test sample according to the trained machine learning model, so as to obtain the prediction result of the target test sample.

[0074] For example, see Figure 4 The following is a detailed description of the machine learning model training method provided in this embodiment. The specific process includes:

[0075] (1) The test set X te The i-th test sample in is defined as the target test sample x te , and according to the target test sample x te and training sample set X to generate a merged set X m =[X; x te ].

[0076] (2) According to the formula Q n =d{|x i -x j |,i<j} (k) Calculate the merged set X m Q n estimator, d is the consistency factor, n is the merged set X m The number of samples, x i and x j Respectively represent the merged set X m The i-th sample and the j-th sample in . Using the formula The merged set X m Standardize and obtain the standardized merged set, namely the standardized data U m , vector v x For the combined set X m The vector consisting of the median of each variable in the standardized merged set X m The target test sample in is denoted as u te .

[0077] (3) Initially, the target test sample u te As the current subset H in the Minimum Regularized Covariance Determinant (MRCD) old , that is, H old =u te , set the initial h value to 2, and perform the following loop operations:

[0078] S3.1. Calculate subset H old The mean Covariance matrix and the regularized covariance matrix R old The regularized covariance matrix is ​​calculated as follows:

[0079]

[0080] Among them, the target matrix M is the identity matrix, is the covariance matrix, and ρ is the regularization parameter. Initially, the subset H old The mean Equal to u te , the regularized covariance matrix R old is a diagonal matrix with elements on the diagonal being truncated values, and the truncation value is set to 0.1.

[0081] S3.2, according to the formula Calculate the merged set X m All samples in the set H old The Mahalanobis distance MD old (i), i = 1, 2, ..., n + 1, merge the set X m The samples in the table are calculated according to the Mahalanobis distance MD. old (i) Sort by size, from the merged set X m Take the Mahalanobis distance MD old (i) The smallest h samples are used as subset H new , and H new ={π(1),π(2),...,π(h)}, π(1) represents the merged set Xm to the subset H old The Mahalanobis distance MD old (i) The smallest sample, then calculate the subset H new The mean and the covariance matrix Regularized covariance matrix R new .

[0082] S3.3. Detection subset H new The regularized covariance matrix R newThe determinant det(R new ) and subset H old The regularized covariance matrix R old The determinant det(R old ) are equal, if so, calculate the merged set X m All samples in the set H new The Mahalanobis distance MD new (i), and subset H new As the subset H with the minimum regularized covariance determinant AMRCD Then, execute S3.4; if not, As R new As R old , that is, perform an update operation R old =R new , which is equivalent to subsetting the subset H new As the updated subset H old , repeat S3.2 until the determinant of the regularized covariance matrix R no longer decreases, and the subset H obtained at this time new As the combined set X m The subset H with the minimum regularized covariance determinant in AMRCD , the number of samples in the subset is h.

[0083] S3.4. In order to obtain a robust distance cutoff value for data in high-dimensional space, it is necessary to merge the set X m Samples from the middle to subset H AMRCD The Mahalanobis distance MD AMRCD (i) Convert to logarithmic form to obtain LD i =log(0.1+MD i ), where 0.1 is to prevent values ​​close to zero from appearing in the logarithmic term, MD i For the i-th sample to subset H AMRCD The Mahalanobis distance.

[0084] Then, according to the formula Calculate the cutoff value c of the robust distance in the high-dimensional space and merge the set X m Medium to subset H AMRCD The samples whose Mahalanobis distance is less than or equal to the cutoff value c are taken as the subset U AMRCD . Among them, for the subset H AMRCD For samples whose Mahalanobis distance is less than or equal to the cutoff value c, the weight w corresponding to the sample can be set i is 1, and for the subset H AMRCD The Mahalanobis distance MD new (i) For samples greater than the cutoff value c, the weight w corresponding to the sample can be set iis 0, and then the subset U is generated by sample weighting AMRCD .

[0085] If the target test sample u te Does not belong to subset U AMRCD Or subset U AMRCD If the number of samples is less than γ, γ∈[20,100], re-execute step (3) and replace the value of h with h+1. Otherwise, continue with step (4).

[0086] (4) Through the subset U AMRCD Get the original sub-dataset X AMRCD , subset X AMRCD Except for the target test sample x te The remaining training set data is used as the sub-training set data X sub , sub-training set X sub The corresponding label is denoted as y sub It should be noted that in quantitative analysis, the label is a certain value, while in classification prediction, the label is a type of value.

[0087] (5) In the regression task, use the sub-training set data X sub and label y sub Build a machine learning model for quantitative analysis and test it on the target sample x te Make predictions and get fitting labels That is, the fitted value. In the classification task, using the sub-training set data X sub and label y sub Build a machine learning model for classification prediction and test it on the target sample x te Make predictions and get the target test sample x te Fitting label That is, the type value. For the test set data X te All target test samples x te By performing the above steps, the label values ​​of all test set data can be predicted.

[0088] Therefore, the training method of the machine learning model provided in this embodiment can be applicable to high-dimensional data, and can identify and screen out training samples in the training set that come from the same distribution as the test samples to construct a machine learning model for regression or classification, effectively improving the prediction accuracy of the machine learning model.

[0089] Based on the same inventive concept as the aforementioned embodiment, the technical effect of the method provided in this embodiment is verified and explained through specific experiments below. For the sake of simplicity of description, the machine learning model training method provided in this embodiment is referred to as the AMRCD method below.

[0090] 1. Classification Task

[0091] In order to demonstrate the effectiveness of the machine learning model training method provided in this embodiment in the classification task, this embodiment takes the use of the K-nearest neighbor (KNN) algorithm to construct a machine learning model as an example, that is, the AMRCD method is combined with the KNN algorithm for classification, hereinafter referred to as the AMRCD-KNN method, and the AMRCD-KNN method is compared with KNN, support vector machine (SVM), linear discriminant analysis (LDA), classification and regression tree (CART) and partial least squares discriminant analysis (PLS-DA) on two real public classification data sets (glass data set and sediment data set) for classification prediction.

[0092] The glass dataset is a machine learning classification dataset built into Matlab. It can classify glass into car window glass or non-car window glass according to its chemical composition. The sample variables are the refractive index, the content of nine elements, sodium, magnesium, aluminum, silicon, potassium, calcium, barium and iron. The glass dataset has a total of 214 samples, including 51 car window glass samples and 163 non-car window samples. 25% of the samples are randomly selected from the glass dataset as the test set, including 13 car window glass samples and 41 non-car window samples in the test set. Accuracy (Acc), sensitivity (Sen), specificity (Spe) and geometric mean (G-mean) are used as evaluation indicators of the classification results.

[0093] The sediment dataset is obtained from the available data of the Sediment Toxicity Database of the National Oceanic and Atmospheric Administration of the United States. The Sediment Toxicity Database contains sediment chemistry and sediment toxicity data collected synchronously from the Atlantic, Gulf, and Pacific coasts of North America. The Sediment Toxicity Database contains information on different bioassay methods, selects 1884 samples of amphibians and crustaceans that survived for 10 days in the experiment, and records whether the sample is toxic or not. The variables of the samples are the content of nine elements: arsenic, cadmium, chromium, copper, lead, mercury, nickel, silver, and zinc. There are 1624 non-toxic samples and 260 toxic samples in the sediment dataset. 25% of the samples are randomly selected as the test set, of which 406 are non-toxic samples and 65 are toxic samples in the test set. The rest of the settings are the same as the glass dataset.

[0094] Table 1

[0095]

[0096] Table 1 shows the prediction results of different classification methods in the glass data set. It can be seen that the machine learning model training method provided in this embodiment combined with KNN (i.e., AMRCD-KNN method) has the highest classification indicators of Acc, Sen, Spe and G-mean, indicating that the AMRCD-KNN method has a good classification effect.

[0097] Table 2

[0098]

[0099] The prediction results of different classification methods in the sediment data set are shown in Table 2. The AMRCD-KNN method is higher than other classification methods in the four classification indicators of Acc, Sen, Spe and G-mean, which shows that the AMRCD-KNN method has a good classification effect.

[0100] 2. Regression Task

[0101] In order to demonstrate the effectiveness of the machine learning model training method provided in this embodiment in the regression task, this embodiment takes the use of partial least squares (PLS) and least squares support vector machine (LSSVM) to construct machine learning models as examples, that is, the AMRCD method is combined with the PLS and LSSVM algorithms for classification, which are hereinafter referred to as AMRCD-PLS and AMRCD-LSSVM methods, and compared with the PLS and LSSVM algorithms for regression prediction.

[0102] In this embodiment, four simulated data sets (respectively, simulation I, simulation II, simulation III and simulation IV) and two real near-infrared spectral data sets are used. These data sets are relatively small in scale and have high-dimensional or multicollinearity problems. The machine learning model training method provided in this embodiment can handle the situation where the training data and test data in the high-dimensional data set come from different distributions. The high-dimensional data sets of simulation I and simulation II contain outliers of two different pollution types, while in the low-dimensional data set of simulation III and the high-dimensional data set of simulation IV, the training set data contains training sample data from multiple different distributions, and the test data comes from only one of the distributions, that is, the four simulated data sets include different pollution situations (including a small number of outliers and data from other distributions).

[0103] First, an uncontaminated normal data set is generated according to the bilinear model. The specific model is as follows:

[0104] T~N k (0k ,Σ t )

[0105] β t ~N k (0 k ,I t )

[0106] X=TI k,p +N p (0 p ,0.1I p )

[0107] y=Tβ t +N(0,0.1)

[0108] Among them, T has a mean of 0 k , the standard deviation is ∑ t Normal distribution of β t The mean value is 0 k , the covariance is I t Normal distribution; 0 k is a k-dimensional vector, ∑ t is a t-dimensional diagonal matrix, I t is the identity matrix of dimension t. When i=j, I k,p (i,j)=1, otherwise I k,p (i, j) = 0. For the PLS regression model, the above setting means that the optimal number of latent variables is k. Therefore, there is no need to divide the validation set from the training set through a cross-validation strategy to determine the optimal parameters of the PLS model.

[0109] Next, contaminated data are constructed for the four simulated datasets, including outliers and data from different distributions.

[0110] Among them, T ò , X ò and ò Representing the contaminated data, the high-leverage outliers in Simulation I were generated as follows:

[0111] T ò ~10N k (0 k ,Σ t )

[0112] X ò =T ò I k,p +N p (0 p ,0.1I p )

[0113] y ò =Tò β t,ò +N(0,0.1)

[0114] Here β t,ò From β t Same distribution.

[0115] In Simulation II, vertical outliers, also called y outliers (only the y value is abnormal), are generated by adjusting the y value based on the uncontaminated T:

[0116] y ò =Tβ t +N(10,0.1)

[0117] In the low-dimensional dataset of Simulation III and the high-dimensional simulation set of Simulation IV, T is drawn from different distributions. ò,r Replace T in the above equation to generate data from a different distribution than the normal dataset:

[0118] T ò,r ~N k (rI k ,Σ t )

[0119] Where r is a random integer, r∈[-50,50].

[0120] Table 3

[0121]

[0122] In Table 3, the training data volume n, data dimension p, optimal number of latent variables k, and test data volume n in the four simulated datasets are listed. te , the amount of polluted data n c and the covariance matrix Σ t In each simulated data set, k components were used to calculate the root mean square error of prediction (RMSEP) of PLS ​​and AMRCD-PLS.

[0123] Table 4 shows the comparative results of PLS ​​and AMRCD-PLS in four simulated data sets. In each type of simulation, the prediction effects of the above algorithms in uncontaminated data were compared. Data were randomly generated 100 times for each simulation, and the average value of RMSEP and its corresponding standard deviation were used to evaluate the model performance. For uncontaminated simulated data, the performance of AMRCD-PLS is comparable to that of PLS, with very small differences. In contaminated simulated data, PLS begins to fail, which is particularly evident in Simulation III and Simulation IV. By selecting training samples from the same distribution for each test sample to build a model, the RMSEP value of AMRCD-PLS is significantly lower than that of PLS, which shows the effectiveness of AMRCD-PLS.

[0124] Table 4

[0125]

[0126] There are two real near-infrared spectroscopy datasets, namely the tablet dataset and the melamine dataset. The training data in the tablet dataset contains four different types of spectra, while the test data only contains one type of spectrum. For the melamine dataset, although the training data and test data come from the same formula, the distribution of the data is quite different, and the test data in the test set may also come from different distributions.

[0127] The samples of the tablet data set are measured by high performance liquid chromatography and are divided into four types. There are 70 samples in the first type and 80 samples in each of the other three types, so all 310 samples are used as source data. Each spectrum provides 404 wavelength channels with a wavelength range of 7398-10507 nanometers, one channel every 7 nanometers. At the same time, the attribute of activity is used as the response variable y. 32 samples are randomly selected from the second type of data in the tablet data set as the test set, and the number of test sets accounts for 40% of the second type of data and about 10% of the total source data. The remaining samples in the source data are used as training sets. Using the cross-validation strategy, the training set is divided into a training set and a validation set, where the training set is used to build a regression model and the validation set is used to select the optimal number of latent variables for the PLS model. For both PLS and LSSVM methods, the RMSEP value is used to evaluate the performance of different methods. The experiment was repeated 100 times, and the average and standard deviation of the RMSEP value were calculated.

[0128] The melamine dataset consists of Fourier transform near infrared absorption spectra and turbidity point readings recorded at Metadynea GmbH during the condensation of melamine formaldehyde resin. The former are recorded online with a fiber optic probe, while the latter are measured offline by determining the turbidity point, the temperature at which the condensate starts to turn turbid. The turbidity point is a measure of polymer length and the amount of crosslinking, an important parameter for defining the physical and chemical properties of the final resin. The dataset contains recordings from multiple batches of four different formulations (R562, R568, R861 and R862) using different raw material ratios and / or additives. The NIR spectra are divided into two spectral ranges (5546-6254 and 6596-6975 cm -1 ), containing 225 and 121 variables respectively. The 121 input variables in recipe R862 are used as the variables of the source data. The source data contains 472 samples, from which 10% of the samples are randomly selected as the test set. The remaining samples in the source data are used as the training set. The rest of the settings are the same as those of the pill data.

[0129] The distribution of the second type of spectral data in the pill data set is significantly different from that of the other three types of spectral data. By selecting training data from the same distribution as the target test sample to build a PLS model, AMRCD-PLS can outperform the classic PLS method. Among them, Table 5 shows the mean and standard deviation of the RMSEP values ​​of AMRCD-PLS and PLS in the real data set. For the pill data set, the RMSEP value of AMRCD-PLS is much lower than that of PLS, that is, the prediction accuracy of AMRCD-PLS is higher than that of PLS. Therefore, it can be seen that selecting training samples from the same distribution as each target test sample to train the machine learning model can indeed significantly improve the prediction performance of the model. In addition, the standard deviation of the RMSEP value of the AMRCD-PLS method is relatively small, which shows that AMRCD-PLS is more stable.

[0130] Table 5

[0131]

[0132] Table 6 shows the mean and standard deviation of the RMSEP values ​​of AMRCD-LSSVM and LSSVM in the real data set. For the pill data set, the RMSEP value of AMRCD-LSSVM is much lower than that of LSSVM, that is, the prediction accuracy of AMRCD-LSSVM is higher than that of LSSVM. Therefore, selecting training samples from the same distribution as each target test sample to train the machine learning model can also significantly improve the performance of the LSSVM model, and the standard deviation of the RMSEP value of the AMRCD-LSSVM method is relatively small, which shows that AMRCD-LSSVM is more stable.

[0133] Table 6

[0134]

[0135] It can be seen from the above comparative experimental results that the machine learning model training method provided in this embodiment can effectively improve the prediction accuracy of the machine learning model.

[0136] Based on the same inventive concept as the above embodiments, an embodiment of the present invention provides a computing device, such as Figure 5 As shown, the computing device includes: a processor 310 and a memory 311 storing a computer program; wherein, Figure 5 The processor 310 shown in the figure is not used to indicate that the number of the processor 310 is one, but is only used to indicate the positional relationship of the processor 310 relative to other devices. In actual applications, the number of the processor 310 may be one or more; similarly, Figure 5 The memory 311 shown in the figure has the same meaning, that is, it is only used to refer to the position relationship of the memory 311 relative to other devices. In practical applications, the number of memories 311 can be one or more. When the processor 310 runs the computer program, the machine learning model training method applied to the above computing device is implemented.

[0137] The computing device may also include: at least one network interface 312. The various components in the computing device are coupled together via a bus system 313. It is understood that the bus system 313 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 313 is not used in the example above. Figure 5 Various buses are labeled as bus system 313.

[0138] The memory 311 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 311 described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable types of memories.

[0139] The memory 311 in the embodiment of the present invention is used to store various types of data to support the operation of the computing device. Examples of these data include: any computer program used to operate on the computing device, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program may include various applications, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention may be included in the application program.

[0140] Based on the same inventive concept as the above-mentioned embodiment, this embodiment further provides a computer-readable storage medium, in which a computer program is stored. The computer-readable storage medium may be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); it may also be various devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program stored in the computer-readable storage medium is executed by the processor, the above-mentioned machine learning model training method is implemented. For the specific steps implemented when the computer program is executed by the processor, please refer to Figure 1 The description of the illustrated embodiment will not be repeated here.

[0141] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0143] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A machine learning model training method, characterized in that: The method comprises: Step S1, generating a merged set according to the target test sample and the training samples in the preset training set; Step S2, determining a target subset in the merged set with a minimum regularized covariance determinant, the Mahalanobis distance from the target test sample to the target subset is less than a robust distance cutoff value, and the number of samples included in the target subset is greater than a preset number threshold; Step S3, determining a target training set according to target training samples whose Mahalanobis distance to the target subset is less than the robust distance cutoff value in the merged set and recognition results corresponding to the annotations of the target training samples in the preset training set; Step S4: train a machine learning model based on the target training set to predict the target test sample according to the trained machine learning model.

2. The method according to claim 1, characterized in that Before step S2, it includes: Calculate the Q of the merged set n Estimation; Q n The estimator is truncated to obtain a diagonal matrix; Normalize the combined set according to the diagonal matrix.

3. The method according to claim 1 or 2, characterized in that: Step S2 comprises: Step S10, determining the number of samples h that the subset should contain in the current iteration; Step S11, based on the number of samples h that the subset should contain in the current iteration, searching for a subset with a minimum regularized covariance determinant from the merged set, and obtaining a first subset with a minimum regularized covariance determinant in the merged set determined in the current iteration; Step S12, determining a robust distance cutoff value according to the Mahalanobis distance from the merged set to the first subset; Step S13, taking the samples whose Mahalanobis distance in the first subset is less than the robust distance cutoff value and are merged into the first subset as the second subset; Step S14, detecting whether the second subset contains the target test sample and whether the number of samples contained in the second subset is greater than a preset number threshold, if not, executing step S15, otherwise executing step S16; Step S15, perform operation h=h+1, and return to step S10; Step S16: determine the second subset as the target subset in the merged set having the minimum regularized covariance determinant.

4. The method according to claim 3, characterized in that Step S11 includes: Step S110, calculating the mean and regularized covariance matrix of the current subset of the minimum regularized covariance determinant; the initial subset of the minimum regularized covariance determinant is set as the target test sample; Step S111, detecting whether the determinant of the regularized covariance matrix of the current subset is the same as the determinant of the regularized covariance matrix of the previous subset, if not, executing step S112, otherwise executing step S114; Step S112, calculating the Mahalanobis distances from all samples in the merged set to the current subset according to the mean and regularized covariance matrix of the current subset; Step S113, sorting all samples in the merged set according to the size of the Mahalanobis distance, and taking the h samples with the smallest Mahalanobis distance in the merged set as the updated current subset of the minimum regularized covariance determinant, and returning to step S110; Step S114: determine the current subset as the first subset in the merged set having the minimum regularized covariance determinant.

5. The method according to claim 3, characterized in that: The number of samples h that the subset should contain in the first iteration is set to 2.

6. The method according to claim 1, characterized in that Machine learning models are used for quantitative analysis and / or classification prediction.

7. A computing device, characterized in that It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the machine learning model training method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the machine learning model training method as described in any one of claims 1 to 6 is implemented.

Images