Method for evaluating sample set division quality based on data set distance

Abstract

The invention discloses a method for evaluating the division quality of a sample set based on a data set distance. The method can overcome the defects of quantification and difficult evaluation of conventional error-based analysis, and tightly grasps the basic hypothesis that a training set and a test set need to be independent from each other and come from the same distribution. The method comprises steps estimating the mean value and variance of a sample set through the decomposition of a distance matrix between samples; calculating the distance between two distributions of the training setand the test set; carrying out probability distribution estimation by using distance distribution obtained by random sampling, calculating probabilities of different partitions, and evaluating the quality of data partition or the adaptability of a partition method to specific data by using exact quantitative indexes. On the basis of simplicity and practicability, evaluation of the effectiveness ofthe sample set division method is given, and a proper method is provided for helping researchers in the biomedical field to select a proper data division method and clarify the real generalization performance of the modeling method.

Description

technical field [0001] The invention relates to the field of biomedical technology, in particular to a method for evaluating the division quality of a sample set based on the distance between data sets. Background technique [0002] Sample division plays an important role in the biomedical field. Its purpose is to generate a test set and estimate the generalization ability of the model. The model is the formal expression of the relationship in the data, and the prediction of unknown samples can be realized through the model. The training error represents the learning ability of the modeling method, and the predictive ability of the unknown sample is the goal pursued by the modeling. Therefore, when performing model training, the data set is generally required to be large enough to cover the scope of future applications. When estimating the generalization ability of the model, the training set is also required to be consistent with the model training range, that is, the tw...

AI Technical Summary

Problems solved by technology

However, the decline in the prediction performance of the test set cannot be completely attributed to the distribution difference between the training set and the test set, and the error value varies greatly with different data sets.

Images