Method for performing iterative modeling on unsaturated information

Abstract

The invention relates to a method for performing iterative modeling on unsaturated information. The method comprises the following steps: A, training an unsaturated data sample to obtain a probabilityvalue of the data sample; B, setting a first confidence coefficient list, and layering the data samples according to the relationship between the probability values and confidence coefficients in thelist to obtain a final confidence coefficient upper bound and a final confidence coefficient lower bound; C, layering again to obtain a training data set; D, predicting the probability values of datasamples except for the training data set, layering the data samples except for the training data set according to the upper / lower bound of the final confidence, and combining a layering result with positive samples and negative samples in the training data set to form a new training data set; and E, iterating the steps B to D until the data samples except for the training data set cannot be layered again to obtain a finally formed new training data set. According to the invention, a universal model is realized, the unsaturated information applied in various occasions can be classified, and the accuracy and the efficiency are relatively high.

Description

technical field [0001] The invention relates to a modeling method according to the type of information samples, specifically a method for iteratively modeling unsaturated information. Background technique [0002] In the field of data mining, it usually takes a period of observation period to obtain the sample labels. Therefore, when the time window is insufficient and the data is small, the sample labels may be less than the actual situation or the confidence level is not enough. This situation will cause some samples to be difficult to distinguish in the actual modeling process (there is not enough confidence in the prediction process to prove whether it is a positive sample or a negative sample), so that the overall AUC (Area Under Curve) of the model, KS (in the model The evaluation indicators used to distinguish the degree of separation between positive and negative samples) and other indicators are low, and the model effect cannot reach the ideal value. The quality of...

AI Technical Summary

Problems solved by technology

[0006] 1: Although data expansion solves the problem of insufficient data, it also introduces noise and bias

The distribution of the new sample is not completely consistent with the original sample, so the formed training sample is also different from the sample distribution of the target field used by the model for prediction. Inconsistent sample distribution will cause the model to be biased. During the prediction process, we get The prediction error will be larger

[0007] 2: Migration learning requires similarity between the main features of the target sample and the original training sample, which is currently mainly used in deep learning, but not applicable to general machine learning methods

Images