Search-ranking-oriented sample selection method based on noise-adding active learning

Abstract

The invention discloses a search-ranking-oriented sample selection method based on noise-adding active learning. The method comprises the following steps of: carrying out noise adding on unmarked samples so as to generate noise samples; carrying out prediction on the noise samples by using a ranking model trained by a training set so as to obtain the fraction distribution of the sample under the current ranking model; converting the fraction distribution into the ranking distribution, and using the variance of DCG (discounted cumulative gain) to measure ranking distribution so as to characterize uncertainty; and carrying out sample selection by using the uncertainty. By using the method disclosed by the invention, effective sample selection can be performed under the condition of insufficient samples in search ranking, and an effect of more effectively enhancing the performance of a model by using fewer samples can be achieved, thereby achieving the purpose of reducing the cost for sample marking.

Description

technical field [0001] The invention belongs to the fields of machine learning and information retrieval, and specifically relates to a sample selection method based on noise injection active learning for search and sorting. Background technique [0002] Sorting is a core problem in the field of information retrieval, such as recommendation, online advertising, etc., and its task is to build a ranking model. Ranking learning belongs to supervised learning. Similar to other supervised learning problems, the quality of the ranking model is highly related to the number of training samples. Usually building a high-quality ranking model requires labeling a large amount of training data. However, in many practical applications, although collecting unlabeled samples is relatively easy, labeled samples are very expensive. The problem of training data labeling has become a bottleneck in building high-quality ranking models. In fact, the amount of information contained in different...

AI Technical Summary

Problems solved by technology

However, one disadvantage of this method is that a set of ranking models is required to estimate the loss. When the initial training samples are insufficient, the accuracy of the ranking model will be very low, resulting in inaccurate estimation of the expected loss and affecting the final effect.

Images