Methods for multi-class cost-sensitive learning

Abstract

Methods for multi-class cost-sensitive learning are based on iterative example weighting schemes and solve multi-class cost-sensitive learning problems using a binary classification algorithm. One of the methods works by iteratively applying weighted sampling from an expanded data set, which is obtained by enhancing each example in the original data set with as many data points as there are possible labels for any single instance, using a weighting scheme which gives each labeled example the weight specified as the difference between the average cost on that instance by the averaged hypotheses from the iterations so far and the misclassification cost associated with the label in the labeled example in question. It then calls the component classification algorithm on a modified binary classification problem in which each example is itself already a labeled pair, and its (meta) label is 1 or 0 depending on whether the example weight in the above weighting scheme is positive or negative, respectively. It then finally outputs a classifier hypothesis which is the average of all the hypotheses output in the respective iterations.

Description

BACKGROUND OF THE INVENTION [0001] 1. Field of the Invention [0002] The present invention generally relates to the field of cost-sensitive learning in the areas of machine learning and data mining and, more particularly, to methods for solving multi-class cost-sensitive learning problems using a binary classification algorithm. This algorithm is based on techniques of data space expansion and gradient boosting with stochastic ensembles. [0003] 2. Background Description [0004] Classification in the presence of varying costs associated with different types of misclassification is important for practical applications, including many data mining applications, such as targeted marketing, fraud and intrusion detection, among others. Classification is often idealized as a problem where every example is equally important, and the cost of misclassification is always the same. The real world is messier. Typically, some examples are much more important than others, and the cost of misclassifyi...

AI Technical Summary

Benefits of technology

[0009] One of the methods of invention, which is based on the idea 1) above, works by repeatedly sampling from the expanded data set, which is obtained by enhancing each example in the original data set with as many data points as there are possible labels for any single instance. It then repeatedly draws sub-sample from this expanded data set using weighted sampling according to a certain example weighting scheme, in which each labeled example is given the weight specified as the difference between the maximum possible misclassification cost for the instance in question and the misclassification associated with the label in the particular labeled example. The example weighting remains constant throughout the iterative sampling procedure. It then finally outputs a classifier hypothesis which is the average of all the hypotheses output in the respective iterations.

[0010] Another one of the methods of invention, which is based on the idea 2) above, works by iteratively applying weighted sampling from the same expanded data set, using a different weighting scheme. The weighting scheme

Problems solved by technology

The real world is messier.

The third category concerns methods that modify the distribution of training examples before applying the classifier learning method, so that the classifier learned from the modified distribution is cost-sensitive.

Unfortunately, current methods in this category suffer from a major limitation: they are well-understood only for

Images