A rare class detection method and device based on a k-nearest neighbor graph

Abstract

The invention discloses a k neighbor graph-based rare class detection method, which comprises the following steps of: firstly, constructing a k neighbor graph of a given unlabeled data set S, and automatically selecting a k value by an algorithm; and then, based on the constructed k neighbor graph, giving a definition of a change coefficient Vc, and calculating a change coefficient Vc value of each node in the data set, finding out the node x with the maximum change coefficient from all the nodes, inquiring a labeler to obtain a category label y of the node x, and respectively adding the x andthe y into the selected data sample set I and the selected data sample real category label set L; carrying out rare class detection by utilizing a method for detecting local mutation of data sample distribution in the data set, and compared with other priori-free rare class detection methods, the KRED method is higher in efficiency and lower in algorithm overhead. And meanwhile, through a methodof automatically selecting the k value, the discovery efficiency of each class in the data set is effectively improved, and the inquiry frequency required for discovering all classes in the data set is remarkably reduced.

Description

technical field

[0001] The invention relates to the technical field of data mining, in particular to a rare class detection method and device based on a k-nearest neighbor graph. Background technique

[0002] Rare class detection is a very important task in data mining. It aims to find those rare classes in unlabeled data sets. practical significance, and thus worthy of further study. For example, in the mass of financial transaction record data, sometimes a small amount of illegal transaction records that exploit the loopholes of the financial system or use fraudulent means are hidden; in the mass of normal network access, there are a small amount of malicious network behavior. In addition to being used for the above practical problems, rare class detection can also obtain a small number of classified data samples from a given unlabeled data set, which can be further used to construct classifiers or for semi-supervised learning methods such as collaborative training and ac...

Images