Kernel function based rare category detection method fusing active learning and nonparametric semi-supervised clustering

Abstract

The invention relates to a kernel function based rare category detection method fusing active learning and nonparametric semi-supervised clustering. For the problems that marked data points are not fully utilized and category related information needs to be specified in advance in a conventional rare category detection method, the invention proposes the kernel function based rare category detection method fusing the active learning and the nonparametric semi-supervised clustering. A data distribution model is optimized by utilizing small amounts of marked data and large amounts of unmarked data with the nonparametric semi-supervised clustering method, and most representative abnormal points in the unmarked data points are selected out in combination with the active learning and submitted to experts for marking, so that the workload of manual marking in a rare category detection process is reduced, the efficiency of the rare category detection process is improved, and the problem in rare category discovery under a nonlinear condition is solved.

Description

technical field [0001] The invention relates to the field of abnormal data detection, in particular to a rare category detection method based on kernel function fusion active learning and non-semi-supervised clustering. Background technique [0002] Anomaly data detection plays a key role in many applications, such as healthcare, fault detection in critical safety systems, and tracking of specific actors in videos. The significance of outlier data points is that they can often give a lot of useful information in a particular application. However, abnormal data points can be divided into two categories. The first category is ordinary abnormal points, which are usually generated by some predictable reasons. The second category belongs to outliers with additional information, which usually require further exploration and analysis. Compared with ordinary outliers, these more interesting outliers usually only occupy a smaller proportion of the total outliers. The rare category...

AI Technical Summary

Problems solved by technology

This process is repeated until a certain proportion of data points have been labeled or no new rare categories are found after a certain number of iterations; it solves how to use manually labeled data points to update The problem of efficiently detecting rare classes in datasets

Images