Spam email filtering system and method capable of intelligently selecting training samples

Abstract

The invention provides a spam email filtering system and a method capable of intelligently selecting training samples. Sample sets which are labeled in categories and sample sets which are not labeled in categories are initialized; samples in the labeled sample sets are used as a training set to train an initial classification model, and uncertainty of every unlabeled sample in the unlabeled sample sets are computed through the classification model, and the uncertainty refers to uncertainty of the samples belonging to categories; P samples with large uncertainty are selected in the samples which are not labeled in category, and the unlabeled P samples are labeled in category; the labeled samples are added in a final training set as labeled samples, and a novel category model is constructed on the final training set. The model is used for filtering emails, and determining whether an email is a spam email or a legal email. According to the spam email filtering system and the method, people can avoid learning samples which are not greatly favorable for category, and accuracy of the classification model is improved.

Description

technical field [0001] The invention relates to the technical field of anti-spam, in particular to a system and method for intelligently selecting samples from an unlabeled sample set for training during the construction of a classifier model and performing spam filtering. Background technique [0002] Spam filtering is a classification and filtering problem based on text content. In essence, it can be attributed to two processes of training and classification, namely: [0003] Process 1: The training process of word segmentation, feature selection, learning, and construction of a classifier for a large number of labeled (known category labels) samples; [0004] Process 2: The classification process of using the classifier to predict the unknown sample category. [0005] Therefore, sample quality is crucial and directly affects classification accuracy. [0006] The traditional passive learning mode faces sample selection problems, for example, the manual labeling of unlabe...

AI Technical Summary

Problems solved by technology

[0006] The sample selection problem faced by the traditional passive learning mode, for example, the manual labeling of unlabeled samples is complicated, and it is impossible to learn selectively for a given sample, etc.

Images