Text classification method and device applying outlier detection algorithm LOF model

Abstract

The embodiment of the invention provides a text classification method and device applying an outlier detection algorithm LOF model. The method comprises the following steps: acquiring a training data set comprising training texts and corresponding category labels, wherein the training texts comprise training texts with known labels and training texts with other labels, which are configured according to a preset proportion; training a classification model by using the training data set; inputting the training text with the known label into the classification model to obtain an embedded representation, an intermediate result and a final representation vector of the training text with the known label; training a plurality of LOF models according to the embedded representation of the training text with the known label, the intermediate result and the final representation vector; and judging whether the test text is of an unknown category according to the classification model and the LOF models. According to the technical scheme provided by the invention, the text classification model can identify an unknown category from the test text, the situation that the text of the unknown category is allocated to a known category label is avoided, and the accuracy of text classification is improved.

Description

technical field [0001] The present application relates to the technical field of natural language processing, in particular to a text classification method and device using an outlier detection algorithm LOF model. Background technique [0002] Text classification is one of the basic tasks in the field of natural language processing technology. It has very rich applications in real life. For example, applications such as public opinion monitoring, news classification, and sentiment classification based on natural language processing technology are all realized through text classification tasks. of. [0003] The text classification task trains the classification model through several fixed categories of training texts, so that the classification model can recognize several fixed categories of texts from unknown texts. The current classification model is usually trained by a deep learning model, and the deep learning model can only give the category judgment of the input text...

AI Technical Summary

Problems solved by technology

[0004] It can be seen that the current text classification tasks implemented by deep learning models cannot correctly identify texts other than known categories, and will also classify texts other than known categories into wrong categories, resulting in a decrease in classification accuracy

Images