Method and device for identifying unknown intention text

Abstract

The embodiment of the invention provides an unknown intention text recognition method and device. According to the scheme, the method comprises the steps that K positive samples and S negative samples corresponding to each training sample are obtained, wherein K and S are both positive integers larger than or equal to 1; using a classifier to obtain sentence representations of the training samples and the corresponding positive samples and negative samples, so that the sentence representations of the samples of the same category are gathered together, and the sentence representations of different categories are far away from each other; determining a decision center of each category according to the sentence representation, and learning a decision boundary of each category; judging whether the to-be-recognized text is located outside decision boundaries of all categories or not; and if yes, determining that the to-be-recognized text is an unknown intention text. According to the embodiment of the invention, comparative learning and classification learning are introduced in a classifier training stage, so that sentence representations of samples of the same category are gathered together, sentence representations of different categories are far away from each other, the effect is better when a decision boundary is trained, and the classifier can recognize a text with an unknown intention more accurately.

Description

technical field [0001] The present application relates to the technical field of natural language processing, and in particular, to a method and device for recognizing unknown intent text. Background technique [0002] Text classification is one of the basic tasks in the field of natural language processing technology. It has very rich applications in real life. For example, applications based on natural language processing technology such as public opinion monitoring, news classification, and sentiment classification are all realized through text classification tasks. . [0003] Currently, the text classification task trains a classification model with several fixed categories of training samples, enabling the classification model to recognize several fixed categories of text from unknown texts, however, for unknown texts that do not belong to these fixed categories (i.e. unknown intent ), the classification model fails to classify. For example: in a news classification s...

AI Technical Summary

Problems solved by technology

[0003] At present, the text classification task trains the classification model through several fixed categories of training samples, so that the classification model can recognize several fixed categories of text from unknown texts. However, for unknown texts that do not belong to these fixed categories (that is, unknown intent ), the classification model cannot classify

For example: in a news classification scenario, if the training samples include labels for the three categories of sports, economy, and entertainment, then the classification model trained using these three categories of training samples can only be used for the three categories of sports, economy, and entertainment. Classify the text to be recognized, and the text to be recognized in the life class belongs to the unknown intention for the classification model, but the classification model cannot recognize this unknown intention

[0004] In addition, in some scenarios, there may be many types of text categories, and the category labels of training samples may only cover part of the categories, that is, the category labels of training samples are incomplete

Images