Target-oriented short text classification method

Abstract

The invention discloses a target-oriented short text classification method, which comprises the following steps of: 1, marking and classifying texts according to requirements; 2, according to needs, labeling positions and attributes of name entities for the text classification; 3, building a deep learning model network, with texts corresponding to data input, classification results corresponding to classification results in the step 1, and name entity recognition corresponding to attributes in the step 2; 4, setting a loss function used during training, the loss function being the weighted sum of the CRF loss function and the text classification loss function, and the weights of the two loss functions are adjusted through tests so that the text classification effect can be optimal. According to the method, in the training process, a data set for text classification needs to be prepared, and the data set needs to be labeled with the name entity recognition result. In the training process, the loss function of the network is set, the function is a weighted sum of a CRF loss function and a text classification loss function, and the weight is adjusted according to a test result. According to the method, the useful information can be accurately extracted under the conditions that the text is short and effective samples are few.

Description

technical field [0001] The invention relates to the technical field of short text classification, in particular to a goal-oriented short text classification method. Background technique [0002] Text classification is a technology in the field of NLP. Its goal is to divide texts into predefined categories. It has applications in many aspects. The current effect is mainly based on deep learning text classification. [0003] Name entity recognition technology is also a technology in the field of NLP. The goal is to identify specific categories of names from text, such as identifying the names of people in articles. At present, LSTM+CRF is generally used for name entity recognition. [0004] Copynet is a text generation model that also has an automatic target guidance mechanism. [0005] User portraits are the basis of Internet marketing today. When creating user portraits, a lot of information is unstructured text information, such as forums and reviews. This type of informa...

AI Technical Summary

Problems solved by technology

[0006] Due to the short text and the source of information, a lot of information is useless, so there is little valuable information, and some key information is mainly represented by numbers, which leads to poor text classification results such as ordinary TextCNN.

Images