Text classification model training method and device, storage medium and computer equipment

Abstract

The invention discloses a text classification model training method, a text classification model training device, a storage medium and computer equipment. The text classification model training methodcomprises the steps of: carrying out clustering on first text samples, and acquiring at least one first text cluster; based on a first preset number of first text samples obtained from each first text cluster, acquiring a text label corresponding to each first text cluster; acquiring a second preset number of first text samples from each first text cluster to serve as first training samples; andestablishing a first training set based on the first training samples and text labels corresponding to the first training samples, and training a text classification model. According to the text classification model training method and the text classification model training device, the texts are clustered, so that the coverage rate of a template is improved while the annotation amount is reduced,the annotation efficiency is greatly improved, and the model effect is further improved.

Description

technical field [0001] The present application relates to the technical field of text classification, in particular to a text classification model training method, device, storage medium and computer equipment. Background technique [0002] The text classification task of natural language processing requires a large amount of labeled text to train the classification model. In the prior art data labeling systems, platforms or methods, the data is sampled and then labeled, and then the labeled data is used to train the model. Combined with the training results, the next round of labeling, training, and tuning is performed. For large-scale text message classification tasks, the characteristics are that the text of text messages is huge, and the templates are various and unevenly distributed every day. If the direct sampling method is used, it will mainly cause two problems, one is the low coverage of the templates, and the other is There is a bias in the amount of data fallin...

AI Technical Summary

Problems solved by technology

For large-scale text message classification tasks, the characteristics are that the text of text messages is huge, and the templates are various and unevenly distributed every day. If the direct sampling method is used, it will mainly cause two problems, one is the low coverage of the templates, and the other is There is a deviation in the amount of data falling into each category, which causes great difficulties for text annotation and model training

In addition, due to certain differences in the templates of SMS messages every day, the model trained with fixed-day labeled data cannot be generalized to all unseen SMS templates.

Images