Deep active learning text classification method based on pre-training model

Abstract

The invention provides a deep active learning text classification method based on a pre-training model. The method comprises the steps of: combining the pre-training model trained on a large number ofgeneral texts and utilizing the pre-training model to obtain semantic codes of the texts as input features of a classifier; then constructing a classifier to start training on an initial training set, based on the initial model and the to-be-marked sample selection strategy and the data supplement strategy provided by the invention, carrying out iteration continuously under participation of manual marking until the maximum number of iterations is reached or the marking budget is exhausted, and subsequently embedding the obtained model into a specific product to obtain a final product; and performing inference procedures. By means of the mode, high recognition accuracy can be obtained with low data labeling cost under the condition that diversity of training samples is guaranteed.

Description

technical field [0001] The invention relates to the technical field of automatic text classification, in particular to a deep active learning text classification method based on a pre-trained model. Background technique [0002] At present, in order to automatically classify texts better, various types of text classification methods have been proposed one after another. For example, the patent with the publication number CN110263173A provides a machine learning method and device for quickly improving the performance of text classification, but this method uses a threshold to divide the samples into automatically generated tags and manual tags when determining samples that need manual labeling. Labeling two parts, when the threshold is set higher, it is easy to increase the cost of labeling, and when the threshold is lower, it will increase the risk of introducing wrong labels (automatic labeling). The patent with the publication number CN107169001A discloses a text classifi...

AI Technical Summary

Problems solved by technology

For example, the patent with the publication number CN110263173A provides a machine learning method and device for quickly improving the performance of text classification, but this method uses a threshold to divide the samples into automatically generated tags and manual tags when determining samples that need manual labeling. Two parts are marked. When the threshold is set higher, it is easy to increase the cost of labeling. When the threshold is lower, it will increase the risk of introducing wrong marks (automatic marks).

The patent with the publication number CN107169001A discloses a text classification model optimization method based on crowdsourcing feedback and active learning, but this method only selects the most uncertain samples in the model for labeling and then iterative training is not suitable for deep models. , and only using uncertain samples for active learning labeling is likely to cause a lack of diversity in the final training samples

The patent with the publication number CN110826470A provides a method for identifying the left and right eyes of the fundus image based on deep active learning, but the method selects a fixed value (0.4-0.6) for the threshold of the active learning marked samples, which easily causes the active learning process to be in the initial stage. In the process of several iterations, the labeled sample data is huge, which leads to an increase in the cost of labeling

Images