Text data detection method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, in particular to a text data detection method and device, electronic equipment and a storage medium. The method comprises: obtaining a sample data set; inputting the text data sample into a pre-trained text classification model to obtain classification probabilities of a plurality of preset text categories, and determining a first probability statistical value of the text data sample according to the classification probabilities; selecting a part of text data samples from the sample data set, and uniformly replacing sample tags of the part of text data samples with alternative sample tags; inputting the text data samples with the alternative sample labels into a text classification model to obtain classification probabilities of a plurality of text categories, and determining second probability statistical values of the text data samples with the alternative sample labels according to the classification probabilities; and determining whether the sample label of the text data sample is correctly labeled or not according to the numerical relationship between the first probability statistical value and the second probability statistical value. By adopting the method provided by the invention, the accuracy of the sample labels of the text data samples can be improved.

Description

technical field [0001] The present application relates to the field of computer technology, and in particular to a text data detection method, device, electronic equipment and storage medium. Background technique [0002] Text classification is a very important module in text processing and is widely used. The quality of labeled data in text classification is very important, and it is related to the actual effect of the classification model. In actual training, a large amount of labeled data is required. If incorrectly labeled data is used to participate in classification model training, the classification accuracy rate will decrease, and classification errors will increase, which will affect the overall classification performance. [0003] General data labeling is done by man-made design rules, and then by human-operated computers. But sometimes the data set is very large, and the data formulated for some special situations or scenarios is special or important, and needs ...

AI Technical Summary

Problems solved by technology

[0004] In order to solve the problem in the prior art that the cost of incorrectly marked text data due to human inspection is too high, the present application provides a text data detection method, device, electronic equipment and storage medium

Images