A text annotation noise detection method, device, storage medium and electronic equipment

Abstract

The invention discloses a text labeling noise detection method, device, storage medium and electronic equipment. The method includes: obtaining a sample data set of a model to be trained; obtaining a model prediction result by using K-fold cross-validation according to the sample data set; The output of the result calculates the trust degree of each data in the sample data set; the noise text of the sample data set is determined according to the relationship between the trust degree and the trust degree threshold. By implementing the present invention, a trust degree measurement index is proposed, and by evaluating the trust degree of data in a data sample set, noise texts are screened out according to a threshold value, which can be used for error correction of engineering data labeling. Moreover, since the detection process of the detection method has nothing to do with the neural model, there is no need to make any changes to the model; therefore, compared with the method of probability estimation and robustness, the detection method has strong integration and simplifies the tedious engineering steps; it can Provides flexible trust configuration, making the detection process and effect more controllable.

Description

technical field [0001] The present invention relates to the technical field of deep learning, in particular to a method, device, storage medium and electronic equipment for detecting text annotation noise. Background technique [0002] In intelligent conversational devices, deep learning neural networks are widely used in various tasks: such as speech classification, intent recognition-semantic slot recognition, etc. The device's analysis of the instructions issued by the user depends on the recognition results of the neural network model. [0003] Due to the influence of factors such as crowdsourcing and subjective judgment differences by multiple annotation personnel, there will be varying degrees of noise / mislabeling data when annotating the corpus for constructing the neural network training set. When training and fitting the neural network, the noise / mislabeling The data will directly affect the effect of the model, and even misidentify user intent. [0004] In the pr...

AI Technical Summary

Problems solved by technology

[0006] In view of this, the embodiments of the present invention provide a text labeling noise detection method, device, storage medium and electronic equipment to solve the technical problem in the prior art that the solution to noise / wrong labeling data is more cumbersome

Images