Punctuation mark labeling model and training method and device thereof, and storage medium

Abstract

The invention discloses a punctuation mark labeling model and a training method and device thereof, and a storage medium. The method comprises the steps of obtaining a first training corpus containingpunctuation marks; inputting the first training corpus into a preset neural network model with a time sequence for pre-training to obtain a pre-trained language sub-model; obtaining a second trainingcorpus containing punctuation marks, removing the punctuation marks from the second training corpus, and labeling corresponding label combinations at front and rear word segmentation units of the removed punctuation marks to obtain a third training corpus; inputting the third training corpus into an initial punctuation mark labeling model for transfer learning training, obtaining a trained punctuation mark labeling model, the punctuation mark labeling model comprising a pre-trained language sub-model, and the third training corpus comprising a punctuation-free text set and a label sequence set. According to the scheme, a large amount of training data does not need to be manually labeled, the recall rate is increased, and the obtained punctuation mark labeling model has good generalizationability and universality.

Description

technical field [0001] Embodiments of the present invention relate to the technical field of computer natural language processing, and in particular to a punctuation mark labeling model and its training method, equipment, and storage medium. Background technique [0002] Existing punctuation recovery schemes usually use sequence labeling, which is mainly applied to recovering punctuation marks in text obtained from speech recognition. Generally, only simple punctuation marks can be added, such as commas and periods. The punctuation added in this way, on the one hand, has low accuracy and poor generalization ability; on the other hand, the marked punctuation is poor in richness, resulting in a poor reading experience. [0003] With the continuous development of deep learning technology, the trained and learned neural network model can be used to predict the punctuation marks of the text obtained from speech recognition and improve the accuracy. However, the learning and trai...

AI Technical Summary

Problems solved by technology

However, the learning and training of the neural network model requires manual labeling of a large amount of training data, and the recall rate of the neural network model obtained by the existing training method is low

In addition, the punctuation annotation model adopted has weak generalization ability and poor versatility

Images