Method and device used for training language model according to corpus sequence

Abstract

The invention aims to provide a method and device used for training a language model according to a corpus sequence. The corpus sequence used for training the target language model is acquired, initial order information of the target language model is set as the current training order, and the following operations are carried out through iteration in combination with the highest order information of the target language model till the current training order exceeds the highest order information, wherein the operations include that according to the current training order, a smoothing algorithm corresponding to the target language model is determined; according to the corpus sequence, the target language model is trained through the smoothing algorithm to acquire an updated target language model; the current training order is updated. In comparison with the prior art, the method and device have the advantages that different smoothing algorithms are adopted for language models with different orders according to the characteristics of the language models with different orders, the advantages of different smoothing algorithms are played, and thus better model establishment effects can be achieved. Furthermore, the method and device can be combined with voice identification, and thus the accuracy of the voice identification can be improved.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to a technology for training a language model according to a corpus sequence. Background technique [0002] Language model modeling mainly obtains a statistical model for language by counting the grammatical distribution in the text corpus, which is used to describe the probability that a text string becomes a natural language. In language model training, in order to describe the zero-order grammar with a certain probability, a smoothing algorithm is usually used to smooth part of the probability of high-frequency grammar to low-frequency grammar by using the idea of ​​"robbing the rich and helping the poor". [0003] At present, there are many smoothing algorithms for language models, such as katz smoothing algorithm, KN smoothing (Kneser-Ney smoothing) algorithm, plus one smoothing algorithm, wb smoothing (Witten-Bell smoothing) algorithm, etc. The more commonl...

AI Technical Summary

Problems solved by technology

However, in language model training, the characteristics of different order language models are not the same, and using the same smoothing algorithm for different order grammars cannot achieve the best results

Images