Generalized reordering statistic translation method and device based on non-continuous phrase

Abstract

The invention provides a generalized reordering statistic translation method and a device based on non-continuous phrases. The device consists of a word alignment module, a language model module, a phrase extraction module, a maximum entropy classifier training module, a minimum error training module and a decoder, provides a generalized reordering module for statistical machine translation basedon phrases, introduces non-continuous phrases, combines continuous phrases and non-continuous phrases by using regulations for any continuous series in a specified script to be translated so as to acquire continuous target translations as more as possibly, and combines the reordering model with a reordering sub model simultaneously to realize local and global reordering of the phrases so as to acquire final target translations for sentences in the source language. The model can grasp local and global reordering knowledge of the phrases, and can acquire the generalization capability of the phrases through non-continuous phrases. Experiment results prove that the model improves the BLUE rating of the reordering model based on the maximum entropy and a translation model based on hierarchicalphrases by about 1.54 percent and 0.66 percent.

Description

technical field [0001] The invention relates to the technical field of natural language processing, and is a new generalized reordering statistical translation method and device based on discontinuous phrases. Background technique [0002] In statistical machine translation, phrase-based translation models have improved word-based translation models. In the phrase-based translation model, a phrase is any continuous substring without syntactic constraints, which can learn some local knowledge, such as local ordering, or translation of multi-word expressions, and the insertion and insertion of words related to local contexts. delete. However, in phrase-based translation models, key issues such as lack of discontinuous phrases, weak phrase reordering ability, and generalization ability are still not effectively addressed. [0003] In order to improve phrase-based translation models, two issues must be addressed. One is the type of phrase, which must include both continuous p...

AI Technical Summary

Problems solved by technology

The second is the reordering of phrases

However, these models are highly dependent on the parser, and the performance of translation models is also limited by the accuracy of the parser

Images