A dynamic decoding method and system for neural machine translation based on entropy

Abstract

The present invention proposes an entropy-based neural machine translation dynamic decoding method and system. By analyzing the relationship between the entropy value of a sentence and the BLEU value, it is found that the average entropy value of words in a sentence with a high BLEU value is lower than that in a sentence with a lower BLEU value. The average entropy value of words is small, and the BLEU value of sentences with low entropy value is generally higher than that of sentences with high entropy value. By calculating the Pearson coefficient between the entropy value of the sentence and the BLEU value, it is found that there is a correlation between the two. Therefore, the present invention proposes that in each time step of the decoding phase of the training process, not only must a certain probability be sampled to select real words or predicted words to obtain context information, but also to calculate the entropy value according to the prediction result of the previous time step, and then according to the entropy The value dynamically adjusts the weight of contextual information. Addresses the issue of error accumulation in neural machine translation models during decoding due to differences in contextual information between training and inference.

Description

technical field [0001] The present invention relates to the technical fields of natural language processing and neural machine translation, and in particular to an entropy-based dynamic decoding method and system for neural machine translation. Background technique [0002] Machine translation is an important task in natural language processing. In recent years, with the rise of deep neural networks, machine translation methods based on neural networks have made great progress and gradually become mainstream machine translation methods. The neural machine translation model mainly consists of three parts: encoder network, decoder network and attention network. [0003] The encoder network is responsible for encoding the source language sentences into a list of hidden vectors, and each word corresponds to a hidden vector representation. The encoder network is usually a multi-layer bidirectional RNN structure, where the forward RNN Sequentially read in the sequence of source...

AI Technical Summary

Problems solved by technology

[0008] The purpose of the present invention is to solve the problem of error accumulation caused by the difference in context information between training and inference in the decoding process of neural machine translation models

Images