Hierarchical structure-based neural-network machine translation model

Abstract

The invention discloses a hierarchical structure-based neural-network machine translation model, and relates to natural language processing based on deep learning. A word alignment tool GIZA++ is used to carry out word alignment on parallel training sentence pairs, and then source language sentences are divided into clauses of monotonous translation according to punctuation and word alignment information; the above-mentioned obtained clause data are used to train a clause classifier; hierarchical structure modeling is carried out on the source language sentences of the parallel sentence pairs; and hierarchical structure decoding is carried out on target language sentences of the parallel sentence pairs. The sentences are divided into the clauses of monotonous translation, and then word-clauses-sentence hierarchical-modeling, attention mechanisms and decoding are carried out: and a bottom-layer recurrent neural network (GRU) encodes semantic representations of the clauses, an upper-layer recurrent neural network encodes information of the sentences, the bottom-layer attention mechanism is devoted to word-level alignment inside the clauses, and the upper-layer attention mechanism is devoted to clause-level alignment.

Description

technical field [0001] The present invention relates to natural language processing based on deep learning, in particular to a neural network machine translation model based on hierarchical structure. Background technique [0002] Natural language processing is an important research direction of computer science artificial intelligence. It studies how to enable effective communication between humans and computers using natural language. It is a subject that integrates linguistics, computer science, and mathematics. Among them, neural machine translation is a very important task. The existing neural network machine translation mainly includes two cyclic neural networks and a contextual semantic generation model based on the attention mechanism: use a cyclic neural network (called encoder Encoder) to learn the semantic representation of the input sentence, use another cyclic neural network The network (called Decoder) combines contextual semantic representations generated b...

AI Technical Summary

Problems solved by technology

[0004] Since it is based on the recurrent neural network, the neural network machine translation model faces two problems: 1) The problem of gradient explosion and gradient disappearance

Although the existing variant models can solve this type of problem to a certain extent, it is still difficult to model long-distance context information when modeling long sequences; 2) The attention mechanism has a large amount of calculation and there is a problem of alignment divergence

Inability to accurately compute relevant contextual semantic representations when modeling long sequences

Therefore, the existing attention mechanism-based recurrent neural machine translation model still has shortcomings, and how to better translate long sentence sequences is still a problem worthy of research

Images