Fusion and compression method of multi-source neural machine translation model

Abstract

The invention relates to a fusion and compression method of a multi-source neural machine translation model, and belongs to the technical field of natural language processing application. The method comprises the following steps: firstly, constructing a Transformer-based multi-source machine translation model; secondly, using a CNN to fuse the coding information of the plurality of sources; thirdly, enabling the model to autonomously select whether to use an auxiliary source language by using a gating mechanism; and finally, according to different distances among the matrixes, carrying out quantitative compression on the matrixes in different modes. Wherein the multi-source machine translation model further comprises an encoder and a decoder, and the compression of the model comprises calculation of matrix distance and quantitative compression. Aiming at the problem of overlarge storage space caused by complex structure and multiple parameters of a multi-source model, the method for quantitatively compressing the model is also explored, and higher compression ratio and higher accuracy are obtained.

Description

technical field [0001] The invention relates to a neural machine translation method, in particular to a multi-source neural machine translation model fusion and compression method, and belongs to the technical field of natural language processing applications. Background technique [0002] In recent years, with the maturity of various tasks of natural language processing, the neural machine translation method based on deep learning has developed rapidly, and has replaced traditional statistical machine translation in many fields, becoming a new mainstream method in academia and industry. . The training of the neural machine translation model is based on large-scale parallel corpus, and it has achieved good results in the translation of English-Chinese, Japanese-Chinese and other languages ​​with rich corpus resources. However, the performance of neural machine translation is poor in the case of insufficient parallel corpora, such as translation between Chinese and minority ...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to solve the following problems in neural machine translation in the past: 1) rely on large-scale corpus for neural machine translation, the problem that the translation quality that exists when corpus is insufficient; Considering the problem of translation quality degradation caused by the poor quality of auxiliary corpus; 3) Aiming at the problem of model storage caused by the large size and many parameters of multi-source models, a fusion and compression method of multi-source neural machine translation models is proposed, using CNN fusion Multi-source neural machine translation solves the problem of insufficient corpus, uses gating mechanism to solve the problem of poor quality auxiliary corpus, and selects different quantization and compression methods according to different matrix distances to solve the problem of model storage

Images