Multi-modal machine translation data enhancement method based on image description generation

A technology of image description and machine translation, applied in natural language translation, neural learning methods, biological neural network models, etc., can solve the problems of insufficient data enhancement technology, data scarcity, etc., achieve smooth pseudo-data and improve robustness , the effect of improving translation performance
CN112257465AActive Publication Date: 2021-01-22沈阳雅译网络技术有限公司
3 Cites 7 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
沈阳雅译网络技术有限公司
Publication Date
2021-01-22

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a multi-modal machine translation data enhancement method based on image description generation, which comprises the following steps of: training an attention mechanism-based image description generation model by using pre-trained image coding information and corresponding image description; encoding and decoding pictures in the existing multi-modal training data by using the trained image description generation model to generate a corresponding source language image description text; translating the generated source language image description text into a target language, and constructing pseudo data; and adding the constructed pseudo data into the multi-modal training data, fusing the picture information in the multi-modal training data with the source language description information, sending the fused information into a multi-modal machine translation model, and generating a target language translation assisted by the image context information in an autoregressive manner. The diversity of the pseudo data is enriched, the performance can be improved from knowledge refinement, and compared with a common data enhancement method adopting a random replacementmode and the like, the invention has great advantages.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to a machine translation data enhancement technology, in particular to a multimodal machine translation data enhancement method based on image description generation. Background technique

[0002] Machine Translation (MT for short) is an experimental discipline that uses computers to translate between natural languages. Using machine translation technology, a source language can be automatically converted into a target language. Machine translation, as a key technology to eliminate barriers in people's cross-language communication, has always been an important part of natural language processing research. Compared with human translation, machine translation is more efficient and lower cost, which is of great significance for promoting national unity and cultural exchanges. Machine translation technology can be summarized as two methods based on rationalism and methods based on empiricism. Since it was proposed in the 1940s, mach...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More