Multi-language translation model determination method and related device

Abstract

The invention discloses a multi-language translation model determination method and a related device, which can be applied to various scenes such as artificial intelligence, natural language processing, machine learning and the like. Translation directions corresponding to the n training tasks are determined through sampling parameters, the training corpora are sampled based on the corresponding translation directions, n sample sets are obtained, the n sample sets are in one-to-one correspondence with the n training tasks, the number of the corpora included in the n sample sets is the same, and the training corpora in the same sample set belong to the same translation direction; the translation direction with a small number of training corpora can be prevented from being ignored. The source language of the training corpus in the n sample sets is input into the initial multi-language translation model, n loss functions in one-to-one correspondence with the n sample sets are obtained according to the corresponding target language corpus, then a total loss function is determined based on the n loss functions, the initial multi-language translation model is trained according to the total loss function, the model generalization degree is improved, and the multi-language translation efficiency is improved. And the translation precision in different translation directions is ensured.

Description

technical field [0001] The present application relates to the field of machine learning, and in particular, to a method and related apparatus for determining a multilingual translation model. Background technique [0002] With the wide application of machine translation technology, machine translation has gradually expanded to multilingual machine translation. Multilingual machine translation means that the same model can support translation between multiple languages, so as to meet the needs of users for multiple translation directions. [0003] However, training of a multilingual machine translation model (multilingual translation model) requires training corpora in multiple translation directions. Usually, the number of training corpora for multiple translation directions is not balanced. In order to avoid over-focusing on the translation directions with a large number of training corpora and ignoring the translation directions with a small number of training corpora dur...

AI Technical Summary

Problems solved by technology

However, oversampling may affect the space occupation of the multilingual translation model during training

Images