Text translation model training method and device and storage medium

Abstract

The invention provides a training method and device for a text translation model and a storage medium, and the training method comprises the steps: receiving a text training sample set containing multiple pairs of natural language texts from a client, carrying out the comparison learning based on the word level, and combining the word frequency information of the natural language texts, the model parameters of the text translation model are optimized, a final text translation model is obtained through multiple rounds of training until the loss function of the text translation model converges, and the text translation model is used for translating one natural language text into another natural language text. Since the training process can optimize the model parameters of the text translation model based on the word frequency information, semantic space representation of words with different word frequencies is optimized, the semantic expression ability of the model to the input text is enhanced, and the performance of the machine translation model is improved.

Description

technical field [0001] The present application relates to the technical field of artificial intelligence, and in particular, to a training method, device and storage medium for a text translation model. Background technique [0002] In machine translation tasks, the Transformer model is a model that uses an attention mechanism to improve the speed of model training, and is currently the most commonly used deep learning model. Usually, the Transformer model includes an encoding module and a decoding module. The input text is first passed through the encoding module to encode the text, and then the encoded data is sent to the decoding module for decoding, and the translated text is obtained after decoding. [0003] However, due to the existence of low-frequency words in the machine translation corpus, these words rarely appear at the output of the model during model training, and their representation will be optimized to push the opposite direction of the representation of mos...

AI Technical Summary

Problems solved by technology

[0003] However, due to the existence of low-frequency words in the machine translation corpus, these words rarely appear at the output of the model during model training, and their representations will be optimized to the opposite direction of most high-frequency words, which will lead to representation space The medium sample representation collapses into a narrower cone, making the boundary between different target word representations smaller and the semantic representation ability worse

The problem of semantic collapse will seriously affect the representation ability of the semantic space of the Transformer model, thereby affecting the effect of machine translation

Images