A machine translation model training method based on memory bank

Abstract

The present invention discloses a machine translation model training method based on a memory bank, comprising the following steps: S1, generating triple training data based on target language retrieval; S2, generating triple training data based on a denoising language model; S3, training a machine translation model based on the generated triple training data. The present invention first uses the triple training data generated based on the denoising language model to train a machine translation pre-training model, and then uses the triple training data generated based on the target language retrieval to fine-tune the machine translation training model, thereby ensuring the accuracy of the machine translation training model and improving work efficiency and translation quality.

Description

Technical Field

[0001] The invention belongs to the technical field of machine translation, and specifically is a machine translation model training method based on a memory library. Background Art

[0002] Memory-based machine translation technology means that in the process of generating the target language from the source language to be translated, the translation results formed when translating similar source language sentences before can be used to train the translation model, provide more accurate information for the current translation, and thus further improve the quality of machine translation.

[0003] When training a memory-based machine translation model, the basic unit is generally required to be (s a , t a ,s a , t b ) such a four-tuple data, where s a is the source language sentence, t a is the target language sentence corresponding to the source language sentence, s b and the source language sentence s a Similar memory source language sentences, t a is the memory bank target language sentence corresponding to the memory bank source language sentence. But in practice, s b Generally only plays a role in a The role of matching and calculating similarity is generally only used in training. b Therefore, training a memory-based machine translation model actually requires a large amount of (s a , t a , t b ). At present, such triples are generally obtained by searching and matching in the source language in the bilingual memory corpus. The search results are generally small and the similarity is not high. Therefore, the data used to train the memory machine translation is relatively sparse, which makes it impossible to guarantee the accuracy of the machine translation training model and affects the high-quality translation of the machine translation. Summary of the invention

[0004] In order to solve the above-mentioned technical problems, the present invention provides a memory-based machine translation model training method, which constructs a large amount of triple training data without using a bilingual memory corpus, thereby training a high-quality memory machine translation model based on the data.

[0005] The technical solution adopted by the present invention is:

[0006] A memory-based machine translation model training method comprises the following steps:

[0007] S1. Generate triple training data based on target language retrieval:

[0008] S1.1. Obtain target monolingual data of parallel corpus. Any sentence in the target monolingual data is denoted as T i ;

[0009] S1.2. Select a sentence pair from the parallel corpus (S a , T a ), calculate the target language sentence T a And each sentence T in the target monolingual data i The similarity sim(T a , T i ), where the similarity sim(T a , T i )The calculation formula is as follows:

[0010]

[0011] In formula (1), T a is the target language sentence to be matched in the parallel corpus, T i is any sentence in the target monolingual data, dist(T a , T i ) indicates T a and T i The edit distance of max(T a , T i ) indicates T a and T i The larger value of the two sentence lengths, sim(T a , T i ) indicates T a and T i similarity;

[0012] S1.3, set a threshold, and compare the sentence with the largest similarity in the target monolingual data with the threshold: if the similarity is greater than the threshold, then the sentence with the largest similarity in the target monolingual data and a sentence pair in the parallel corpus (S a , T a ) combined to form (S a , T a , T b );

[0013] S1.4, combining the remaining sentence pairs in the parallel corpus according to steps S1.2 and S1.3 to obtain triple training data based on target language retrieval;

[0014] S2. Generate triple training data based on denoising language model:

[0015] S2.1. Get any sentence T in the target monolingual data x;

[0016] S2.2. Translate any sentence T x Add noise, and the sentence after adding noise is recorded as T y ;

[0017] S2.3, any sentence T before adding noise x And the sentence T after adding noise y Combined into (T x , T y );

[0018] S2.4, adding noise and combining the remaining sentences of the target monolingual data according to steps S2.2 and S2.3 to form a denoised language training data set;

[0019] S2.5. Train a denoising language model LM using the denoising language training data set, where T y is the model input, T x is the model label;

[0020] S2.6, for a sentence pair in the parallel corpus (S a , T a ) of the target language sentence T a Add noise according to step S2.2 to obtain the sentence T after adding noise. ay ;

[0021] S27, the sentence after adding noise T ay As the input of the trained denoising language model LM, the corresponding predicted sentence is obtained by using the output of the denoising language model LM, denoted as T ax ;

[0022] S2.8, predict the sentence T ax and the sentence pair (S a , T a ) combination, forming (S a , T a , T ax );

[0023] S2.9, add noise and denoise to the remaining sentence pairs in the parallel corpus according to steps S2.6, S2.7 and S2.8 to obtain triple training data generated based on the denoising language model;

[0024] S3. Training a machine translation model based on the generated triple training data, including:

[0025] S3.1, using the triple training data generated based on the denoising language model (S a , T a , T ax ) a and Tax Splice into a sentence S in ;

[0026] S3.2. Using S in and T a Train a machine translation pre-trained model MT pre , where S in is the model input, T a As model label;

[0027] S3.3. Generate triple training data using target language retrieval (S a , T a , T b ) a and T b Splice into a sentence S im ;

[0028] S3.4、S im is the input, T a As a label, continue to fine-tune the machine translation pre-trained model MT pre , and obtain the final memory-based machine translation model MT.

[0029] Furthermore, the noise adding operation in step S2.2 is specifically as follows: randomly replace a word with <mask>Or insert between words <mask>.

[0030] Furthermore, the noise adding operation in step S2.2 is specifically as follows: randomly replace a word with <mask>And insert between certain words <mask>.

[0031] The beneficial effects of the present invention are:

[0032] 1) The training method is simple and fast, making full use of the translation data in the memory bank;

[0033] 2) Using the target language to retrieve and generate triple training data and the denoising language model to generate triple training data, combined with training to obtain a machine translation model, to improve the translation quality of machine translation;

[0034] 3) First, use the triple training data generated based on the denoising language model to train a machine translation pre-training model, and then use the triple training data generated based on the target language retrieval to fine-tune the machine translation training model to ensure the accuracy of the machine translation training model and further improve work efficiency and translation quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of a memory-based machine translation model training method of the present invention. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0037] It should be noted that in the description of the present invention, it should be noted that the orientation or position relationship indicated by the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0038] Example 1

[0039] like Figure 1 As shown, a memory-based machine translation model training method includes the following steps:

[0040] S1. Generate triple training data based on target language retrieval:

[0041] S1.1. Obtain target monolingual data of parallel corpus. Any sentence in the target monolingual data is denoted as T i ;

[0042] S1.2. Select a sentence pair from the parallel corpus (S a , T a ), calculate the target language sentence T a And each sentence T in the target monolingual data i The similarity sim(T a , T i ), where the similarity sim(T a , T i )The calculation formula is as follows:

[0043]

[0044] In formula (1), T a is the target language sentence to be matched in the parallel corpus, T i is any sentence in the target monolingual data, dist(T a , T i ) indicates T a and T i The edit distance of max(T a , T i ) indicates T a and T i The larger value of the two sentence lengths, sim(T a , T i ) indicates T a and T i similarity;

[0045] S1.3, set a threshold, and compare the sentence with the largest similarity in the target monolingual data with the threshold: if the similarity is greater than the threshold, then the sentence with the largest similarity in the target monolingual data and a sentence pair in the parallel corpus (S a , T a ) combined to form (S a , T a , T b );

[0046] S1.4, combining the remaining sentence pairs in the parallel corpus according to steps S1.2 and S1.3 to obtain triple training data based on target language retrieval;

[0047] S2. Generate triple training data based on denoising language model:

[0048] S2.1. Get any sentence T in the target monolingual data x ;

[0049] S2.2. Translate any sentence T x Add noise, and the sentence after adding noise is recorded as T y ;

[0050] S2.3, any sentence T before adding noise x And the sentence T after adding noise y Combined into (T x , T y );

[0051] S2.4, adding noise and combining the remaining sentences of the target monolingual data according to steps S2.2 and S2.3 to form a denoised language training data set;

[0052] S2.5. Train a denoising language model LM using the denoising language training data set, where T y is the model input, T x is the model label;

[0053] S2.6, for a sentence pair in the parallel corpus (S a , T a ) of the target language sentence T a Add noise according to step S2.2 to obtain the sentence T after adding noise. ay ;

[0054] S27, the sentence after adding noise T ay As the input of the trained denoising language model LM, the corresponding predicted sentence is obtained by using the output of the denoising language model LM, denoted as T ax ;

[0055] S2.8, predict the sentence T ax and the sentence pair (S a , T a ) combination, forming (S a , T a , T ax );

[0056] S2.9, add noise and denoise to the remaining sentence pairs in the parallel corpus according to steps S2.6, S2.7 and S2.8 to obtain triple training data generated based on the denoising language model;

[0057] S3. Training a machine translation model based on the generated triple training data, including:

[0058] S3.1, using the triple training data generated based on the denoising language model (S a , T a , T ax ) a and T ax Splice into a sentence S in ;

[0059] S3.2. Using S in and T a Train a machine translation pre-trained model MT pre , where S in is the model input, T a As model label;

[0060] S3.3. Generate triple training data using target language retrieval (S a , T a , T b ) a and T b Splice into a sentence S im ;

[0061] S3.4、S im is the input, T a As a label, continue to fine-tune the machine translation pre-trained model MT pre , and obtain the final memory-based machine translation model MT.

[0062] As a further step in this embodiment, the noise adding operation in step S2.2 is specifically: randomly replacing a certain word with <mask>Or insert between words <mask>.

[0063] As a further step in this embodiment, the noise adding operation in step S2.2 is specifically: randomly replacing a certain word with <mask>And insert between certain words <mask>.

[0064] The working principle of the present invention is as follows: a sentence with the greatest similarity in target monolingual data is obtained by using a similarity calculation formula, and the sentence with the greatest similarity in the target monolingual data is combined with a sentence pair in a parallel corpus to obtain triple training data generated based on target language retrieval; sentences in the target monolingual data are randomly denoised to obtain a denoised language training data group, a denoised language model LM is obtained by training the denoised language training data group, sentence pairs in the parallel corpus are input into the denoised language model to obtain corresponding predicted sentences, and then the predicted sentences and the sentence pairs in the parallel corpus are combined to generate triple training data based on the denoised language model; a machine translation pre-training model is firstly trained by using the triple training data generated based on the denoised language model and the triple training data generated based on the target language retrieval, and then a machine translation model is fine-tuned by using the triple training data generated based on the target language retrieval to improve the quality of machine translation.

[0065] The embodiments described above are merely descriptions of preferred implementation modes of the present invention and are not intended to limit the scope of the present invention. Without departing from the principles and essence of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the scope of protection determined by the claims of the present invention.< / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask> < / mask>

Claims

1. A memory-based machine translation model training method, characterized in that: The steps include: S1. Generate triple training data based on target language retrieval: S1.

1. Obtain target monolingual data of parallel corpus. Any sentence in the target monolingual data is denoted as T i ; S1.

2. Select a sentence pair from the parallel corpus (S a , T a ), calculate the target language sentence T a And each sentence T in the target monolingual data i The similarity sim(T a , T i ), where the similarity sim(T a , T i )The calculation formula is as follows: In formula (1), T a is the target language sentence to be matched in the parallel corpus, T i is any sentence in the target monolingual data, dist(T a , T i ) indicates T a and T i The edit distance of max(|T a |,|T i |) indicates T a and T i The larger value of the two sentence lengths, sim(T a , T i ) indicates T a and T i similarity; S1.3, set a threshold, and compare the sentence with the largest similarity in the target monolingual data with the threshold: if the similarity is greater than the threshold, then the sentence with the largest similarity in the target monolingual data and a sentence pair in the parallel corpus (S a , T a ) combined to form (S a , T a , T b ); S1.4, combining the remaining sentence pairs in the parallel corpus according to steps S1.2 and S1.3 to obtain triple training data based on target language retrieval; S2. Generate triple training data based on denoising language model: S2.

1. Get any sentence T in the target monolingual data x ; S2.

2. Translate any sentence T x Add noise, and the sentence after adding noise is recorded as T y ; S2.3, any sentence T before adding noise x And the sentence T after adding noise y Combined into (T x , T y ); S2.4, adding noise and combining the remaining sentences of the target monolingual data according to steps S2.2 and S2.3 to form a denoised language training data set; S2.

5. Train a denoising language model LM using the denoising language training data set, where T y is the model input, T x is the model label; S2.6, for a sentence pair in the parallel corpus (S a , T a ) of the target language sentence T a Add noise according to step S2.2 to obtain the sentence T after adding noise. ay ; S27, the sentence after adding noise T ay As the input of the trained denoising language model LM, the corresponding predicted sentence is obtained by using the output of the denoising language model LM, denoted as T ax ; S2.

8. Predict the sentence T ax and the sentence pair (S a , T a ) combination, forming (S a , T a , T ax ); S2.9, add noise and denoise to the remaining sentence pairs in the parallel corpus according to steps S2.6, S2.7 and S2.8 to obtain triple training data generated based on the denoising language model; S3. Training a machine translation model based on the generated triple training data, including: S3.1, using the triple training data generated based on the denoising language model (S a , T a , T ax ) a and T ax Splice into a sentence S in ; S3.

2. Using S in and T a Train a machine translation pre-trained model MT pre , where S in is the model input, T a As model label; S3.

3. Generate triple training data using target language retrieval (S a , T a , T b ) a and T b Splice into a sentence S im ; S3.4、S im is the input, T a As a label, continue to fine-tune the machine translation pre-trained model MT pre , and obtain the final memory-based machine translation model MT.

2. The memory-based machine translation model training method according to claim 1, characterized in that: The noise adding operation in step S2.2 is specifically as follows: randomly replace a word with <mask>Or insert between words <mask> 。< / mask> < / mask> 3. The memory-based machine translation model training method according to claim 1, characterized in that: The noise adding operation in step S2.2 is specifically as follows: randomly replace a word with <mask>And insert between certain words <mask> 。< / mask> < / mask>

Images