Multi-language neural machine translation method of automatic editing-after-translation mechanism

Abstract

The invention discloses a multi-language neural machine translation method of an automatic editing-after-translation mechanism, and relates to the field of deep learning and natural language processing. In order to solve the problem that a translation error is generated when an existing multi-language neural machine translation method is applied to a translation task, the invention provides a method which takes a multi-source Transform as a basic framework and comprises the following steps of: firstly, processing a first language text by using a neural machine translation model to generate a second language text; secondly, the second language text is processed in a back translation mode to generate a training text of the first language, and finally a training instance containing the first language reference true value text and the first language training text is automatically generated for training. The multi-language neural machine translation method of an automatic translation-after-translation mechanism is made according to the three function modularization. According to the method, errors generated by the multi-language neural machine translation model can be remarkably corrected, and the multi-language neural machine translation performance is improved.

Description

Technical Field

[0001] The invention relates to a multilingual neural machine translation method with an automatic post-editing mechanism, belonging to the technical field of natural language processing. Background Art

[0002] Machine translation can replace words in texts of different languages ​​and translate from the first language to the second language. However, phrases in the source language text may have specific meanings, and simply replacing words for words will result in inaccurate translations. Manual post-editing will incur expensive labor costs. This has created a major obstacle to the popularization of machine translation models.

[0003] Neural machine translation uses artificial neural networks to predict the possible ordering of words, matching knowledge of different languages ​​through a shared semantic space, and can effectively utilize the overall input of the text to translate entire phrases or sentences at a time, rather than individual characters or words. However, in existing multilingual neural machine translation models, sharing language knowledge will introduce interference between languages, and this interference at the network level and parameter level also affects the translation effect.

[0004] Post-editing is an important part of current neural machine translation technology, which requires correcting inappropriate parts of the translation. However, current post-editing methods have the problems of low accuracy, poor effect, and excessive reliance on manual assistance during the editing process. To address this problem, the present invention proposes a multilingual neural machine translation method with an automatic post-editing model. Summary of the invention

[0005] In view of the limitations of current multilingual neural machine translation models and post-editing, the present invention provides an automatic post-editing method, which reduces labor expenses and equipment expenses by automatically generating training sets, can explicitly distinguish the grammatical differences between the source language and the target language, and then perform differentiated modeling for different grammars, thereby taking cross-language information into account when outputting translation results, improving the accuracy of post-editing, and further improving the translation performance of the multilingual neural machine translation model.

[0006] The method described in the present invention is to improve the translation performance of the neural machine translation model by using an automatic post-editing (APE) mechanism to solve mistranslations in one or more words generated during the translation process for the translated text generated by the existing neural machine translation model.

[0007] The technical solution of the present invention is a multi-language neural machine translation method with automatic post-editing mechanism, the flow chart is as follows Figure 1 The steps include:

[0008] S1. Use the generated neural machine translation model to translate the real text in the first language into the second text language;

[0009] S2, back-translating the generated second language text to generate a training text in the first language;

[0010] S3, stores the first language real text and training text as the data set of the APE model;

[0011] S4, using the automatically generated data set to train the APE model;

[0012] S5. Use the trained multilingual neural machine translation model with automatic post-editing mechanism to train the test set, obtain the translation results and compare them with the translation results of the multilingual neural machine translation model;

[0013] Further generalization of S1 includes the following three steps:

[0014] Step 1: Train a multilingual neural machine translation model based on the multi-source Transformer framework.

[0015] Step 2: Based on the real text in the first language, the existing first language text dataset is translated into the second language text in an unsupervised manner.

[0016] Step 3: Based on the real text in the first language, the existing first language text dataset is processed in an unsupervised manner, and the existing first language text dataset is translated into all language texts supported by the current model, so as to expand the automatically generated training set and improve the generalization ability of the network.

[0017] Further generalization of S2 includes:

[0018] The generated language text is used to translate in an unsupervised manner to generate training text in the first language. Automatically generated training text can save device resources and improve the accuracy and robustness of the model.

[0019] Further summarizing S3 includes:

[0020] In the dataset generation stage, random noise processing is performed on the real text to supplement the training set. By adding one or more words to the text, deleting one or more words, reordering the text content, and adding noise to the real text to generate training text.

[0021] Further, the S4 includes:

[0022] In the training phase of the proposed automatic post-translation editing model. First, the automatically generated dataset is input into the APE model for training. A multilingual APE model can be generated for multilingual training texts. This model can reduce the training iterations in training a single APE model, thereby freeing up storage space. Secondly, the gender errors of words are distinguished, and the inherent gender of the words retained in the real text language is compared with the gender of the words in the training text. The APE model can learn grammatical gender through context and avoid translation errors caused by grammatical gender in some languages. Finally, for the text templates and structured datasets determined by the template engine, the APE model can generate processed natural language output, avoid incorrect application of templates and improve the efficiency of neural translation.

[0023] The beneficial effects of the present invention are:

[0024] 1. The present invention proposes a multilingual neural machine translation method based on a multi-source Transformer framework. This method can effectively solve the problem of being unable to retain language features in neural machine translation and improve translation performance.

[0025] 2. The present invention proposes a method for constructing APE, which can effectively solve the errors caused by translation between languages ​​with grammatical gender and languages ​​without grammatical gender, making it possible to explicitly apply language feature knowledge in multilingual neural machine translation models.

[0026] 3. The present invention effectively utilizes the role of engine templates in translation. By introducing the APE method, the model can flexibly process text templates and structured data sets, effectively alleviating the problems caused by the template engine in natural language output.

[0027] 4. The present invention effectively utilizes the existing data sets to realize the automatic generation of APE method data sets, solves the problem that the existing post-editing method data sets need to occupy a large amount of human resources, and improves the resource utilization of the multilingual neural machine translation model.

[0028] 5. The present invention solves the problem of poor representation ability of existing multilingual neural machine translation methods in translation tasks and improves the multilingual neural translation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to clearly illustrate the technical solution of the present invention, the drawings required for use in the existing technical description will be introduced below. Ordinary technicians in this field can also obtain other drawings based on these drawings without paying any creative work.

[0030] Figure 1 It is an overall flow chart for implementing the technical solution proposed in the present invention;

[0031] Figure 2 A flowchart generated for a data set disclosed in accordance with the present invention;

[0032] Figure 3 A diagram of the multi-source Transformer architecture disclosed in the present invention;

[0033] Figure 4 A flowchart for generating training text according to the present invention;

[0034] Figure 5 A flowchart of random noise generation according to the present invention;

[0035] Figure 6 A flowchart of a training method for an automatic post-editing model disclosed in the present invention;

[0036] Figure 7 A flowchart for use of a multilingual neural machine translation model according to an automatic post-editing model method disclosed in the present invention. DETAILED DESCRIPTION

[0037] The following is a clear and complete description of the technical solution for implementing the present invention in combination with the drawings in the embodiments of the present invention. What is described is a part but not all of the embodiments of the present invention. Based on the embodiments in the invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] The entire process of the present invention is divided into two stages: data set preparation and training.

[0039] 1. Dataset Preparation Phase

[0040] like Figure 2 Shown

[0041] In the data set preparation phase, the generated neural machine translation model is first used to process the real text in the first language to obtain the text in the second language. The obtained second language text is then back-translated through the existing model to obtain the first language training text. The training text and the real text are stored in the training set as a set of training data. Finally, the system determines whether to continue the operation of generating the training set to complete the automatic generation of the data set.

[0042] (1) Multi-source Transformer training under ciphertext data enhancement

[0043] like Figure 3 shown.

[0044] The neural machine translation model is a translation model generated by multi-source Transformer training under ciphertext data enhancement. This method generates ciphertext by inserting plaintext, thereby retaining language features and achieving data enhancement. Using the multi-source Transformer model, one encoder is used for source data and another encoder is used for language features. The encoders share parameters and embedding matrices, and the source data is expanded three times through multi-source learning to capture specific information in the language and improve translation performance.

[0045] (2) Automatically generate training text in an unsupervised manner

[0046] like Figure 4 shown.

[0047] First, the neural machine translation model is used to process the real text of the source language, and the intermediate text that may produce translation errors is back-translated to obtain the training text that may contain random noise, and the training text and the real text are added to the training set as a set of training data. The generation process of the existing data set must transmit the relevant data to the client for manual review, and the reviewer provides annotations to generate the training set of the APE model. The present invention reduces the resources consumed by the APE model before training by automatically generating training sets, and can also generate a large amount of training data from multiple languages, thereby improving the accuracy and robustness of the APE model in generating edited text during training. Multilingual data sets can reduce the amount of automatically generated training data, and can generate APE model data sets in all languages ​​supported by the multilingual neural machine translation model.

[0048] (3) Check the context of the real text

[0049] Determine the context of the real text and ensure that the processed training text can share the same context. This can provide sufficient context information when training the model and improve the model's ability to identify grammatical errors. By checking the context of the real text and retaining it, the training text is shared to ensure that the training text has the real context.

[0050] (4) Use random noise to process real text

[0051] like Figure 5 Shown

[0052] Use random noise to process real text to ensure the richness of the training set. Random noise processing includes deleting, adding, and modifying the content of real text. Data processed with random noise can improve the stability and efficiency of model training and accelerate the convergence of the model.

[0053] Table 1 Effect of random noise processing on real text

[0054]

[0055] In the first example, the real text "Let's go to Beijing together" is subjected to random noise processing by adding the noise text "with him" to obtain the noisy training text "Let's go to Beijing with him". The real text and the noisy training text are stored as a training set.

[0056] In the second example, the real text "Let's go to Beijing together" is subjected to random noise processing by deleting the text "together" to obtain the noisy training text "Let's go to Beijing", and the real text and the noisy training text are stored as a training group.

[0057] In the third example, the real text "Let's go to Beijing together" is subjected to random noise processing by modifying the text order of "Beijing" and "we" to obtain the noisy training text "Beijing together we go", and the real text and the noisy training text are stored as a training group.

[0058] 2. Training Phase

[0059] like Figure 6 Shown

[0060] The APE model is trained using the automatically generated training set, which contains real text and training text with random noise. The system uses the training text as the input of the APE model, obtains context and edits the text through the APE model, and obtains the initial prediction output. The predicted output is compared with the real text to obtain new weight parameters. The obtained parameters are back-propagated to the APE model to update the model weights.

[0061] To generate a multilingual APE model, the APE model is trained with a small number of automatically generated training sets in different languages. The multilingual APE model can identify languages ​​with grammatical gender and implement grammatical correction in post-editing.

[0062] Table 2 shows a comparison of word meanings between a group of languages ​​with grammatical gender and languages ​​without grammatical gender.

[0063]

[0064] In the first group of examples, the French words “un aide” and “une aide” with grammatical gender are translated as “assistant” in Chinese without grammatical gender. In the post-editing of the APE model from the source language with grammatical gender to the target language without grammatical gender, the translation results of these two words were modified to “male assistant” and “female assistant”.

[0065] In the second group of examples, the French words “le chat noir” and “la rose noire” with grammatical gender are translated as “black cat” in Chinese without grammatical gender. In the post-editing of the APE model from the source language with grammatical gender to the target language without grammatical gender, the translation results of these two words were modified to “black male cat” and “black male cat”.

[0066] The multilingual APE model avoids errors in translation from a source language with grammatical gender to a target language without grammatical gender, from a source language without grammatical gender to a target language with grammatical gender, and from a source language with grammatical gender to a target language with grammatical gender.

[0067] After each group of training is completed, the system determines whether there is unprocessed training group data, and repeatedly trains the APE model using the remaining training group data. When the training threshold is reached, the system can alternately execute the trained group or end the APE model training by itself and store the final model weight.

[0068] 3. Testing Phase

[0069] like Figure 7 Shown

[0070] Using multi-source Transformer as the framework of the neural machine translation model can preserve language features, improve translation performance, and enhance the effect of automatically generated training sets.

[0071] The real text is used as input data to pass into the neural machine translation model. The output data obtained by the neural machine translation model may contain grammatical errors. The data output by the model is passed into the APE model as input data to obtain the post-edited text. The APE model will perform grammatical correction on the errors in the engine template and generate edited text with correct grammar by processing natural language. The text will be displayed in the client as result data.

[0072] Table 3 shows a translation error corrected by the template engine based on the method provided by the present invention.

[0073]

[0074] The example shows that according to the source language texts "There are three dogs in the garden" and "There is a dog in the garden" input by the user, the neural machine translation model references the template engine, and the template engine determines a text template "There are *dogs in the #" and two sets of structured data "*=3; #=garden" and "*=1; #=garden" in response to the user input. However, the introduction of the second set of structured data into the text template will obviously cause errors. The APE model generates the edited text "There is one dog in the garden" by processing the natural language output of "There are one dogs in the garden", ensuring that this method can give the correct translation result.

[0075] It can be seen from the above technology that the present invention introduces post-editing to ensure the accuracy of translation results while achieving neural machine translation results. When generating translation results, APE distinguishes between source language segments and target language segments, and then performs differentiated modeling for different language segments, thereby taking cross-language information into account when outputting translation results, improving post-editing performance, and improving the accuracy of translation results.

[0076] The above embodiments of the present invention are only used to explain and illustrate the technical solutions of the present invention, rather than to limit the present invention. Although the present invention has been described in detail with reference to specific embodiments, it should be understood by those skilled in the art that various forms of modifications, adjustments and equivalent transformations may be made to the present invention without departing from the spirit and scope of the present invention. These modifications and transformations should be deemed to be within the scope of protection of the present invention and protected by the claims of the present invention.

Claims

1. A multilingual neural machine translation method with an automatic post-editing mechanism, characterized in that: include: S11: Automatically generate data sets and preprocess data: Use the generated multilingual neural machine translation model to process the real text of the target language text to generate an intermediate text, and then back-translate the intermediate text to obtain a noisy training set; S12: Multilingual neural machine translation model training with automatic post-editing mechanism: A contrastive learning method based on real text is proposed, which uses the dataset automatically generated by S11 and the real text to construct contrastive learning for model weight optimization; S13: Perform multilingual neural machine translation: Deploy the trained multilingual neural machine translation model with automatic post-editing mechanism as a multilingual neural machine translation system to implement post-editing of multilingual neural machine translation; S14: The test text set is trained using a multilingual neural machine translation model with an automatic post-editing mechanism, and finally a visualization result is obtained, which is then compared with the visualization result of the multilingual neural machine translation model. The multilingual neural machine translation method with automatic post-editing mechanism according to claim 1, characterized in that S11 comprises: S21: Train a multilingual neural machine translation model based on a multi-source Transformer framework to improve initial translation results; S22: The real text in the first language is translated in an unsupervised manner through a multilingual neural machine translation model to generate an intermediate text containing random noise; S23: The intermediate text is back-translated in an unsupervised manner through a multilingual neural machine translation model to generate a first language training text containing random noise, and the training text and the real text are used as a training group to automatically generate a dataset; S24: using randomly generated noise to process the real text to generate a training text containing noise, and using the training text containing noise and the real text as a training group to automatically generate a data set; S25: Expand the dataset with multiple languages ​​supported by the multilingual neural machine translation model to generate a multilingual APE model to ensure that all languages ​​supported by the multilingual neural machine translation model can be automatically translated and edited; S26: Automatically generate a large number of data sets in an unsupervised manner, reduce the resources required for computing, improve resource utilization, and improve the accuracy and robustness of the post-edited text generated by the APE model during the training process; S27: Preserve the context of real text to ensure that the training set uses real context.

2. The multilingual neural machine translation method with automatic post-editing mechanism according to claim 1, characterized in that: The S12 includes: S31: Train the APE model, use the training set generated in S11 as input, obtain preliminary prediction output, use the difference comparison method based on the real text to obtain new weight parameters, and propagate them to the APE model to update the model weights; S32: For translation between a first language containing grammatical gender and a second language not containing grammatical gender, or between a first language containing grammatical gender and a second language containing grammatical gender, the APE model identifies and implements grammatical correction in post-translation editing.

3. The multilingual neural machine translation method with automatic post-editing mechanism according to claim 1, characterized in that: The S13 includes: S41: Use the template engine provided by the multilingual neural machine translation model to perform natural language processing and output the standard translated text after correcting grammatical errors.

4. The multilingual neural machine translation method with automatic post-editing mechanism according to claim 1, characterized in that: The S14 includes: S51: Adopting a multi-source Transformer architecture in the traditional neural machine translation model, using dual encoders to preserve language features and improve translation performance; S52: Automatically generate training sets for APE models in traditional neural machine translation models, using unsupervised methods to improve resource utilization and increase the efficiency and speed of training set generation; S53: Add the APE model to the traditional neural machine translation model, take into account cross-language information, and improve the accuracy of post-translation compilation, thereby improving the translation performance of the multilingual neural machine translation model.

Images