Machine Translation Quality Evaluation Method, Device and Storage Medium

Abstract

The present disclosure relates to the field of computer technologies, and particularly to a method, an apparatus, and a storage medium for machine translation quality assessment. The method includes: generating pseudo-parallel corpora according to pre-configured original parallel corpora, where the pseudo-parallel corpora include a plurality of pseudo-parallel sentence pairs, each pseudo-parallel sentence pair includes a source monolingual sentence and a corresponding pseudo-target monolingual sentence, and the similarity degree of the data distribution of the pseudo-target monolingual sentence and the data distribution of the real machine translation is greater than a similarity threshold; training an original quality assessment model according to the pseudo-parallel corpora to obtain a target quality assessment model. In the embodiments of the present disclosure, a computer device generates pseudo-parallel corpora according to pre-configured original parallel corpora. Since the similarity degree of the data distribution of the pseudo-target monolingual sentence and the data distribution of the real machine translation is greater than the similarity threshold, after training the original quality assessment model according to the pseudo-parallel corpora, the obtained target quality assessment model has a significant improvement in translation quality.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, and a storage medium for machine translation quality assessment. Background Art

[0002] Automatic machine translation quality assessment mainly falls into two directions. The first direction is machine translation quality assessment with a reference translation, that is, given a standard translation, i.e., a reference translation, the quality of the output translation is evaluated by comparing the similarity between the output translation of the machine translation system and the reference translation. The second direction is machine translation quality assessment without a reference translation, that is, a method for directly evaluating the quality of the translation result given the source text and the target text of the machine translation system, without relying on a reference translation. The machine translation quality assessment method involved in this article is a machine translation quality assessment method without a reference translation.

[0003] In the related art, the commonly used method for machine translation quality assessment without a reference translation includes the following steps: In the pre-training stage, a feature extractor is trained on a large-scale original parallel corpus, and an additional quality assessment model is trained on the labeled translation quality assessment data. When performing the translation quality assessment task, according to the pre-trained feature extractor, the translation quality assessment data to be evaluated is extracted as a set of high-dimensional representations; according to the extracted high-dimensional representations, the pre-trained quality assessment model is used to evaluate the quality of the translations in the translation quality assessment data.

[0004] However, in the above method, due to the large difference between the data distribution of the original parallel corpus and the data distribution of the real translation quality assessment data, the knowledge of the original parallel corpus cannot be fully applied to the translation quality assessment task, resulting in poor effects of the current machine translation quality assessment. Summary of the Invention

[0005] In view of this, the present disclosure provides a method, an apparatus, and a storage medium for machine translation quality assessment.

[0006] The technical solution includes:

[0007] According to one aspect of the present disclosure, there is provided a method for machine translation quality assessment, which is used in a computer device, and the method includes:

[0008] Generating a pseudo-parallel corpus according to a pre-configured original parallel corpus, the pseudo-parallel corpus includes a plurality of pseudo-parallel sentence pairs, the pseudo-parallel sentence pairs include source monolingual sentences and corresponding pseudo-target monolingual sentences, and the similarity degree between the data distribution of the pseudo-target monolingual sentences and the data distribution of the real machine translation translations is greater than a similarity threshold;

[0009] The original quality assessment model is trained according to the pseudo-parallel corpus to obtain a target quality assessment model, and the target quality assessment model is used to perform machine translation quality assessment on the sentence pairs to be evaluated.

[0010] In a possible implementation manner, the original parallel corpus includes a plurality of original parallel sentence pairs, and the original parallel sentence pairs include source monolingual sentences and corresponding correct target monolingual sentences; the generating the pseudo-parallel corpus according to the pre-configured original parallel corpus includes:

[0011] For each of the original parallel sentence pairs in the original parallel corpus, a pseudo-target monolingual sentence is generated by calling a generator model, and the pseudo-target monolingual sentence is different from the correct target monolingual sentence;

[0012] According to the source monolingual sentence and the pseudo-target monolingual sentence, the pseudo-parallel sentence pair is obtained.

[0013] In another possible implementation manner, the generating the pseudo-target monolingual sentence by calling a generator model for each of the original parallel sentence pairs in the original parallel corpus includes:

[0014] For each of the original parallel sentence pairs in the original parallel corpus, the source monolingual sentence is encoded by calling the generator model;

[0015] At least one word in the correct target monolingual sentence is masked;

[0016] According to the encoded source monolingual sentence and the masked correct target monolingual sentence, the at least one masked word is reconstructed to obtain the pseudo-target monolingual sentence.

[0017] In another possible implementation manner, the generator model is an auto-encoding structure based on a source encoder and a masked language model encoder, and there is a one-to-one correspondence relationship between multiple words in the pseudo-target monolingual sentence and multiple words in the correct target monolingual sentence.

[0018] In another possible implementation manner, after the at least one masked word is reconstructed according to the encoded source monolingual sentence and the masked correct target monolingual sentence to obtain the pseudo-target monolingual sentence, it further includes:

[0019] According to the pseudo-target monolingual sentence and the correct target monolingual sentence, a marking parameter corresponding to the pseudo-target monolingual sentence is generated;

[0020] Among them, the marking parameter is used to indicate whether each word in the pseudo-target monolingual sentence is obtained by machine translation, and / or the proportion of the number of words obtained by machine translation in the pseudo-target monolingual sentence to the total number of words, where the total number of words is the total number of words in the pseudo-target monolingual sentence.

[0021] In another possible implementation manner, training the original quality evaluation model according to the pseudo-parallel corpus to obtain a target quality evaluation model includes:

[0022] For each pseudo-parallel sentence pair in the pseudo-parallel corpus, calling a discriminator model to obtain a training result, where the discriminator model is a model based on a source encoder and a target encoder;

[0023] Comparing the training result with the marking parameter corresponding to the pseudo-parallel sentence pair to obtain a calculation loss, where the calculation loss is used to indicate the error between the training result and the marking parameter;

[0024] Training the target quality evaluation model according to the calculation losses corresponding to multiple pseudo-parallel sentence pairs respectively.

[0025] In another possible implementation manner, after training the original quality evaluation model according to the pseudo-parallel corpus to obtain a target quality evaluation model, it further includes:

[0026] Obtaining a sentence pair to be evaluated, where the sentence pair to be evaluated includes a source monolingual sentence and a target monolingual sentence;

[0027] Calling the trained target quality evaluation model according to the sentence pair to be evaluated to obtain an evaluation result.

[0028] According to another aspect of the present disclosure, there is provided a machine translation quality evaluation device for a computer device, and the device includes:

[0029] A generation module, configured to generate a pseudo-parallel corpus according to a pre-configured original parallel corpus, where the pseudo-parallel corpus includes multiple pseudo-parallel sentence pairs, the pseudo-parallel sentence pairs include a source monolingual sentence and a corresponding pseudo-target monolingual sentence, and the similarity degree of the data distribution of the pseudo-target monolingual sentence to the data distribution of a real machine translation target is greater than a similarity threshold;

[0030] A training module, configured to train an original quality evaluation model according to the pseudo-parallel corpus to obtain a target quality evaluation model, where the target quality evaluation model is used to perform machine translation quality evaluation on a sentence pair to be evaluated.

[0031] In a possible implementation, the original parallel corpus includes a plurality of original parallel sentence pairs, and each original parallel sentence pair includes a source monolingual sentence and a corresponding correct target monolingual sentence; the generating module is further configured to:

[0032] For each original parallel sentence pair in the original parallel corpus, generate the pseudo target monolingual sentence by calling a generator model, and the pseudo target monolingual sentence is different from the correct target monolingual sentence;

[0033] Obtain the pseudo parallel sentence pair according to the source monolingual sentence and the pseudo target monolingual sentence.

[0034] In another possible implementation, the generating module is further configured to:

[0035] For each original parallel sentence pair in the original parallel corpus, encode the source monolingual sentence by calling the generator model;

[0036] Mask at least one word in the correct target monolingual sentence;

[0037] Reconstruct the at least one masked word according to the encoded source monolingual sentence and the masked correct target monolingual sentence to obtain the pseudo target monolingual sentence.

[0038] In another possible implementation, the generator model is an auto-encoding structure based on a source encoder and a masked language model encoder, and there is a one-to-one correspondence between multiple words in the pseudo target monolingual sentence and multiple words in the correct target monolingual sentence.

[0039] In another possible implementation, the generating module is further configured to:

[0040] Generate a marking parameter corresponding to the pseudo target monolingual sentence according to the pseudo target monolingual sentence and the correct target monolingual sentence;

[0041] Wherein, the marking parameter is used to indicate whether each word in the pseudo target monolingual sentence is obtained by machine translation, and / or the proportion of the number of words obtained by machine translation in the pseudo target monolingual sentence to the total number of words, and the total number of words is the total number of words in the pseudo target monolingual sentence.

[0042] In another possible implementation, the training module is further configured to:

[0043] For each pseudo parallel sentence pair in the pseudo parallel corpus, call a discriminator model to obtain a training result, and the discriminator model is a model based on a source encoder and a translation encoder;

[0044] Compare the training result with the token parameters corresponding to the pseudo-parallel sentence pairs to obtain a calculated loss, where the calculated loss is used to indicate the error between the training result and the token parameters;

[0045] Train the target quality evaluation model according to the calculated losses corresponding to the respective pseudo-parallel sentence pairs.

[0046] In another possible implementation, the apparatus further includes an acquisition module and a call module;

[0047] The acquisition module is configured to acquire a sentence pair to be evaluated, where the sentence pair to be evaluated includes a source monolingual sentence and a target monolingual sentence;

[0048] The call module is configured to call the trained target quality evaluation model according to the sentence pair to be evaluated to obtain an evaluation result.

[0049] According to another aspect of the present disclosure, there is provided a computer device, where the computer device includes: a processor; and a memory for storing processor-executable instructions;

[0050] Wherein, the processor is configured to:

[0051] Generate a pseudo-parallel corpus according to a pre-configured original parallel corpus, where the pseudo-parallel corpus includes a plurality of pseudo-parallel sentence pairs, the pseudo-parallel sentence pairs include a source monolingual sentence and a corresponding pseudo-target monolingual sentence, and the similarity degree between the data distribution of the pseudo-target monolingual sentence and the data distribution of the real machine translation translation is greater than a similarity threshold;

[0052] Train an original quality evaluation model according to the pseudo-parallel corpus to obtain a target quality evaluation model, where the target quality evaluation model is used to perform machine translation quality evaluation on a sentence pair to be evaluated.

[0053] According to another aspect of the present disclosure, there is provided a non-volatile computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.

[0054] In the embodiments of the present disclosure, a computer device generates a pseudo-parallel corpus according to a pre-configured original parallel corpus, the pseudo-parallel corpus includes a plurality of pseudo-parallel sentence pairs, and the pseudo-parallel sentence pairs include a source monolingual sentence and a corresponding pseudo-target monolingual sentence; an original quality evaluation model is trained according to the pseudo-parallel corpus to obtain a target quality evaluation model. Since the similarity degree between the data distribution of the pseudo-target monolingual sentence and the data distribution of the real machine translation translation is greater than a similarity threshold, the pseudo-target monolingual sentence can be modeled in a manner close to the machine translation quality evaluation task, ensuring the effect of the subsequent target quality evaluation model performing machine translation quality evaluation on the sentence pair to be evaluated. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings included in and forming a part of the specification, together with the specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and are used to explain the principles of the present disclosure.

[0056] Figure 1 A schematic structural diagram of a computer device provided by an exemplary embodiment of the present disclosure is shown;

[0057] Figure 2 A flowchart of a machine translation quality evaluation method provided by an exemplary embodiment of the present disclosure is shown;

[0058] Figure 3 A schematic diagram of the principle of a machine translation quality evaluation method provided by an exemplary embodiment of the present disclosure is shown;

[0059] Figure 4 A flowchart of a machine translation quality evaluation method provided by another exemplary embodiment of the present disclosure is shown;

[0060] Figure 5 A schematic diagram of the principle of the training process of a generator model provided by an exemplary embodiment of the present disclosure is shown;

[0061] Figure 6 A schematic diagram of the principle of the usage process of a generator model provided by an exemplary embodiment of the present disclosure is shown;

[0062] Figure 7 A schematic diagram of the principle of the training process of a discriminator model provided by an exemplary embodiment of the present disclosure is shown;

[0063] Figure 8 A flowchart of a machine translation quality evaluation method provided by another exemplary embodiment of the present disclosure is shown;

[0064] Figure 9 A schematic structural diagram of a machine translation quality evaluation device provided by an exemplary embodiment of the present disclosure is shown;

[0065] Figure 10 A block diagram of a device for performing a machine translation quality evaluation method according to an exemplary embodiment is shown;

[0066] Figure 11 A block diagram of a device for performing a machine translation quality evaluation method according to another exemplary embodiment is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. Like reference numerals in the drawings denote functionally identical or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0068] As used herein, the term "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or better than other embodiments.

[0069] In addition, for a better illustration of the present disclosure, numerous specific details are given in the following detailed description. Those skilled in the art should understand that the present disclosure can be implemented without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0070] In the related art, the commonly used machine translation quality assessment method without reference translations includes the following steps: In the pre-training stage, a feature extractor is trained on a large-scale raw parallel corpus, and an additional quality assessment model is trained on the labeled translation quality assessment data. When performing the translation quality assessment task, according to the pre-trained feature extractor, the translation quality assessment data to be evaluated is extracted as a set of high-dimensional representations; according to the extracted high-dimensional representations, the pre-trained quality assessment model is used to evaluate the quality of the translations in the translation quality assessment data.

[0071] The machine translation quality assessment method without reference translations can improve the performance of the quality assessment model by introducing the knowledge of the large-scale raw parallel corpus when the number of labeled quality assessment data is scarce. However, there are significant differences between the data distributions of the raw parallel corpus and the real translation quality assessment data. That is, the translations in the raw parallel corpus are natural and correct, while the translations in the translation quality assessment data are generated by a machine translation system and may contain some errors. The feature extractor trained based on the correct translations may not be able to make correct predictions when faced with the translation quality assessment data containing error data, resulting in the inability to fully apply the knowledge of the raw parallel corpus to the translation quality assessment task.

[0072] In related technologies, due to the large difference between the data distribution of the original parallel corpus and the data distribution of the real translation quality evaluation data, it is impossible to better learn the information required for the translation quality evaluation task from the original parallel corpus. In the embodiments of the present disclosure, a computer device generates a pseudo-parallel corpus according to a pre-configured original parallel corpus. Since the pseudo-parallel corpus can include multiple pseudo-parallel sentence pairs and is easy to expand, this solution has sustainability. In addition, since the similarity degree between the data distribution of the pseudo-target monolingual sentences and the data distribution of the real machine translation translations is greater than the similarity threshold, after training the original quality evaluation model according to the pseudo-parallel corpus, the obtained target quality evaluation model has a significant improvement in translation quality.

[0073] First, some terms related to the present disclosure are introduced.

[0074] Model fine-tuning (FT): For a trained model, it is retrained with a small amount of data in a specific professional field to improve the performance of the model in that professional field.

[0075] Parallel corpus (English: parallel corpus): Refers to text written in different languages in contrast, with sentences aligned with each other and translated with each other.

[0076] Monolingual corpus (English: monolingual corpus): Refers to text written in a single language. The monolingual corpus is the basis for constructing the parallel corpus.

[0077] Parallel sentence pair: Refers to a sentence pair written in different languages in contrast, with sentences aligned with each other and translated with each other. Among them, a parallel sentence pair includes a source monolingual sentence and a target monolingual sentence. The source monolingual sentence is also called the original text, and the target monolingual sentence is the translation corresponding to the source monolingual sentence.

[0078] Parallel word pair: Refers to a word pair written in different languages in contrast, with words translated with each other. A professional parallel word pair refers to a parallel word pair in a specific professional field.

[0079] Word alignment (English: word alignment): Refers to the operation of aligning words with the same meaning in the target monolingual words and the source monolingual words in the parallel corpus used for training the machine translation model. Due to the differences in grammar, expression methods, and usage habits of different languages, there are significant differences in the word alignment relationships in the parallel corpus. Establishing the corresponding relationship between the vocabulary in the parallel corpus can improve the translation quality of the model.

[0080] Bilingual evaluation under study (BLEU) is an automatic evaluation index for machine translation quality. BLEU is an algorithm used to measure the similarity between the text obtained by machine translation and the translation reference text. The larger the BLEU value, the higher the quality of machine translation.

[0081] Original parallel corpus: pre-configured bilingual parallel corpus. Original parallel corpus is text written in two languages, with sentences aligned and translated into each other.

[0082] It should be noted that bilingualism can be English-French, English-German, French-English, German-French, etc.

[0083] The original parallel corpus includes a plurality of original parallel sentence pairs, and the original parallel sentence pairs include source monolingual sentences and corresponding correct target monolingual sentences. The source monolingual sentences are also called original texts, and the correct target monolingual sentences are the correct translations corresponding to the source monolingual sentences.

[0084] Pseudo-parallel corpus: a pseudo-parallel corpus obtained based on the original parallel corpus. The pseudo-parallel corpus includes multiple pseudo-parallel sentence pairs, each of which includes a source monolingual sentence and a corresponding pseudo-target monolingual sentence.

[0085] Among them, for an original parallel sentence pair, after replacing at least one word in the correct target monolingual sentence, a pseudo target monolingual sentence corresponding to the source monolingual sentence can be obtained, that is, a pseudo parallel sentence pair includes a source monolingual sentence and a pseudo target monolingual sentence. The source monolingual sentence is also called the original text, and the pseudo target monolingual sentence is the pseudo translation corresponding to the source monolingual sentence.

[0086] The degree of similarity between the data distribution of the pseudo-target monolingual sentence and the data distribution of the true machine translation version is greater than the similarity threshold.

[0087] Please refer to Figure 1 , which shows a schematic diagram of the structure of a computer device provided by an exemplary embodiment of the present disclosure.

[0088] The computer device may be a terminal or a server. The terminal includes a tablet computer, a laptop computer, a desktop computer, etc. The server may be a single server, a server cluster consisting of several servers, or a cloud computing service center.

[0089] like Figure 1 As shown, the computer device includes a processor 10, a memory 20 and a communication interface 30. Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0090] The processor 10 is the control center of the computer device, connecting various parts of the entire computer device through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 20, and by invoking the data stored in the memory 20, it executes various functions of the computer device and processes data, thereby exercising overall control over the computer device. The processor 10 can be implemented by a CPU or by a Graphics Processing Unit (GPU).

[0091] The memory 20 can be used to store software programs and modules. The processor 10 executes various functional applications and data processing by running the software programs and modules stored in the memory 20. The memory 20 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system 21, virtual modules, and application programs required for at least one function (such as neural network model training, etc.); the data storage area can store data created according to the use of the computer device. The memory 20 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read Only Memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc. Correspondingly, the memory 20 can also include a memory controller to provide the processor 10 with access to the memory 20.

[0092] Among them, the processor 20 is used to execute the following functions: generating pseudo-parallel corpora according to pre-configured original parallel corpora, the pseudo-parallel corpora including a plurality of pseudo-parallel sentence pairs, the pseudo-parallel sentence pairs including source monolingual sentences and corresponding pseudo-target monolingual sentences, and the degree of similarity between the data distribution of the pseudo-target monolingual sentences and the data distribution of the real machine translation translations being greater than a similarity threshold; training an original quality assessment model according to the pseudo-parallel corpora to obtain a target quality assessment model, the target quality assessment model being used to perform machine translation quality assessment on sentence pairs to be evaluated. Next, an exemplary embodiment is used to introduce the machine translation quality assessment method provided by the embodiments of the present disclosure.

[0093] Please refer toFigure 2 , which shows a flowchart of a machine translation quality evaluation method provided by an exemplary embodiment of the present disclosure. In this embodiment, the machine translation quality evaluation method is applied to Figure 1 the computer device shown in the figure for illustration. The machine translation quality evaluation method includes:

[0094] Step 201: Generate pseudo-parallel corpora according to pre-configured original parallel corpora. The pseudo-parallel corpora include multiple pseudo-parallel sentence pairs, and each pseudo-parallel sentence pair includes a source monolingual sentence and a corresponding pseudo-target monolingual sentence. The similarity degree between the data distribution of the pseudo-target monolingual sentence and the data distribution of the true machine translation is greater than a similarity threshold.

[0095] The computer device obtains the pre-configured original parallel corpora and generates pseudo-parallel corpora according to the pre-configured original parallel corpora.

[0096] Among them, the original parallel corpora include multiple original parallel sentence pairs, and each original parallel sentence pair includes a source monolingual sentence and a corresponding correct target monolingual sentence. The pseudo-parallel corpora include multiple pseudo-parallel sentence pairs, and each pseudo-parallel sentence pair includes a source monolingual sentence and a corresponding pseudo-target monolingual sentence.

[0097] For the same source monolingual sentence, the corresponding pseudo-target monolingual sentence is different from the corresponding correct target monolingual sentence. Optionally, the pseudo-target monolingual sentence is a target monolingual sentence obtained by replacing at least one word in the correct target monolingual sentence.

[0098] The pseudo-target monolingual sentence and the correct target monolingual sentence have the same length, and there is a one-to-one correspondence between multiple words in the pseudo-target monolingual sentence and multiple words in the correct target monolingual sentence.

[0099] The similarity degree between the data distribution of the pseudo-target monolingual sentence and the data distribution of the true machine translation is greater than a similarity threshold. Among them, the similarity threshold is set by default by the computer device or is user-defined. The embodiments of the present disclosure do not limit this.

[0100] Optionally, for the same source monolingual sentence, the translation accuracy rate and / or translation naturalness of the corresponding pseudo-target monolingual sentence in the pseudo-parallel corpora are lower than those of the corresponding correct target monolingual sentence in the original parallel corpora.

[0101] Step 202: Train the original quality evaluation model according to the pseudo-parallel corpora to obtain a target quality evaluation model, which is used to evaluate the machine translation quality of the sentence pair to be evaluated.

[0102] To improve the translation quality, after a computer device obtains pseudo-parallel corpora, the original quality assessment model can be trained based on multiple pseudo-parallel sentence pairs included in the pseudo-parallel corpora to obtain a target quality assessment model. That is, by training the original quality assessment model with the pseudo-parallel corpora, the translation quality of the target quality assessment model obtained after training can be guaranteed to be improved.

[0103] Among them, the target quality assessment model is used to perform machine translation quality assessment on the sentence pair to be evaluated. The sentence pair to be evaluated includes a source monolingual sentence and a target monolingual sentence.

[0104] The target quality assessment model is used to convert the input sentence pair to be evaluated into an evaluation result, and the evaluation result is used to indicate the translation quality of the target monolingual sentence in the sentence pair to be evaluated.

[0105] The target quality assessment model is used to represent the correlation between the sentence pair to be evaluated and the evaluation result.

[0106] The target quality assessment model is a preset mathematical model, and this target quality assessment model includes the model coefficients between the sentence pair to be evaluated and the evaluation result.

[0107] In summary, in the embodiments of the present disclosure, a computer device generates pseudo-parallel corpora according to pre-configured original parallel corpora. The pseudo-parallel corpora include multiple pseudo-parallel sentence pairs, and the pseudo-parallel sentence pairs include source monolingual sentences and corresponding pseudo-target monolingual sentences; the original quality assessment model is trained according to the pseudo-parallel corpora to obtain a target quality assessment model. Since the data distribution of the pseudo-target monolingual sentences is more similar to the data distribution of the real machine translation translations than the similarity threshold, the pseudo-target monolingual sentences can be modeled in a way closer to the machine translation quality assessment task, ensuring the effect of the subsequent target quality assessment model in performing machine translation quality assessment on the sentence pair to be evaluated.

[0108] In addition, in the related art, limited by the fact that the original parallel corpora are unlabeled, in the pre-training stage, only by predicting the words in the bilingual sentence pairs can the bilingual sentence pairs be modeled. The task defined during the pre-training of the feature extractor is to predict each word in the translation according to the context of the original text and the translation. However, the goal of the machine translation quality assessment task is to predict the quality of each word in the translation and the overall quality of the sentence, rather than predicting what each word is. Defining different training goals on the parallel corpora and the translation quality assessment data may cause the feature extractor to not obtain the features most suitable for the translation quality assessment task.

[0109] Similarly, due to data limitations, in related technologies, only word-level prediction tasks are available. When the model uses the original parallel corpus to train the feature extractor, it is unable to obtain sentence-level representations. When applied to the quality translation evaluation task, an additional quality evaluation model needs to be introduced to combine the high-dimensional representations at the word level to obtain the high-dimensional representation at the sentence level. This means that the quality evaluation model fails to learn sentence-level knowledge from the parallel corpus.

[0110] In the embodiments of the present disclosure, by defining a translation task on the original parallel corpus, a generator model is used to generate pseudo-target monolingual sentences, and the corresponding token parameters for the pseudo-target monolingual sentences are automatically generated. Then, the pseudo-parallel corpus is provided to the discriminator model for training. The training task of the discriminator model is to determine whether each word in the pseudo-target monolingual sentence is machine-translated. Moreover, in order to obtain sentence-level representations, the training task of the discriminator also includes predicting the proportion of the number of machine-translated words in the pseudo-target monolingual sentence to the total number of words. Such a training task is closer to the subsequent translation quality evaluation task itself.

[0111] In a schematic example, as Figure 3 shown, the computer device trains the generator model 32 according to the original parallel corpus 31. After the training of the generator model 32 is completed, the model parameters of the generator model 32 are fixed. The computer device uses the trained generator model 32 to generate multiple pseudo-target monolingual sentences, thereby obtaining the pseudo-parallel corpus 33. The computer device trains the discriminator model 34 according to the pseudo-parallel corpus 33 to obtain the target quality evaluation model 35. The computer device invokes the target quality evaluation model 35 to perform machine translation quality evaluation according to the sentence pair 36 to be evaluated. The following uses schematic embodiments to further introduce the machine translation quality evaluation method provided by the embodiments of the present disclosure.

[0112] Please refer to Figure 4 , which shows a flowchart of the machine translation quality evaluation method provided by another exemplary embodiment of the present disclosure. In this embodiment, it is used in the Figure 1 shown user device as an example. The method includes the following steps.

[0113] Step 401, generate a pseudo-parallel corpus according to the pre-configured original parallel corpus.

[0114] Among them, the original parallel corpus includes multiple original parallel sentence pairs, and each original parallel sentence pair includes a source monolingual sentence and the corresponding correct target monolingual sentence. The pseudo-parallel corpus includes multiple pseudo-parallel sentence pairs, and each pseudo-parallel sentence pair includes a source monolingual sentence and the corresponding pseudo-target monolingual sentence. The data distribution of the pseudo-target monolingual sentence is more similar to the data distribution of the real machine translation translation than the similarity threshold.

[0115] For each original parallel sentence pair in the original parallel corpus, the computer device generates a pseudo-target monolingual sentence by invoking a generator model, where the pseudo-target monolingual sentence is different from the correct target monolingual sentence; based on the source monolingual sentence and the pseudo-target monolingual sentence, a pseudo-parallel sentence pair is obtained. Thus, pseudo-parallel sentence pairs corresponding to each of the multiple original parallel sentence pairs, namely a pseudo-parallel corpus, are obtained.

[0116] Optionally, for each original parallel sentence pair in the original parallel corpus, the source monolingual sentence is encoded by invoking a generator model; at least one word in the correct target monolingual sentence is masked; based on the encoded source monolingual sentence and the masked correct target monolingual sentence, the at least one masked word is reconstructed to obtain a pseudo-target monolingual sentence.

[0117] Among them, the generator model is an auto-encoding structure based on a source text encoder and a masked language model (Masked Language Model) encoder. For example, the source text encoder is a transformer model encoder.

[0118] Optionally, the at least one masked word is randomly determined. That is, the computer device randomly determines and masks at least one word in the correct target monolingual sentence.

[0119] Optionally, the source monolingual sentence X = {x1, …, x m}, the corresponding correct target monolingual sentence Y = {y1, …, y n}, m is the length of the source monolingual sentence, and n is the length of the correct target monolingual sentence. The generator model includes an N-layer source text encoder and an N-layer masked language model encoder, where N is a positive integer (for example, N is 6). The computer device inputs the source monolingual sentence into the source text encoder to obtain the hidden layer state of the source monolingual sentence; the computer device randomly masks at least one word in the correct target monolingual sentence. For example, the computer device uses a special [MASK] token to replace the word y t in the correct target monolingual sentence to obtain the masked correct target monolingual sentence Y mask = {y1, …, [MASK], y t+1 , …, y n}. The computer device inputs the masked correct target monolingual sentence Y mask into the masked language model encoder for encoding, and obtains information about the hidden layer state of the source monolingual sentence through a cross-attention mechanism; at the top of the masked language model, the hidden layer representation at position t is used to predict the real word y t , that is, the masked word is reconstructed to obtain the pseudo-target monolingual sentence Y'.

[0120] Before the computer device generates a pseudo-target monolingual sentence by invoking a generator model, the computer device obtains the trained generator model.

[0121] The reconstructed word task is actually a multi-classification task. During the model training phase, the loss gradient of this reconstructed word task is backpropagated and used to update the model parameters of the generator model. After the training of the generator model is completed, all the model parameters of the generator model are fixed. The computer device uses the trained generator model to generate multiple pseudo-target monolingual sentences, thereby obtaining pseudo-parallel corpora.

[0122] It should be noted that the training process of the generator model can be analogously referred to the above model usage process and will not be elaborated here.

[0123] Since the generator model has an auto-encoding structure, the length of the pseudo-target monolingual sentence is the same as that of the correct target monolingual sentence, and there is a one-to-one correspondence between the multiple words in the pseudo-target monolingual sentence and the multiple words in the correct target monolingual sentence in terms of position. With the help of this characteristic, the computer device can determine whether each word in the pseudo-target monolingual sentence is machine-translated based on whether the words in the same position in the pseudo-target monolingual sentence and the correct target monolingual sentence are equal, and further obtain the proportion of the number of machine-translated words in the pseudo-target monolingual sentence to the total number of words. The pseudo-target monolingual sentence obtained in this way has a data distribution closer to that of the real machine translation translation, and can automatically generate marking parameters.

[0124] Optionally, after the computer device reconstructs at least one masked word based on the encoded source monolingual sentence and the masked correct target monolingual sentence to obtain a pseudo-target monolingual sentence, it further includes: generating marking parameters corresponding to the pseudo-target monolingual sentence according to the pseudo-target monolingual sentence and the correct target monolingual sentence. The marking parameters are used to indicate whether each word in the pseudo-target monolingual sentence is machine-translated, and / or the proportion of the number of machine-translated words in the pseudo-target monolingual sentence to the total number of words, where the total number of words is the total number of words in the pseudo-target monolingual sentence.

[0125] Optionally, the marking parameters include a first marking parameter and / or a second marking parameter. The first marking parameter is used to indicate whether each word in the pseudo-target monolingual sentence is machine-translated, and the second marking parameter includes the proportion of the number of machine-translated words in the pseudo-target monolingual sentence to the total number of words.

[0126] Schematically, the first marking parameter includes a numerical value corresponding to each word in the pseudo-target monolingual sentence. When the numerical value corresponding to a word is the first numerical value, it is used to indicate that the word is machine-translated; when the numerical value corresponding to the word is the second numerical value, it is used to indicate that the word is not machine-translated. For example, the first numerical value is 0 and the second numerical value is 1. The embodiments of the present disclosure do not limit this.

[0127] Optionally, when the word in the pseudo-target monolingual sentence is the same as the word in the same position in the correct target monolingual sentence, it is determined that the word in the pseudo-target monolingual sentence is not machine-translated; when the word in the pseudo-target monolingual sentence is different from the word in the same position in the correct target monolingual sentence, it is determined that the word in the pseudo-target monolingual sentence is machine-translated.

[0128] In a schematic example, taking the generation of a pseudo-target monolingual sentence according to an original parallel sentence pair as an example, the source monolingual is English and the target monolingual is German. The training process of the generator model is as Figure 5 shown. Source monolingual sentence: X = {prints, all, objects, within, the, printable, area, of, the, paper}; corresponding correct target monolingual sentence: Y = {druckt, alle, Objekte, innerhalb, des, druckbaren, Bereichs}. After the training of the generator model is completed, the model parameters of the generator model are fixed. The usage process of the generator model is as Figure 6 shown. The computer device uses the trained generator model to generate the pseudo-target monolingual sentence Y ′ = {druckt, alle, Unterlagen, innerhalb, des, druckbaren, Papier}, and automatically generates marking parameters, including the first marking parameter O′ = 11011101 and the second marking parameter q′ = 0.25.

[0129] Step 402, for each pseudo-parallel sentence pair in the pseudo-parallel corpus, call the discriminator model to obtain the training result. The discriminator model is a model based on the source encoder and the target encoder.

[0130] The computer device uses the pseudo-parallel corpus as the training sample set of the target quality assessment model and defines a discrimination task on the pseudo-parallel corpus.

[0131] For each pseudo-parallel sentence pair in the pseudo-parallel corpus, the computer device calls the discriminator model to obtain the training result.

[0132] Among them, the discriminator model is a model based on the source encoder and the target encoder. Optionally, the discriminator model includes a Transformer model.

[0133] For each pseudo-parallel sentence pair in the pseudo-parallel corpus, the computer device inputs the source monolingual sentence and the corresponding pseudo-target monolingual sentence into the discriminator model, and outputs the training result.

[0134] For each pseudo-parallel sentence pair in the pseudo-parallel corpus, the computer device invokes the discriminator model to encode the source monolingual sentence and encode the pseudo-target monolingual sentence. After encoding, a word-level binary classification task is performed to obtain the training result.

[0135] Optionally, the training result includes predicted token parameters, where the predicted token parameters are used to indicate whether each word in the predicted pseudo-target monolingual sentence is machine-translated, and / or the proportion of the number of machine-translated words in the predicted pseudo-target monolingual sentence to the total number of words. That is, the training result includes the predicted first token parameter and / or the second token parameter.

[0136] In a schematic example, based on Figure 6 the example shown, after the computer device generates the pseudo-target monolingual sentence Y ′ the training process of the discriminator model is as Figure 7 shown. The computer device trains the discriminator model according to the source monolingual sentence X = {prints, all, objects, within, the, printable, area, of, the, paper} and the pseudo-target monolingual sentence Y ′ = {druckt, alle, Unterlagen, innerhalb, des, druckbaren, Papier}, and obtains the training result, where the training result includes the first token parameter O' = 11011101 and the second token parameter q' = 0.25.

[0137] Step 403: Compare the training result with the token parameter corresponding to the pseudo-parallel sentence pair to obtain a calculation loss, where the calculation loss is used to indicate the error between the training result and the token parameter.

[0138] For each pseudo-parallel sentence pair in the pseudo-parallel corpus, the computer device compares the training result with the token parameter corresponding to the pseudo-parallel sentence pair to obtain a calculation loss. Optionally, the calculation loss is represented by cross-entropy.

[0139] Step 404: Train a target quality assessment model according to the calculation losses corresponding to the multiple pseudo-parallel sentence pairs.

[0140] Optionally, the computer device trains a target quality evaluation model according to the respective calculation losses of multiple pseudo-parallel statements by using the error backpropagation algorithm.

[0141] Optionally, the computer device determines the gradient direction of the target quality evaluation model according to the calculation loss through the backpropagation algorithm, and updates the model parameters in the target quality evaluation model layer by layer from the output layer of the target quality evaluation model.

[0142] Optionally, the discriminator model includes an N-layer source text encoder and an N-layer target text encoder, where N is a positive integer. The computer device transforms the source monolingual statement X into a hidden layer state by calling the source text encoder. The computer device encodes according to the pseudo-target monolingual statement and the hidden layer state of the source monolingual statement X by calling the target text encoder. At the top layer of the target text encoder, a word-level binary classification task (i.e., predicting the first label parameter) and a sentence-level regression task (i.e., predicting the second label parameter) are performed according to the hidden layer state of the pseudo-target monolingual statement. The computer device uses the losses of the binary classification task and the regression task to perform gradient backpropagation and update the model parameters of the discriminator model.

[0143] The discriminator model will be introduced to the real translation quality evaluation data for model fine-tuning. The training method of the discriminator model during model fine-tuning is the same as that during pre-training. Optionally, based on the above-trained target quality evaluation model, the method further includes the following steps, as Figure 8 shown:

[0144] Step 801, obtain the statement pair to be evaluated. The statement pair to be evaluated includes a source monolingual statement and a target monolingual statement.

[0145] Optionally, when the computer device receives a machine translation quality evaluation instruction, it obtains the input statement pair to be evaluated. The statement pair to be evaluated includes a source monolingual statement and a target monolingual statement.

[0146] Step 802, according to the statement pair to be evaluated, call the trained target quality evaluation model to obtain an evaluation result.

[0147] Optionally, the computer device calls the trained target quality evaluation model according to the statement pair to be evaluated to obtain an evaluation result; the computer device displays the evaluation result.

[0148] The evaluation result is used to indicate whether each word in the pseudo-target monolingual statement is machine-translated, and / or the proportion of the number of machine-translated words in the pseudo-target monolingual statement to the total number of words. The total number of words is the total number of words in the pseudo-target monolingual statement.

[0149] In summary, the embodiments of the present disclosure provide a method for evaluating the quality of machine translation. On the one hand, a generator is defined on a large-scale original parallel corpus, which can transform the original parallel corpus into a pseudo-parallel corpus whose data distribution is closer to that of the real machine translation translation in terms of data distribution. On the other hand, it is proposed to use pseudo-target monolingual sentences that correspond one-to-one in position to replace the machine translation output translation generated from left to right in the traditional sense, so that the position information can be used to automatically mark the pseudo-target monolingual sentences. On the other hand, it is proposed to use a discriminative task instead of a generative task as the pre-training task, making full use of the characteristics of the pseudo-parallel corpus, so that the training tasks and objectives of the discriminator model during pre-training and fine-tuning are closer. On the other hand, the marking parameter is also used to indicate the proportion of the number of words obtained by machine translation in the pseudo-target monolingual sentence to the total number of words, so that the discriminator model can obtain a sentence-level representation during the pre-training stage.

[0150] From the application level, on the one hand, the method for evaluating the quality of machine translation provided by the embodiments of the present disclosure is applicable to various different language pairs, including but not limited to English-German, English-Russian, English-Chinese, Chinese-English, etc. On the other hand, a large-scale original parallel corpus and a real translation quality evaluation corpus can be used simultaneously. By introducing a pseudo-parallel corpus, the originally different two-stage tasks are transformed into an overall task. On the other hand, the target end naturally has bidirectional capabilities, does not require the introduction of additional markings, has a small parameter scale, and high model training efficiency. On the other hand, based on the structure of the transformer model, the knowledge of other pre-trained language models based on the transformer model can be simply migrated. On the other hand, the model is simple and direct, highly understandable, and the machine translation quality evaluation effect of the target quality evaluation model is good.

[0151] The following is the device embodiment of the present disclosure. For the parts not elaborated in detail in the device embodiment, reference can be made to the technical details disclosed in the above method embodiment.

[0152] Please refer to Figure 9 , which shows a schematic structural diagram of a machine translation quality evaluation device provided by an exemplary embodiment of the present disclosure. The machine translation quality evaluation device can be implemented as all or part of a computer device through software, hardware, and a combination of both. The device includes: a generation module 910 and a training module 920.

[0153] The generation module 910 is configured to generate a pseudo-parallel corpus according to a pre-configured original parallel corpus. The pseudo-parallel corpus includes a plurality of pseudo-parallel sentence pairs. The pseudo-parallel sentence pair includes a source monolingual sentence and a corresponding pseudo-target monolingual sentence. The similarity degree of the data distribution of the pseudo-target monolingual sentence and the data distribution of the real machine translation translation is greater than a similarity threshold;

[0154] A training module 920, configured to train an original quality assessment model based on pseudo-parallel corpora to obtain a target quality assessment model, where the target quality assessment model is used to perform machine translation quality assessment on a sentence pair to be evaluated.

[0155] In a possible implementation, the original parallel corpora include multiple original parallel sentence pairs, and an original parallel sentence pair includes a source monolingual sentence and a corresponding correct target monolingual sentence; the generation module 910 is further configured to:

[0156] For each original parallel sentence pair in the original parallel corpora, call a generator model to generate a pseudo-target monolingual sentence, where the pseudo-target monolingual sentence is different from the correct target monolingual sentence;

[0157] Obtain a pseudo-parallel sentence pair according to the source monolingual sentence and the pseudo-target monolingual sentence.

[0158] In another possible implementation, the generation module 910 is further configured to:

[0159] For each original parallel sentence pair in the original parallel corpora, encode the source monolingual sentence by calling a generator model;

[0160] Mask at least one word in the correct target monolingual sentence;

[0161] Reconstruct the at least one masked word according to the encoded source monolingual sentence and the masked correct target monolingual sentence to obtain a pseudo-target monolingual sentence.

[0162] In another possible implementation, the generator model is an auto-encoding structure based on a source encoder and a masked language model encoder, and there is a one-to-one correspondence between multiple words in the pseudo-target monolingual sentence and multiple words in the correct target monolingual sentence in terms of position.

[0163] In another possible implementation, the generation module 910 is further configured to:

[0164] Generate a marking parameter corresponding to the pseudo-target monolingual sentence according to the pseudo-target monolingual sentence and the correct target monolingual sentence;

[0165] Wherein, the marking parameter is used to indicate whether each word in the pseudo-target monolingual sentence is obtained by machine translation, and / or, the proportion of the number of words obtained by machine translation in the pseudo-target monolingual sentence to the total number of words, and the total number of words is the total number of words in the pseudo-target monolingual sentence.

[0166] In another possible implementation, the training module 920 is further configured to:

[0167] For each pseudo-parallel sentence pair in the pseudo-parallel corpus, a discriminator model is called to obtain a training result, and the discriminator model is a model based on the source encoder and the target encoder;

[0168] The training result is compared with the labeled parameter corresponding to the pseudo-parallel sentence pair to obtain a calculated loss, and the calculated loss is used to indicate the error between the training result and the labeled parameter;

[0169] Based on the calculated losses corresponding to multiple pseudo-parallel sentence pairs respectively, a target quality assessment model is trained.

[0170] In another possible implementation, the device further includes: an acquisition module and a call module;

[0171] The acquisition module is used to acquire the sentence pair to be evaluated, and the sentence pair to be evaluated includes a source monolingual sentence and a target monolingual sentence;

[0172] The call module is used to call the trained target quality assessment model according to the sentence pair to be evaluated to obtain an evaluation result.

[0173] It should be noted that when the device provided in the above embodiments implements its functions, only the above-mentioned division of each functional module is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to actual needs, that is, the content structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0174] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0175] The embodiments of the present disclosure further provide a computer device, which includes: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to: implement the methods in the above-mentioned method embodiments.

[0176] The embodiments of the present disclosure further provide a non-volatile computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the methods in the above-mentioned method embodiments are implemented.

[0177] Figure 10 It is a block diagram of a device 1000 for performing a machine translation quality assessment method shown according to an exemplary embodiment. For example, the device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0178] Refer to Figure 10, Device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0179] The processing component 1002 generally controls the overall operation of the device 1000, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0180] The memory 1004 is configured to store various types of data to support the operation of the device 1000. Examples of such data include instructions for any application or method operating on the device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0181] The power supply component 1006 provides power to various components of the device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 1000.

[0182] The multimedia component 1008 includes a screen that provides an output interface between the device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0183] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC) that is configured to receive external audio signals when the device 1000 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 1004 or transmitted via the communication component 1016. In some embodiments, the audio component 1010 further includes a speaker for outputting audio signals.

[0184] The I / O interface 1012 provides an interface between the processing component 1002 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0185] The sensor component 1014 includes one or more sensors for providing a status assessment of various aspects of the device 1000. For example, the sensor component 1014 can detect the on / off state of the device 1000, the relative positioning of components, such as the display and the keypad of the device 1000. The sensor component 1014 can also detect a change in the position of the device 1000 or a component of the device 1000, the presence or absence of user contact with the device 1000, the orientation or acceleration / deceleration of the device 1000, and the temperature change of the device 1000. The sensor component 1014 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 1014 can also include a light sensor, such as a CMOS or a CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 1014 can further include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0186] The communication component 1016 is configured to facilitate communication between the device 1000 and other devices in a wired or wireless manner. The device 1000 can access a communication standard-based wireless network, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0187] In an exemplary embodiment, the device 1000 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0188] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1004 including computer program instructions, and the above computer program instructions can be executed by the processor 1020 of the device 1000 to complete the above method.

[0189] Figure 11 is a block diagram of a device 1100 for performing a machine translation quality assessment method shown according to another exemplary embodiment. For example, the device 1100 can be provided as a server. Referring to Figure 11 , the device 1100 includes a processing component 1122, which further includes one or more processors, and memory resources represented by a memory 1132 for storing instructions executable by the processing component 1122, such as application programs. The application programs stored in the memory 1132 can include one or more modules each corresponding to a set of instructions. In addition, the processing component 1122 is configured to execute instructions to perform the above method.

[0190] The device 1100 may further include a power component 1126 configured to perform power management of the device 1100, a wired or wireless network interface 1150 configured to connect the device 1100 to a network, and an input / output (I / O) interface 1158. The device 1100 can operate based on an operating system stored in the memory 1132, such as Windows ServerTM, MacOS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.

[0191] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1132 including computer program instructions, and the computer program instructions can be executed by a processing component 1122 of the device 1100 to complete the above method.

[0192] The present disclosure may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0193] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as being a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0194] The computer-readable program instructions described herein may be downloaded to respective computing / processing devices from a computer-readable storage medium, or may be downloaded to an external computer or an external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0195] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.

[0196] Aspects of the present disclosure are described herein with reference to the flowchart and / or block diagram of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer - readable program instructions.

[0197] These computer - readable program instructions can be provided to a processor of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data - processing apparatus, create a means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions includes a manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0198] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, causing a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0199] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions.

[0200] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the technical improvement of technologies in the market, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for evaluating the quality of machine translation, characterized in that, In a computer device, the method includes: Generating pseudo-parallel corpora according to pre-configured original parallel corpora, where the pseudo-parallel corpora include multiple pseudo-parallel sentence pairs, the pseudo-parallel sentence pairs include source monolingual sentences and corresponding pseudo-target monolingual sentences, and the similarity degree of the data distribution of the pseudo-target monolingual sentences to the data distribution of real machine translation translations is greater than a similarity threshold; Training an original quality assessment model according to the pseudo-parallel corpora to obtain a target quality assessment model, where the target quality assessment model is used to perform machine translation quality assessment on sentence pairs to be evaluated; The original parallel corpora include multiple original parallel sentence pairs, and the original parallel sentence pairs include source monolingual sentences and corresponding correct target monolingual sentences; the generating of pseudo-parallel corpora according to pre-configured original parallel corpora includes: For each of the original parallel sentence pairs in the original parallel corpora, generating the pseudo-target monolingual sentence by calling a generator model, where the pseudo-target monolingual sentence is different from the correct target monolingual sentence, and the pseudo-target monolingual sentence is a target monolingual sentence obtained by replacing at least one word in the correct target monolingual sentence; Obtaining the pseudo-parallel sentence pair according to the source monolingual sentence and the pseudo-target monolingual sentence.

2. The method according to claim 1, characterized in that, The generating of the pseudo-target monolingual sentence by calling a generator model for each of the original parallel sentence pairs in the original parallel corpora includes: For each of the original parallel sentence pairs in the original parallel corpora, encoding the source monolingual sentence by calling the generator model; Masking at least one word in the correct target monolingual sentence; Reconstructing the masked at least one word according to the encoded source monolingual sentence and the masked correct target monolingual sentence to obtain the pseudo-target monolingual sentence.

3. The method according to claim 2, wherein The generator model is an auto-encoding structure based on a source encoder and a masked language model encoder, and there is a one-to-one correspondence in position between multiple words in the pseudo-target monolingual sentence and multiple words in the correct target monolingual sentence.

4. The method according to claim 3, wherein After reconstructing the masked at least one word according to the encoded source monolingual sentence and the masked correct target monolingual sentence to obtain the pseudo-target monolingual sentence, it further includes: Generating a marking parameter corresponding to the pseudo-target monolingual sentence according to the pseudo-target monolingual sentence and the correct target monolingual sentence; Wherein, the marking parameter is used to indicate whether each word in the pseudo-target monolingual sentence is obtained by machine translation, and / or, the proportion of the number of words obtained by machine translation in the pseudo-target monolingual sentence to the total number of words, and the total number of words is the total number of words in the pseudo-target monolingual sentence.

5. The method according to claim 4, characterized in that The training of the original quality assessment model according to the pseudo-parallel corpora to obtain a target quality assessment model includes: For each of the pseudo-parallel sentence pairs in the pseudo-parallel corpora, obtaining a training result by calling a discriminator model, and the discriminator model is a model based on a source encoder and a translation encoder; Compare the training result with the marked parameters corresponding to the pseudo-parallel sentence pairs to obtain a calculated loss, where the calculated loss is used to indicate the error between the training result and the marked parameters; Train the target quality evaluation model according to the calculated losses respectively corresponding to multiple pseudo-parallel sentence pairs.

6. The method according to any one of claims 1 to 5, characterized in that, After training the original quality evaluation model according to the pseudo-parallel corpus to obtain the target quality evaluation model, it further includes: Obtain a sentence pair to be evaluated, where the sentence pair to be evaluated includes a source monolingual sentence and a target monolingual sentence; According to the sentence pair to be evaluated, call the trained target quality evaluation model to obtain an evaluation result.

7. A machine translation quality evaluation device, characterized in that, For use in a computer device, the device includes: A generation module, configured to generate a pseudo-parallel corpus according to a pre-configured original parallel corpus, where the pseudo-parallel corpus includes multiple pseudo-parallel sentence pairs, and each pseudo-parallel sentence pair includes a source monolingual sentence and a corresponding pseudo-target monolingual sentence, and the similarity degree of the data distribution of the pseudo-target monolingual sentence and the data distribution of the real machine translation translation is greater than a similarity threshold; A training module, configured to train an original quality evaluation model according to the pseudo-parallel corpus to obtain a target quality evaluation model, where the target quality evaluation model is used to evaluate the machine translation quality of a sentence pair to be evaluated; The original parallel corpus includes multiple original parallel sentence pairs, and each original parallel sentence pair includes a source monolingual sentence and a corresponding correct target monolingual sentence; the generation module is further configured to: For each original parallel sentence pair in the original parallel corpus, call a generator model to generate the pseudo-target monolingual sentence, where the pseudo-target monolingual sentence is different from the correct target monolingual sentence, and the pseudo-target monolingual sentence is a target monolingual sentence obtained by replacing at least one word in the correct target monolingual sentence; Obtain the pseudo-parallel sentence pair according to the source monolingual sentence and the pseudo-target monolingual sentence.

8. A computer device, characterized in that, The computer device includes: a processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Generate a pseudo-parallel corpus according to a pre-configured original parallel corpus, where the pseudo-parallel corpus includes multiple pseudo-parallel sentence pairs, and each pseudo-parallel sentence pair includes a source monolingual sentence and a corresponding pseudo-target monolingual sentence, and the similarity degree of the data distribution of the pseudo-target monolingual sentence and the data distribution of the real machine translation translation is greater than a similarity threshold; Train an original quality evaluation model according to the pseudo-parallel corpus to obtain a target quality evaluation model, where the target quality evaluation model is used to evaluate the machine translation quality of a sentence pair to be evaluated; The original parallel corpus includes multiple original parallel sentence pairs, and each original parallel sentence pair includes a source monolingual sentence and a corresponding correct target monolingual sentence; the generating the pseudo-parallel corpus according to the pre-configured original parallel corpus includes: For each of the original parallel sentence pairs in the original parallel corpus, a pseudo-target monolingual sentence is generated by calling a generator model, the pseudo-target monolingual sentence is different from the correct target monolingual sentence, and the pseudo-target monolingual sentence is a target monolingual sentence obtained by replacing at least one word in the correct target monolingual sentence; According to the source monolingual sentence and the pseudo-target monolingual sentence, the pseudo-parallel sentence pair is obtained.

9. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images