A machine translation trace detection method

Abstract

The application discloses a machine translation trace detection method, comprising the following steps: obtaining the similarity of each sentence pair in a bilingual corpus array with respect to a forward translation array, the similarity of each sentence pair in the bilingual corpus array with respect to a round-trip translation array, and the similarity of the round-trip translation array and the forward translation array; and averaging the three similarity values to obtain a score of the degree of machine translation. The machine translation trace is detected based on the similarity of the given to-be-detected translation, the forward translation array and the round-trip translation array, the similarity is calculated by using a classical algorithm, the threshold value is easy to determine through practice, the machine round-trip translation trace can be detected, and the detection result is more accurate, that is, the possibility that the to-be-detected text comes from a machine translation engine can be accurately represented.

Description

Technical Field

[0001] This invention relates to the field of machine translation proficiency detection technology, and in particular to a method for detecting traces in machine translation. Background Technology

[0002] Translation, as an information conversion and dissemination activity, refers to the process of transforming information from one language into information in another. To improve translation efficiency and accessibility, machine translation is increasingly being applied in various translation scenarios. However, due to the inherent error rate and insufficient match between current machine translation results and the original meaning, it cannot completely replace human translation.

[0003] In the language service industry, when reviewing translated manuscripts, or when school teachers guide students in translating assignments, a common requirement is to identify and evaluate traces of machine translation, that is, to assess the likelihood that the text being reviewed originated from a machine translation engine. Currently, common evaluation methods involve comparing the text to be tested with a forward machine translation engine translation, using the similarity between the two as the evaluation metric, or using a language model for scoring. However, both of these methods fail to identify traces of machine translation in round-trip translation, and due to the non-linear nature of deep learning models, the final scoring threshold is difficult to determine. Summary of the Invention

[0004] In view of this, the present invention provides a machine translation trace detection method to solve the technical problem that the current detection methods are not accurate enough and cannot identify back-and-forth translation traces.

[0005] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or to describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0006] The present invention adopts the following technical solution:

[0007] This invention provides a method for detecting machine translation artifacts, comprising:

[0008] Obtain the similarity between the target language text and the forward translation array for each sentence pair in the bilingual corpus array, the similarity between the target language text and the round-trip translation array, and the similarity between the round-trip translation array and the forward translation array;

[0009] The average of the three similarity scores is used as the score for the machine translation accuracy.

[0010] The forward translation array is composed of multiple forward translation texts obtained by translating the source language text of each sentence pair in the bilingual corpus array into the source language text through each machine in the forward translation engine array.

[0011] Specifically, the target language text of each sentence pair in the bilingual corpus array is translated into multiple reverse translation texts by each machine in the reverse translation engine array to form a reverse translation array; the round-trip translation array is composed of multiple round-trip translation texts obtained by each reverse translation text in the reverse translation array being translated into multiple reverse translation texts by each machine in the forward translation engine array to form a round-trip translation array.

[0012] Furthermore, the process of obtaining the similarity between the target language text and the forward translation array includes: calculating the edit distance or translation revision rate between the target language text and each forward translation text in the forward translation array; calculating the average of the edit distance or translation revision rate, and using the average as the similarity value between the target language text and the forward translation array.

[0013] Furthermore, the process of obtaining the similarity between the target language text and the round-trip translation array includes: calculating the edit distance or translation revision rate between the target language text and each round-trip translation text in the round-trip translation array; calculating the average of the edit distance or translation revision rate, and using the average as the similarity value of the target language text relative to the round-trip translation array.

[0014] Furthermore, the process of obtaining the similarity between the round-trip translation array and the forward translation array includes: calculating the edit distance or translation revision rate between each round-trip translation text in the round-trip translation array and each forward translation text in the forward translation array; calculating the average value of the edit distance or translation revision rate, and using the average value as the similarity value between the round-trip translation array and the forward translation array.

[0015] Furthermore, the machine translation trace detection method further includes: outputting a result array in the form of (e, f, s, s1), where e is the source language text of each sentence pair in the bilingual corpus array; f is the target language text of each sentence pair in the bilingual corpus array; s is the average of the three similarity values; s1 is the similarity value of the target language text relative to the forward translation array, and the value of s1 is used to evaluate the post-translation editing workload.

[0016] Furthermore, the machine translation trace detection method further includes: sorting all the result arrays, with the sorting rule being ascending order based on the s value in each result array; and taking the result arrays ranked in the top 10% and with s values ​​less than a preset similarity threshold as a partial output similar to machine translation.

[0017] Furthermore, the machine translation trace detection method further includes: traversing all sentence pairs in the bilingual corpus array and determining whether the length of the source language text and the length of the target language text in each sentence pair both reach a preset length threshold; if both reach the preset length threshold, then the process of obtaining three similarity values ​​is performed; otherwise, the sentence pair is discarded.

[0018] Furthermore, the machine translation trace detection method further includes: detecting the alignment degree of the bilingual corpus array; if the alignment degree is lower than a preset alignment threshold, then aligning the sentences in the bilingual corpus array.

[0019] The beneficial effects of this invention are as follows: it detects traces of machine translation based on the similarity between the given translation to be tested, the forward translation array of the original text, and the round-trip translation array. The similarity calculation adopts a classic algorithm, and the threshold is easy to determine through practice. It can detect traces of machine round-trip translation, making the detection results more accurate, and can accurately characterize the possibility that the text to be tested comes from a machine translation engine. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a machine translation trace detection method according to the present invention. Detailed Implementation

[0022] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0023] The following is an explanation of the terms used in this article:

[0024] Language direction refers to the direction of language translation. For example, if the translation needs to be from Chinese to English, then the language direction is from Chinese to English.

[0025] Text length refers to the number of words or characters in a text (depending on the specific case). The length of text t is represented as |t|.

[0026] Forward translation refers to the act of translating source language text into target language text.

[0027] Forward translation text refers to text generated through forward translation.

[0028] Back translation (or reverse translation) refers to the act of translating target language text into source language text.

[0029] Reverse translation text refers to text generated through reverse translation.

[0030] Round-trip translation refers to the process of translating the source language text forward and then back, or translating the target language text back and then forward.

[0031] Round-trip translation text refers to text generated through round-trip translation.

[0032] Edit distance refers to the minimum number of modifications a text x needs to undergo to become text y. Modifications include deletion, addition, and replacement, and can be represented by Ed(x, y). The smaller the value, the closer x and y are. This invention calculates the edit distance between two texts using any existing method for quantifying the degree of string difference, which will not be elaborated upon here.

[0033] Translation revision rate refers to the edit distance between the evaluated translation x and the reference translation y divided by the text length of the reference translation y, and can also be represented by Ed(x, y).

[0034] The average value refers to the average value of an array, for example, the array [x1, x2, x3...x...]. n The average value of ] is expressed as avg([x1, x2, x3...x...). n ]).

[0035] Because translators may employ a strategy of multiple round trip translations to circumvent machine translation detection—that is, first using machine translation to translate the source language into the target language, then using machine translation to translate the translation back into the source language, and then translating the back-translated source language back into the target language again—this process can be repeated multiple times until the target language translation is sufficiently similar to the initial forward machine translation. Therefore, current detection mechanisms cannot accurately identify round trip translation traces, resulting in inaccurate detection results.

[0036] In response to the above technical problems, such as Figure 1As shown in some illustrative embodiments, the present invention provides a method for detecting machine translation traces, comprising the following steps:

[0037] 101: Obtain the original text and the translation to be tested, and determine the language direction. For example, if the original text is Chinese, then the corresponding English text is the translation to be tested. The purpose of this embodiment is to evaluate the degree to which the translation to be tested comes from the machine translation engine.

[0038] 102: Arrange the original text and the translation to be tested in parallel according to the order of the sentences to obtain the bilingual corpus array C. Detect the alignment degree of the bilingual corpus array C and determine whether the alignment degree of the current sentence is lower than the preset alignment threshold.

[0039] 103: If the alignment degree is lower than the preset alignment threshold, then align the sentences in the bilingual corpus C. Given bilingual texts that are translations of each other, the process, technique, and method of establishing correspondences between sentences according to translation relationships are called sentence alignment. This invention can use existing text detection and sequence alignment tools for detecting the degree of sentence alignment and for the operation method of sentence alignment.

[0040] After alignment, each sentence in the bilingual corpus array C is extracted and formed into multiple sentence pairs according to the correspondence. That is, the bilingual corpus array C contains multiple sentence pairs (e, f). Each sentence pair consists of source language text and target language text corresponding to the source language text. For any sentence pair (e, f), e refers to the source language text and f refers to the target language text translated from the source language text.

[0041] 104: Iterate through all sentence pairs (e, f) in the bilingual corpus array C, and determine whether the lengths of the source language text and the target language text in each sentence pair (e, f) both reach the preset length thresholds. Specifically, set the threshold for the length of the source language text to k and the threshold for the length of the target language text to z, then determine whether the conditions are met: |e|≥k and |f|≥z.

[0042] 105: If the conditions |e|≥k and |f|≥z are not met, then the current sentence pair (e,f) will be discarded, that is, the excessively short sentence segments will be sampled and filtered out to improve the detection efficiency.

[0043] If the length of the source language text and the length of the target language text in the current sentence pair (e, f) both reach the preset length threshold, that is, satisfying |e|≥k and |f|≥z, then proceed with the process of obtaining three similarity values, which includes steps 106, 107 and 108.

[0044] 106: Obtain the target language text f of each sentence pair (e, f) in the bilingual corpus array C relative to the forward translation array o = [o1, o2, o3...o...]. nThe similarity of ].

[0045] Wherein, the forward translation array o = [o1, o2, o3...o... n ] is composed of multiple forward translation texts obtained by machine translating the source language text e of each sentence pair (e, f) in the bilingual corpus array C through each source language text e in the forward translation engine array.

[0046] Assume the forward translation engine array has n translation engines capable of forward translation. Therefore, for each sentence pair (e, f), the source language text e can be translated by the n forward translation engines to obtain n forward translation texts, labeled as o1, o2, o3...o n , o1, o2, o3...o n Form a forward translation array o, i.e., o = [o1, o2, o3...o2, o3...o3...o4...o5...o6...o7...o8...o9...o1...o2...o3...o3...o4...o5...o6...o7...o9...o1...o2...o3...o3...o3...o4...o3 ... n ].

[0047] Specifically, the target language text f is obtained relative to the forward translation array o = [o1, o2, o3...o...]. n The process of similarity assessment includes:

[0048] First, calculate the target language text f and the forward translation array o = [o1, o2, o3...o...]. n The edit distance or translation revision rate of each forward translation text in the target language text f is calculated separately from the forward translation text o1, forward translation text o2, and so on up to the forward translation text o. n By calculating the edit distance or translation revision rate between the values, we can obtain n edit distance or translation revision rate values, denoted as Ed(f, o1), Ed(f, o2), Ed(f, o3)...Ed(f, o2) n ).

[0049] Then, calculate the average of the n edit distances or translation revision rates calculated in the previous step, i.e., calculate avg([Ed(f, o1), Ed(f, o2), Ed(f, o3)...Ed(f, o2)). n Let s1 be the value of the target language text f and the similarity value of the forward translation array.

[0050] 107: Obtain the target language text f of each sentence pair (e, f) in the bilingual corpus array C relative to the round-trip translation array g = [g1, g2, g3...g m The similarity of ].

[0051] The target language text f of each sentence pair (e, f) in the bilingual corpus C is processed by the machine translation of the target language text f in the reverse translation engine array to form multiple reverse translation arrays h = [h1, h2, h3...h...]. n ].

[0052] The reverse translation engine array is configured with n translation engines capable of reverse translation. Therefore, for each sentence pair (e, f), the target language text f can be translated by the n reverse translation engines to obtain n reverse translation texts, labeled as h1, h2, h3...h1. n h1, h2, h3...h n The reverse translation array h is formed, i.e., h = [h1, h2, h3...h...]. n ].

[0053] The round-trip translation array is composed of the reverse translation array h = [h1, h2, h3...h...]. n The various reverse translation texts h1, h2, h3...h in the text. n The reverse-translated text h1, h2, h3...h is generated by machine translation in the forward translation engine array. n The resulting multiple round-trip translated texts are composed of this text.

[0054] Specifically, for each reverse translation text h1, h2, h3...h n After translation by an array of forward translation engines (n forward translation engines), m forward translation texts can be obtained, labeled as g1, g2, g3...g m And m≥n, let g1, g2, g3...g m The round-trip translation array g is formed, i.e., g = [g1, g2, g3...g m ].

[0055] If the elements of the round-trip translation array g and the reverse translation array h correspond one-to-one, meaning only one forward translation engine is used for forward translation, then m = n; if each item in the reverse translation array h contains n forward translation texts, meaning an array of forward translation engines is used for forward translation, then m = n. 2 The specific choice depends on the actual needs.

[0056] Specifically, the target language text f is obtained relative to the round-trip translation array g = [g1, g2, g3...g...]. m The process of similarity assessment includes:

[0057] First, calculate the target language text f and the round-trip translation array g = [g1, g2, g3...g...]. mThe edit distance or translation revision rate of each round-trip translation text in the target language text f is calculated separately from the round-trip translation texts g1, g2, and so on up to the round-trip translation text g. m By calculating the edit distance or translation revision rate between the values, we can obtain m edit distance or translation revision rate values, denoted as Ed(f, g1), Ed(f, g2), Ed(f, g3)...Ed(f, g4) respectively. m ).

[0058] Then, calculate the average of the m edit distances or translation revision rates calculated in the previous step, i.e., calculate avg([Ed(f, g1), Ed(f, g2), Ed(f, g3)...Ed(f, g...)). m Let s2 be the similarity value of the target language text f relative to the round-trip translation array g.

[0059] 108: Get the round-trip translation array g = [g1, g2, g3...g m ] and the forward translation array o = [o1, o2, o3...o n The similarity of ] is determined by the following process:

[0060] First, calculate the round-trip translation array g = [g1, g2, g3...g m The array of round-trip translated texts and forward translated texts in the array is o = [o1, o2, o3...o...]. n The edit distance or translation revision rate between each forward translation text in the [reference] is calculated. Specifically, the edit distance or translation revision rate between the round-trip translation text g1 and the forward translation text o1, o2, and so on, is calculated separately. n The edit distance or translation revision rate between them; then calculate the round-trip translation text g2 and the forward translation text o1, forward translation text o2, and so on, until the forward translation text o. n Edit distance or translation revision rate between; until the round-trip translated text g is calculated. m With the forward translation text o1, forward translation text o2, and so on up to the forward translation text o n By calculating the edit distance or translation revision rate between the values, we can obtain m×n edit distance or translation revision rate values, denoted as Ed(g1, o1), Ed(g1, o2), Ed(g1, o3)...Ed(g1, o2) n ), Ed(g2, o1), Ed(g2, o2)...Ed(g2, o n ...Ed(g) m o n ).

[0061] Then, calculate the average of the edit distance or translation revision rate calculated in the previous step, i.e., calculate avg([Ed(g1, o1), Ed(g1, o2), Ed(g1, o3)...Ed(g1, o2)...Ed ... n ), Ed(g2, o1), Ed(g2, o2)...Ed(g2, o n ...Ed(g) m o n Let s3 be the similarity value of the round-trip translation array g relative to the forward translation array o.

[0062] 109: The average of the three similarity scores s1, s2, and s3 is used as the score for the machine translation level. If Ed is the edit distance function, the machine translation level score for the current sentence is as follows:

[0063] s = (s1 + s2 + s3) / (|f| × 3);

[0064] If Ed is a translation revision rate function, then the machine translation quality score for the current sentence is as follows:

[0065] s = (s1 + s2 + s3) / 3.

[0066] 110: Output the result array R as a tuple of (e, f, s, s1), that is, the result is pushed into R as a tuple of (e, f, s, s1), where e is the source language text of each sentence pair in the bilingual corpus array C; f is the target language text of each sentence pair in the bilingual corpus array C; s is the average of the three similarity values; s1 is the similarity value of the target language text f relative to the forward translation array o, and the value of s1 is used to evaluate the post-translation editing workload. That is, this invention evaluates the post-translation editing workload while evaluating the degree of machine translation traces, making the result data richer.

[0067] 111: Sort all result arrays R in ascending order according to the value of s in each result array, and take the top 10% of the result arrays with s values ​​less than the preset similarity threshold p as the output of machine translation-like parts.

[0068] In practice, the similarity threshold p is determined by empirical data from specific language directions and domains. Ultimately, s for each item is used as the score for machine translation detection, and s1 is used as the basis for post-editing workload. This invention not only assesses the degree of machine translation artifacts and the workload of post-editing, but also filters out parts that resemble machine translation and outputs them independently, making the detection results more objective and accurate, and avoiding misjudgments of the entire text.

[0069] Since translators primarily use publicly available machine translation engines from mainstream providers, the models, data, and strategies used by most of these providers are similar. This means that for any given machine translation provider, the training corpus for any language pair (S, T) is bidirectionally shared; that is, the corpus for the forward engine (ST) and the backward engine (TS) is identical. From a statistical translation perspective, their language models, alignment models, and reordering models are also consistent, a situation that also holds true from a neural network translation perspective.

[0070] Therefore, it can be roughly assumed that:

[0071] Given a language pair (S, T) and a corpus pair (E, F), if a T language text t is a typical machine translation output, it must approach the engine's target language training corpus F, i.e., t ~ F. Here, "approaching" means that the language model is close to the distribution of F, the word style is close to F, and even the original text exists in F.

[0072] If a language text t approaches F, it must also have n typical reverse translations s' approaching E, i.e., s' ~ E. This rule also applies to S to T. If a language text t does not approach the target text of the training corpus, it cannot be considered similar to machine translation, and there must not be any of the aforementioned s'. In other words, the round-trip translation texts of a machine translation t are similar to itself; the source text s of a machine translation t is similar to t after machine translation; and in multiple round-trip translations of a machine translation t, the round-trip results t' are similar to each other.

[0073] Therefore, this invention utilizes the similarity between a given target translation, a forward translation array *o*, and a round-trip translation array *g*, to detect machine translation artifacts. Specifically, given a text pair between the target translation and the source text, multiple sentence pairs (e, f) are obtained. A forward translation engine array is used to translate *e* to *o*, a backward translation engine array is used to translate *f* to *h*, and then a forward translation engine array is used to translate *h* to *g*. Finally, three similarity scores *s1*, *s2*, and *s3* are calculated, and the average of these similarity scores is used as the basis for detecting machine translation artifacts, resulting in more accurate detection. Furthermore, to enhance the detection effect, for machine translation of arbitrary language directions, this invention employs a translation array composed of multiple translation engines.

[0074] The following example illustrates this:

[0075] Let f be the target language text; o be the forward translation text; g be the round-trip translation text; and S be the total similarity. Ter(x, y) is used as the translation revision rate function for texts x and y, where y is the reference translation. For ease of demonstration, a translation array is not used here.

[0076] Group 1: I wonder if Young Master Wang is inside?

[0077] Version A:

[0078]

[0079] Ter(f,o) = 0.75;

[0080] Ter(f, g) = 0.4;

[0081] Ter(g,o)=0.5;

[0082] S=(0.75+0.4+0.5) / 3=0.550.

[0083] Version B:

[0084]

[0085]

[0086] Ter(f, o) = 0.0;

[0087] Ter(f, g) = 0.333;

[0088] Ter(g,o)=0.375;

[0089] S=(0.0+0.333+0.375) / 3=0.236.

[0090] Group Two: Changes in the way literary works exist and are disseminated will inevitably affect literature itself. (Version A)

[0091]

[0092] Ter(f, o) = 0.632;

[0093] Ter(f, g) = 0.333;

[0094] Ter(g,o)=0.632;

[0095] S=(0.632+0.333+0.632) / 3=0.532.

[0096] Version B:

[0097]

[0098] All sentence pairs are identical, therefore the translation revision rate is 0.

[0099] Group 3: Lift the control lever handle on the right side of the slide box upwards to rotate the main shaft forward; return the control lever handle to the middle position to stop the main shaft from rotating; press the control lever handle down to rotate the main shaft in reverse.

[0100] Version A:

[0101]

[0102] Ter(f,o) = 0.306;

[0103] Ter(f, g) = 0.292;

[0104] Ter(g,o)=0.070;

[0105] S=(0.306+0.292+0.070) / 3=0.223.

[0106] Note that the translation revision rates of the original and forward translations obtained from the round-trip translation are already relatively high, while Ter(g,o) drags down the score. This is a typical case of round-trip translation detection.

[0107] Version B:

[0108]

[0109] Ter(f,o) = 0.347;

[0110] Ter(f, g) = 0.25;

[0111] Ter(g,o)=0.449;

[0112] S=(0.347+0.25+0.449) / 3=0.349.

[0113] Compared to version A, the human-translated version has a higher Ter(g,o) score and a higher final translation revision rate, proving that the text is not machine-translated.

[0114] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for machine translation trace detection, the method comprising: The method comprises: obtaining similarity of target language texts of each sentence pair in a bilingual corpus array with respect to a forward translation array, similarity of the target language texts with respect to a round-trip translation array, and similarity of the round-trip translation array with respect to the forward translation array; averaging the three similarity values to obtain a score of machine translation degree; the forward translation array is composed of a plurality of forward translation texts obtained by machine translation of source language texts of each sentence pair in the bilingual corpus array by each machine translation in a forward translation engine array; a reverse translation array is composed of a plurality of reverse translation texts obtained by machine translation of target language texts of each sentence pair in the bilingual corpus array by each machine translation in a reverse translation engine array; the round-trip translation array is composed of a plurality of round-trip translation texts obtained by machine translation of each reverse translation text in the reverse translation array by each machine translation in the forward translation engine array.

2. The method of claim 1, wherein, The process of obtaining the similarity of the target language texts with respect to the forward translation array comprises: calculating the edit distance or translation revision rate of the target language texts with respect to each forward translation text in the forward translation array; averaging the edit distance or translation revision rate to obtain the similarity value of the target language texts with respect to the forward translation array.

3. The method of claim 2, wherein, The process of obtaining the similarity of the target language texts with respect to the round-trip translation array comprises: calculating the edit distance or translation revision rate of the target language texts with respect to each round-trip translation text in the round-trip translation array; averaging the edit distance or translation revision rate to obtain the similarity value of the target language texts with respect to the round-trip translation array.

4. The method of claim 3, wherein, The process of obtaining the similarity of the round-trip translation array with respect to the forward translation array comprises: calculating the edit distance or translation revision rate of each round-trip translation text in the round-trip translation array with respect to each forward translation text in the forward translation array; averaging the edit distance or translation revision rate to obtain the similarity value of the round-trip translation array with respect to the forward translation array.

5. The method of claim 4, wherein, Further comprising: outputting the result array in the form of (e, f, s, s1) tuples, wherein, e is the source language text of each sentence pair in the bilingual corpus array; f is the target language text of each sentence pair in the bilingual corpus array; s is the average of the three similarity values; s1 is the similarity value of the target language texts with respect to the forward translation array, and the value of s1 is used to evaluate the post-editing workload.

6. The method of claim 5, wherein, Further comprising: sorting all the result arrays in ascending order according to the s value in each result array; outputting the top 10% of the result arrays with s values less than a preset similarity threshold as the part of similar machine translation.

7. The method of claim 6, wherein, Further comprising: traversing all the sentence pairs in the bilingual corpus array, and determining whether the length of the source language text and the length of the target language text in each sentence pair meet a preset length threshold; If both reach the preset length threshold, a process of obtaining three similarity values is performed, otherwise the sentence pair is discarded.

8. The method of claim 7, wherein, Also includes: Detecting the alignment degree of the bilingual corpus array, if the alignment degree is lower than the preset alignment threshold, align each sentence in the bilingual corpus array.

Images