Translation effect evaluation method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a translation effect evaluation method and device, electronic equipment and a storage medium. The original text is translated to obtain a plurality of candidate translations and an accuracy score corresponding to each candidate translation. The plurality of candidate translations are sorted according to the accuracy score and a preset sorting rule to obtain a first sorting sequence. Each candidate translation is evaluated according to a translation quality evaluation index to obtain a quality evaluation score corresponding to each candidate translation. The plurality of candidate translations are sorted according to the quality evaluation score and the preset sorting rule to obtain a second sorting sequence. An error value between the first sorting sequence and the second sorting sequence is calculated. The error value is displayed so that a user can obtain the translation effect of the translation process. The error value can reflect the accuracy of the accuracy score made by the translation process on the plurality of candidate translations. Compared with the prior art which does not evaluate the accuracy score, the evaluation of the translation effect of the present application is more comprehensive.

Description

Technical Field

[0001] This application relates to the field of computers, and specifically to a method, apparatus, electronic device, and storage medium for evaluating translation effectiveness. Background Technology

[0002] Existing machine translation models typically translate the original text first, obtaining multiple candidate translations. Then, the machine translation model selects the translation it deems most accurate from these candidate translations and outputs that translation.

[0003] However, the translations selected by machine translation models are not necessarily the most accurate. Therefore, existing methods for evaluating the translation performance of machine translation models based on their output are simplistic and not comprehensive enough. Summary of the Invention

[0004] This application provides a translation effect evaluation method, apparatus, electronic device, and storage medium, which can improve the problem of incompleteness in the existing technology for evaluating translation effects.

[0005] This application provides a translation effect evaluation method, including: translating the original text to obtain multiple candidate translations obtained from the translation process and an accuracy score corresponding to each candidate translation, wherein the accuracy score is a score determined by the translation process that reflects the translation accuracy of the corresponding candidate translation; Based on the accuracy score and the preset sorting rules, the multiple candidate translations are sorted to obtain a first sorting sequence; Each candidate translation is evaluated according to the translation quality assessment index to obtain a quality assessment score for each candidate translation. The translation quality assessment index is an indicator that measures the degree of matching between the candidate translation and the standard translation. Based on the quality assessment score and the preset sorting rules, the multiple candidate translations are sorted to obtain a second sorting sequence; Calculate the error value between the first sorted sequence and the second sorted sequence, wherein the error value reflects the translation effect of the translation process; The error values ​​are displayed so that users can obtain the translation effect of the translation process.

[0006] This application embodiment also provides a translation effect evaluation device, including: a candidate translation acquisition unit, used to perform translation processing on the original text to obtain multiple candidate translations obtained in this translation processing and an accuracy score corresponding to each candidate translation, wherein the accuracy score is a score determined by the translation processing that reflects the translation accuracy of the corresponding candidate translation; The first sorting unit is used to sort the multiple candidate translations according to the accuracy score and the preset sorting rules to obtain a first sorting sequence; A quality assessment unit is used to evaluate each candidate translation according to translation quality assessment indicators to obtain a quality assessment score for each candidate translation, wherein the translation quality assessment indicators are indicators that measure the degree of matching between the candidate translation and the standard translation. The second sorting unit is used to sort the multiple candidate translations according to the quality assessment score and the preset sorting rules to obtain a second sorting sequence; An error calculation unit is used to calculate the error value between the first sorted sequence and the second sorted sequence, wherein the error value reflects the translation effect of the translation process; An error value display unit is used to display the error value so that the user can obtain the translation effect of the translation processing.

[0007] In some embodiments, the original text includes l The phrases, and the multiple candidate translations include k A candidate translation; the candidate translation acquisition unit includes: Candidate phrase determination subunit, used to determine the subunit based on the original text and k The previously generated ones i -1 phrases, determine multiple candidate phrases, and each candidate phrase is selected as the first... i The probability values ​​of each word group, where... i greater than 1 and not greater than l Positive integers; The probability value selection subunit is used to select, from a plurality of probability values, the one that meets a preset requirement. k 1 probability value, and obtain k The probability values ​​corresponding to the th probability value are respectively the th i a phrase; Phrase combination subunit, used to combine k The first one mentioned i Each phrase and its preceding words i -1 word combination, resulting in k Before i a phrase; The steps above are repeated sequentially in the candidate phrase determination subunit, probability value selection subunit, and phrase combination subunit to obtain the following results: k One candidate translation; The scoring calculation subunit is used for... k For each candidate translation, an accuracy score is calculated based on the probability values ​​of the multiple word groups that make up the candidate translation.

[0008] In some embodiments, the candidate phrase determining subunit is specifically used for... k The already generated first i -1 words in each of the generated prefixes i -1 phrase, based on the original text and the already generated preceding text. i -1 phrases, counted in the vocabulary. a Each of the candidate word groups is used as the first... i The probability values ​​of each word group were obtained. k * a A probability value, wherein the vocabulary includes a One alternative phrase, a It is a positive integer; The probability value selection sub-unit is specifically used to select from... k * a Select from the probability values ​​that meet the preset requirements. k 1 probability value, and obtain k The probability values ​​corresponding to the th probability value are respectively the th i A phrase.

[0009] In some embodiments, the probability value selection subunit includes: The probability value sequence subunit is used for the... k * a The probability values ​​are sorted in descending order to obtain a probability value sequence. The probability value truncation subunit is used to truncate the probability value sequence. k A probability value.

[0010] In some embodiments, the probability value selection subunit includes: The probability value average subunit is used to calculate the... k * a The average of the probability values; Randomly select a secondary subunit for use from the k * a Randomly select from a probability value k A probability value greater than the average value.

[0011] In some embodiments, the scoring calculation subunit is specifically used to calculate the product of the probability values ​​of multiple word groups that make up the candidate translation, and the product is the accuracy score corresponding to the candidate translation.

[0012] In some embodiments, the scoring calculation subunit is specifically used to perform a logarithmic operation on the probability value of each word group in a plurality of word groups that make up the candidate translation, to obtain a plurality of logarithmic operation results; The sum of the results of the multiple logarithmic operations is calculated, and the sum is the accuracy score corresponding to the candidate translation.

[0013] In some embodiments, the candidate translation acquisition unit includes: Global translation determines sub-units, used to determine all Each candidate translation and its corresponding probability value, wherein, for l Each phrase in the group except the last one has a The vocabulary includes 10 candidate phrases as the following phrases of the given phrase, and a probability value for each candidate phrase as the following phrase. a One alternative phrase, a It is a positive integer; Globally capture sub-units for use with The probability values ​​are sorted in descending order, and the first few are truncated. k Each probability value and the previous k Each probability value corresponds to a candidate translation.

[0014] In some embodiments, the quality assessment unit includes: The hit subunit is used to calculate, for each candidate translation, the number of times a phrase consisting of n consecutive words in the candidate translation is hit in the standard translation, denoted as the target hit count, where n is a positive integer; The target number subunit is used to calculate the number of word groups consisting of n consecutive words in the candidate translation, denoted as the target number. The target ratio subunit is used to calculate the target ratio of the number of target hits to the number of targets, wherein the target ratio reflects the degree of matching between the candidate translation and the standard translation based on the word group composed of the n consecutive words; The evaluation score determination subunit is used to determine the quality evaluation score corresponding to the candidate translation based on the parameter reflecting the word count of the original text and the target ratio.

[0015] In some embodiments, the hit subunit is specifically used to: if the number of times the phrase consisting of n consecutive words appears in the candidate translation is greater than the number of times the phrase consisting of n consecutive words appears in the standard translation, then the number of times the phrase consisting of n consecutive words appears in the standard translation is taken as the hit count; If the number of times the phrase consisting of n consecutive words appears in the candidate translation is less than or equal to the number of times the phrase consisting of n consecutive words appears in the standard translation, then the number of times the phrase consisting of n consecutive words appears in the candidate translation is taken as the hit count.

[0016] In some embodiments, the error calculation unit is specifically used to calculate a first value based on the order of each candidate translation in the first sorting sequence and the order of each candidate translation in the second sorting sequence, wherein the first value is used to reflect the sorting accuracy of the first sorting sequence for the plurality of candidate translations. A second value is calculated based on the order of each candidate translation in the second sorting sequence, wherein the second value is used to reflect the sorting accuracy of the second sorting sequence for the plurality of candidate translations; The error value is calculated based on the first value and the second value.

[0017] In some embodiments, the error calculation unit is specifically used to calculate a third value based on the order of each candidate translation in the first sorting sequence and the quality evaluation score corresponding to each candidate translation, wherein the third value is used to reflect the sorting accuracy of the first sorting sequence for the multiple candidate translations. A fourth value is calculated based on the order of each candidate translation in the second sorting sequence and the quality assessment score corresponding to each candidate translation, wherein the fourth value is used to reflect the sorting accuracy of the second sorting sequence for the multiple candidate translations. The error value is calculated based on the third and fourth values. In some embodiments, the error calculation unit is specifically used to calculate a third value based on the order of each candidate translation in the first sorting sequence and the quality evaluation score corresponding to each candidate translation, wherein the third value is used to reflect the sorting accuracy of the first sorting sequence for the multiple candidate translations. A fifth value is calculated based on the order of each candidate translation in the second sorting sequence and the upper limit of the quality assessment score corresponding to each candidate translation, wherein the fifth value is used to reflect the sorting accuracy of the second sorting sequence for the multiple candidate translations. The error value is calculated based on the third and fifth values.

[0018] This application also provides an electronic device, including a memory storing multiple instructions; the processor loads the instructions from the memory to execute the steps in any of the translation effect evaluation methods provided in this application.

[0019] This application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute steps in any of the translation effect evaluation methods provided in this application.

[0020] This application embodiment can translate the original text to obtain multiple candidate translations and an accuracy score for each candidate translation. Then, based on a preset sorting rule and the accuracy scores, the multiple candidate translations are sorted to obtain a first sorting sequence based on the accuracy scores. Each candidate translation is evaluated according to translation quality assessment indicators to obtain a quality assessment score for each candidate translation. Then, based on the quality assessment scores and the aforementioned preset sorting rule, the multiple candidate translations are sorted to obtain a second sorting sequence based on the quality assessment scores. The error value between the two is calculated and displayed so that the user can obtain the translation effect of the translation process.

[0021] In this application, the accuracy score is a rating given by the translation processing steps to multiple candidate translations, while the quality assessment score is a rating given based on translation quality assessment indicators. By sorting both scores according to the same ranking rules and then calculating the error between them, the difference between the score given by the translation processing steps and the score based on the translation quality assessment indicators can be obtained more accurately. This difference reflects the accuracy of the accuracy score given by the translation processing steps to multiple candidate translations. Compared with existing technologies that do not evaluate accuracy scores, this application provides a more comprehensive evaluation of translation effectiveness. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1a This is a schematic diagram of a scenario illustrating the translation effectiveness evaluation method provided in the embodiments of this application; Figure 1b This is a flowchart illustrating the translation effectiveness evaluation method provided in the embodiments of this application; Figure 1c This is a flowchart illustrating the process of obtaining the i-th word group in the translation effect evaluation method provided in this application embodiment; Figure 2 This is a schematic diagram illustrating the application of the translation effect evaluation method provided in this application embodiment in a scenario of evaluating the translation effect of a machine translation model; Figure 3 This is a schematic diagram of a translation effect evaluation device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] This application provides a method, apparatus, electronic device, and storage medium for evaluating translation effectiveness.

[0026] Specifically, the translation effectiveness evaluation device can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet, smart Bluetooth device, laptop, or personal computer (PC); the server can be a single server or a server cluster consisting of multiple servers.

[0027] In some embodiments, the translation effect evaluation device can also be integrated into multiple electronic devices. For example, the translation effect evaluation device can be integrated into multiple servers, and the translation effect evaluation method of this application can be implemented by multiple servers.

[0028] In some embodiments, the server may also be implemented as a terminal.

[0029] For example, refer to Figure 1a This electronic device can translate the original text, obtaining multiple candidate translations and an accuracy score for each candidate translation. The accuracy score reflects the translation accuracy of the corresponding candidate translation. Based on the accuracy scores and preset sorting rules, the multiple candidate translations are sorted to obtain a first sorted sequence. Each candidate translation is evaluated according to a translation quality assessment index, resulting in a quality assessment score for each candidate translation. The translation quality assessment index measures the degree of matching between the candidate translation and the standard translation. Based on the quality assessment score and preset sorting rules, the multiple candidate translations are sorted to obtain a second sorted sequence. The error value between the first and second sorted sequences is calculated, reflecting the translation effect. The error value is displayed so that the user can understand the translation effect.

[0030] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0031] In this embodiment, a method for evaluating translation effectiveness is provided, such as... Figure 1bAs shown, the specific process of this translation effectiveness evaluation method can be summarized in steps 101 to 106 as follows: 101. The original text is translated to obtain multiple candidate translations and an accuracy score for each candidate translation.

[0032] The source text and candidate translations are two different natural languages. Natural languages ​​typically refer to languages ​​that evolve naturally with culture, such as Chinese, English, and Japanese. Through translation processing, one natural language (i.e., the source text) can be translated into another natural language (i.e., multiple candidate translations).

[0033] The original text may include multiple sentences: for example, the original text may be a paragraph consisting of multiple related sentences, or it may be multiple different sentences that are not related to each other; the original text may also be a single sentence. It should be understood that the number of sentences included in the original text should not be construed as a limitation on this application.

[0034] For ease of description, let's assume the original text includes... l A phrase is a combination of words. Optionally, a single word can constitute a phrase; multiple words can also constitute a phrase, for example, two words can form a phrase, three words can form a phrase; half a word can also constitute a phrase, such as common prefixes or suffixes in English. The number of words constituting a phrase can be set by the developers according to their work habits or preferences. In this application, we will take a single word as an example to introduce this application.

[0035] The accuracy score is a score reflecting the translation accuracy of the corresponding candidate translation, determined by the translation processing. The translation processing steps can be specifically performed by the machine translation model under test; in this case, the accuracy score is the score of the machine translation model's accuracy in translating the source text into the candidate translation. The machine translation model under test can include, but is not limited to, Transformer models and Recurrent Neural Networks (RNN) models. The specific type of machine translation model should not be construed as a limitation of this application.

[0036] In one specific implementation, it is advisable to assume multiple candidate translations as follows: k One candidate translation. Step 101 specifically includes steps 1011 to 1015.

[0037] 1011. Based on the original text and k The previously generated ones i -1 phrases, determine multiple candidate phrases, and each candidate phrase is selected as the first... i The probability values ​​of each word group, where... igreater than 1 and not greater than l Positive integers.

[0038] The already generated front i -1 phrase is the first of the candidate translations i -1 phrase. k The already generated first i -1 phrases respectively with k There are one candidate translation. Multiple alternative phrases are pre-recorded in a thesaurus.

[0039] Optionally, step 1011 may include the following steps: for k The already generated first i -1 words in each of the generated prefixes i -1 phrase, based on the original text and the already generated preceding text. i -1 phrases, counted in the vocabulary. a Each of the candidate word groups is used as the first... i The probability values ​​of each word group were obtained. k * a Each probability value; the vocabulary includes a One alternative phrase, a It is a positive integer.

[0040] Please see Figure 1c For each already generated previous i -1 word groups: B1_(i-1), B2_(i-1), ..., Bk_(i-1). The machine translation model to be tested can be based on the original text and the generated words. i -1 word groups, calculate all words in the vocabulary list respectively. a Each of the candidate word groups in the nth candidate word group is used as the first... i The probability value of each word group can be obtained. a A probability value, a Each probability value and a Each of the candidate phrases corresponds to a specific phrase; please refer to [link / reference]. Figure 1c For example, for B1_(i-1), we can calculate the probabilities of it being followed by B1_(i), B2_(i), ..., Ba_(i), and there are a total of a There are several probability values; the same steps can be performed for B2_(i-1), ..., Bk_(i-1). Since there are a total of... k The already generated first i -1 phrases, therefore, there are a total of k * a A probability value, k * a Each probability value and k * aEach of the candidate word groups corresponds to one other word.

[0041] 1012. Select the probability values ​​that meet the preset requirements from among the multiple probability values. k 1 probability value, and obtain k The probability values ​​corresponding to the th probability value are respectively the th i A phrase.

[0042] Optionally, step 1012 may include the following steps: from k * a Select from the probability values ​​that meet the preset requirements. k 1 probability value, and obtain k The probability values ​​corresponding to the th probability value are respectively the th i A phrase.

[0043] Please see Figure 1c B1'_(i), B2'_(i), ..., Bk'_(i) are the selected ones. k The first i A phrase.

[0044] In one specific implementation, the step "from k * a Select from the probability values ​​that meet the preset requirements. k "a probability value" includes: for the... k * a The probability values ​​are sorted in descending order to obtain a probability value sequence; the first truncation is taken from the probability value sequence. k A probability value.

[0045] In the above implementation, the preset requirement can be the requirement with the highest probability value, therefore it can be... k * a The probability values ​​are sorted in descending order to obtain a probability value sequence, and the first few values ​​are extracted from the probability value sequence. k There are several probability values. Each time a probability value is selected, the highest value is chosen. k A single probability value can ensure the overall accuracy of the candidate translations without requiring extensive computation, thus achieving a balance between accuracy and efficiency.

[0046] In another specific implementation, step "from k * a Select from the probability values ​​that meet the preset requirements. k "a probability value" includes: calculating the... k * a The average of the probability values; from the k * a Randomly select from a probability value k A probability value greater than the average value.

[0047] In the above embodiments, after calculating k * a After averaging the probability values, we can... k * a A probability value is randomly selected from a set of probability values, and then compared with the average value. Probability values ​​that are greater than the average value are retained, until the retained probability value reaches the average value. k Each probability value. The above selection... k Using probability values ​​can greatly improve computational efficiency and save computational resources.

[0048] It should be understood that the specific requirements of the preset requirements in step 1012 may be other implementation methods besides the two listed above, and the specific requirements should not be construed as a limitation of this application.

[0049] 1013, will k The first one mentioned i Each phrase and its preceding words i -1 word combination, resulting in k Before i A phrase.

[0050] 1014. Repeat the above steps until you get... k 10 candidate translations.

[0051] For the candidate translations, except for the first phrase, all other phrases can be generated by repeating steps 1011 to 1013. The first phrase can be generated as follows: The machine translation model under test can calculate the entire vocabulary based on the source text. a Each of the candidate word groups is used as the probability value of the first word group, thus obtaining... a A probability value, a Each probability value and a Each of the candidate phrases corresponds to one another. Then from... a Select from the probability values ​​that meet the preset requirements. k 1 probability value, and obtain k The probability values ​​correspond to the first word group.

[0052] Among them, we obtained k The process of processing the first phrase is the step executed by the machine translation model under test in the first time step, resulting in... k Before i The phrases represent the machine translation model to be tested in the th... i The steps are executed at each time step.

[0053] Alternatively, in one specific implementation, the entire vocabulary... a Each candidate phrase may include a terminator, if obtained in a certain instance... k The already generated previous i In each phrase, there exists a pre-generated word. i The generated phrase, the first i The first word of the phrase i If the first phrase is a terminator, it means that a complete sentence has been selected as a candidate translation, and then the next step is to proceed. i When selecting +1 phrase, the selection quantity can be... k -1.

[0054] For example, see Figure 1c Let B1_(i) be a terminator, then it means that the part containing B1_(i) has already been generated. i Each phrase already forms a complete sentence. Therefore, in selecting the first... i When adding 1 more phrases, we can consider B2_(i), B3_(i), ..., Bk_(i). k -1 phrases, calculate separately a Each of the candidate word groups in the nth candidate word group is used as the first... i +1 probability value for phrases, that is, a total of ( k -1)* a A probability value. Then from ( k -1)* a Select from the probability values ​​that meet the preset requirements. k -1) probability values, and obtain ( k The probability values ​​corresponding to the -1)th probability values ​​are respectively the th i +1 phrase.

[0055] 1015. Regarding k For each candidate translation, an accuracy score is calculated based on the probability values ​​of the multiple word groups that make up the candidate translation.

[0056] It is understandable that the candidate translations are obtained by translating the original text; therefore, although the original text includes... l There are 100 phrases, but the number of phrases in the candidate translations is not necessarily 100. l One, can be greater than l One, or less than l Although the number of phrases in the candidate translation is uncertain, each phrase in the multiple phrases that make up a complete candidate translation has its own corresponding probability value, and the accuracy score of the candidate translation can be calculated based on the probability value.

[0057] In one specific implementation, the step "calculate the accuracy score corresponding to the candidate translation based on the probability values ​​of the multiple word groups that make up the candidate translation" includes: calculating the product of the probability values ​​of the multiple word groups that make up the candidate translation, wherein the product is the accuracy score corresponding to the candidate translation.

[0058] In another specific embodiment, the step "calculating the accuracy score corresponding to the candidate translation based on the probability values ​​of multiple word groups constituting the candidate translation" includes: performing a logarithmic operation on the probability value of each word group in the multiple word groups constituting the candidate translation to obtain multiple logarithmic operation results; calculating the sum of the multiple logarithmic operation results, where the sum is the accuracy score corresponding to the candidate translation. It should be understood that the specific method for calculating the accuracy score corresponding to a candidate translation based on the probability values ​​of multiple word groups in a candidate translation should not be construed as a limitation of this application.

[0059] By identifying multiple candidate translations corresponding to the original text using the above method, we can ensure the overall accuracy of the candidate translations without requiring extensive calculations, thus achieving a balance between accuracy and efficiency.

[0060] In another specific embodiment, step 101 specifically includes the following: determining all Each candidate translation and its corresponding probability value, wherein, for l Each phrase in the group except the last one has a The vocabulary includes 10 candidate phrases as the following phrases of the given phrase, and a probability value for each candidate phrase as the following phrase. a One alternative phrase, a It is a positive integer; for The probability values ​​are sorted in descending order, and the first few are truncated. k Each probability value and the previous k Each probability value corresponds to a candidate translation.

[0061] For the original text included l Each phrase in the group except the last one has a There are 10 candidate phrases that will be used as the following phrases for the current phrase. Therefore, without filtering the candidate phrases, a total of 1000 phrases will be generated. One candidate translation, can obtain all One candidate translation and all The probability values ​​corresponding to each candidate translation, and then from Choose the largest probability value from the given probability values. k Each probability value, and the largest one. k Each probability value corresponds to a candidate translation. Through the above implementation method, the selected...k The candidate translations with the highest accuracy scores are... k 10 candidate translations.

[0062] 102. Based on the accuracy score and the preset sorting rules, sort the multiple candidate translations to obtain a first sorting sequence.

[0063] The preset sorting rule is a pre-set sorting rule. Optionally, the preset sorting rule can be a sorting rule that arranges the accuracy scores in descending order or a sorting rule that arranges the accuracy scores in ascending order. The specific rules of the preset sorting rule should not be construed as a limitation of this application.

[0064] For ease of description, let's take the sorting rule of descending order as an example to continue the explanation.

[0065] 103. Evaluate each candidate translation according to the translation quality assessment indicators to obtain the quality assessment score corresponding to each candidate translation.

[0066] The translation quality assessment index is used to measure the degree of matching between the candidate translation and the standard translation. A high or low quality assessment score reflects the degree of matching between the candidate translation and the standard translation.

[0067] Optionally, step 103 may specifically include the following steps 1031 to 1034: 1031. For each candidate translation, calculate the number of times a phrase consisting of n consecutive words in the candidate translation is hit in the standard translation, denoted as the target hit count, where n is a positive integer.

[0068] A series of n consecutive words refers to n words that are adjacent and consecutive in the candidate translation, which can be represented by an n-gram. In one implementation, n can take the values ​​1, 2, 3, and 4. Then, for a word group consisting of 1 consecutive word, a word group consisting of 2 consecutive words, a word group consisting of 3 consecutive words, and a word group consisting of 4 consecutive words, steps 1031 to 1033 can be executed.

[0069] For example, let's consider a candidate translation: I have an apple. In the above candidate translations: There are four 1-grams consisting of consecutive single words: I, have, an, and apple. There are three 2-gram phrases consisting of two consecutive words: I have, have an, and an apple. There are two 3-gram phrases consisting of three consecutive words: "I have an" and "have an apple". There is one 4-gram consisting of four consecutive words: I have an apple.

[0070] The hit count refers to the number of times a given n-gram appears in a candidate translation that matches the number of times that n-gram appears in the standard translation. For example, if an n-gram appears 8 times in a candidate translation and 10 times in the standard translation, it means that the n-gram hits 8 out of 10 times in the standard translation; conversely, if an n-gram appears 12 times in a candidate translation and 10 times in the standard translation, it means that the n-gram hits all 10 times in the standard translation.

[0071] Alternatively, the specific calculation of the number of hits can be achieved through the following steps: If the number of times the phrase consisting of n consecutive words appears in the candidate translation is greater than the number of times the phrase consisting of n consecutive words appears in the standard translation, then the number of times the phrase consisting of n consecutive words appears in the standard translation is taken as the hit count; if the number of times the phrase consisting of n consecutive words appears in the candidate translation is less than or equal to the number of times the phrase consisting of n consecutive words appears in the standard translation, then the number of times the phrase consisting of n consecutive words appears in the candidate translation is taken as the hit count.

[0072] 1032. Calculate the number of word groups consisting of n consecutive words in the candidate translation, and denote it as the target number.

[0073] 1033. Calculate the target hit count to the target number, whereby the target ratio reflects the degree of matching between the candidate translation and the standard translation based on the word group composed of the n consecutive words.

[0074] According to the formula Calculate the target ratio ,in, This represents the number of times an n-gram in the candidate translation is matched in the standard translation. Used to count the number of candidate sentences in the above candidate translations. Used to count the number of n-grams in each candidate sentence. This represents the minimum number of times a certain n-gram appears in the candidate translation or the standard translation. Let n be the number of n-grams in the candidate translations. This represents the number of times a certain n-gram appears in the candidate translation.

[0075] 1034. Determine the quality assessment score corresponding to the candidate translation based on the parameter reflecting the word count of the original text and the target ratio.

[0076] According to the formula Calculate quality assessment score .

[0077] in, BP A parameter that reflects the word count of the original text. , c The length of the average vocabulary of sentences in the candidate translations. c It can be calculated by dividing the total number of words in the candidate translation by the number of sentences in the candidate translation; r This refers to the average vocabulary length of sentences in the standard translation. r It can be calculated by dividing the total number of words in the standard translation by the number of sentences in the standard translation; N is the maximum value of n in the n-gram; For the corresponding The weight value can be set manually by the developer or it can be 1 / N.

[0078] The quality assessment score of each candidate translation can be calculated through steps 1031 to 1033 above.

[0079] 104. Based on the quality assessment score and the preset sorting rules, sort the multiple candidate translations to obtain a second sorting sequence.

[0080] The preset sorting rule in step 104 is the same as the preset sorting rule in step 102. We will continue to use a sorting rule with a descending order as an example. Step 104 yields a second sorted sequence based on the quality assessment score in descending order.

[0081] 105. Calculate the error value between the first sorted sequence and the second sorted sequence.

[0082] The error value reflects the translation effect of the translation process.

[0083] In one specific embodiment, step 105 includes the following steps: calculating a first value based on the order of each candidate translation in the first sorting sequence and the order of each candidate translation in the second sorting sequence, wherein the first value is used to reflect the sorting accuracy of the first sorting sequence for the plurality of candidate translations; calculating a second value based on the order of each candidate translation in the second sorting sequence, wherein the second value is used to reflect the sorting accuracy of the second sorting sequence for the plurality of candidate translations; and calculating the error value based on the first value and the second value.

[0084] According to the formula: Calculate the first value ,in, for k The first sorted sequence consists of 10 candidate translations. for k Any one of the candidate translations; , for Second sorting sequence The serial number value in the middle, for k The second sorting sequence consists of candidate translations; . The specific value can measure the first sorted sequence In k The ranking accuracy of each candidate translation is indicated by a numerical value; the larger the value, the higher the ranking accuracy, and the smaller the value, the lower the ranking accuracy.

[0085] According to the formula: Calculate the second value . The specific value can measure the second sorted sequence In k The ranking accuracy of each candidate translation is indicated by a numerical value; the larger the value, the higher the ranking accuracy, and the smaller the value, the lower the ranking accuracy.

[0086] According to the formula: Calculate the error value .

[0087] In the above embodiments, it is based on k Error value calculation performed on each candidate translation, i.e. k The first sorting sequence consists of candidate translations. and k The second sorting sequence consists of candidate translations. Calculation of the error value between them. Based on kThe calculations performed on each candidate translation can achieve the translation effect of translation processing while reducing the amount of computation.

[0088] Alternatively, if computing resources permit, all can also be... Error values ​​are calculated for each candidate translation: based on all The accuracy score for each candidate translation in the candidate translations will be calculated. The candidate translations are sorted in descending order of accuracy score to obtain the sorted sequence. Based on all The quality assessment score for each candidate translation in the candidate translations will be... The candidate translations are sorted in descending order of their quality assessment scores to obtain the sorted sequence. Then sort the sequence. With sorted sequence Calculate the error value between them. Sort sequence. With sorted sequence The error values ​​between them can more accurately reflect the translation effect of the translation process.

[0089] In another specific embodiment, step 105 includes the following steps: calculating a third value based on the order of each candidate translation in the first sorting sequence and the quality assessment score corresponding to each candidate translation, wherein the third value is used to reflect the sorting accuracy of the first sorting sequence for the multiple candidate translations; calculating a fourth value based on the order of each candidate translation in the second sorting sequence and the quality assessment score corresponding to each candidate translation, wherein the fourth value is used to reflect the sorting accuracy of the second sorting sequence for the multiple candidate translations; and calculating the error value based on the third value and the fourth value. According to the formula: Calculate the third value ,in, Candidate translations Quality assessment score Through the formula above calculate.

[0090] According to the formula: Calculate the fourth value .

[0091] According to the formula: Calculate the error value .

[0092] In another specific embodiment, step 105 includes the following steps: calculating a third value based on the order of each candidate translation in the first sorting sequence and the quality assessment score corresponding to each candidate translation, wherein the third value is used to reflect the sorting accuracy of the first sorting sequence for the multiple candidate translations; calculating a fifth value based on the order of each candidate translation in the second sorting sequence and the upper limit of the quality assessment score corresponding to each candidate translation, wherein the fifth value is used to reflect the sorting accuracy of the second sorting sequence for the multiple candidate translations; and calculating the error value based on the third value and the fifth value.

[0093] because Need to do all A sorted sequence of candidate translations The processing involves a large amount of computation, therefore it can be done using the fifth value. approximate .

[0094] According to the formula: Calculate the error value .

[0095] 106. Display the error value so that the user can obtain the translation effect of the translation processing.

[0096] After calculating the error value, it can be displayed so that users can intuitively understand the translation effect of the translation process.

[0097] The translation performance evaluation scheme provided in this application can be applied not only to evaluating the translation performance of a machine translation model under test, but also to content generation, such as article summary generation. In the process of generating an article summary based on its content, multiple candidate summaries are generated, and these candidate summaries need to be ranked. The method provided in this application can also evaluate the quality of the generated content.

[0098] The method provided in this application embodiment can translate the original text to obtain multiple candidate translations and an accuracy score for each candidate translation. Then, based on a preset sorting rule and the accuracy scores, the multiple candidate translations are sorted to obtain a first sorting sequence based on the accuracy scores. Each candidate translation is evaluated according to a translation quality assessment index to obtain a quality assessment score for each candidate translation. Then, based on the quality assessment score and the aforementioned preset sorting rule, the multiple candidate translations are sorted to obtain a second sorting sequence based on the quality assessment score. The error value between the two is calculated and displayed so that the user can obtain the translation effect of the translation process.

[0099] This application embodiment can obtain the difference between the score made by the translation processing step and the score made according to the translation quality assessment index by calculating the error value between the first sorting sequence and the second sorting sequence. This difference can reflect the accuracy of the accuracy score made by the translation processing step for multiple candidate translations. Compared with the prior art, which does not evaluate the accuracy score, this application provides a more comprehensive evaluation of the translation effect.

[0100] The method described in the above embodiments will be further described in detail below.

[0101] In this embodiment, the method of this application embodiment will be described in detail by taking the evaluation of the translation effect of the machine translation model under test as an example.

[0102] like Figure 2 As shown, the specific process of a translation effectiveness evaluation method is as follows: 201. Input the original text into the machine translation model under test to obtain multiple candidate translations output by the machine translation model under test and the accuracy score corresponding to each candidate translation. The accuracy score is obtained by the machine translation model under test.

[0103] 202. Sort the multiple candidate translations in descending order of accuracy scores to obtain the first sorted sequence.

[0104] 203. For each candidate translation, calculate the quality assessment score of the candidate translation based on the degree of matching between the candidate translation and the standard translation.

[0105] 204. Sort the multiple candidate translations in descending order of quality assessment scores to obtain the second sorting sequence.

[0106] 205. Based on the order of multiple candidate translations in the first sorting sequence, the order of multiple candidate translations in the second sorting sequence, and the quality assessment score of each candidate translation, calculate the error value between the first sorting sequence and the second sorting sequence.

[0107] Among them, the error value reflects the translation effect of the machine translation model under test. The smaller the error value, the better the translation effect of the translation model under test.

[0108] 206. Display the error value so that the user can obtain the translation effect of the machine translation model under test based on the error value.

[0109] As shown above, the accuracy score is the score given by the machine translation model under test to multiple candidate translations, while the quality assessment score is the score given to multiple candidate translations based on the degree of matching between the candidate translations and the standard translation. By sorting both scores in descending order and then calculating the error between them, the difference between the score given by the machine translation model under test and the score based on the degree of matching between the candidate translations and the standard translation can be obtained more accurately. This difference reflects the accuracy of the accuracy score given by the machine translation model under test to multiple candidate translations and the model's ability to rank the candidate translations. Compared with existing technologies that do not evaluate the accuracy of the machine translation model's accuracy score, this application provides a more comprehensive evaluation of the translation performance of the machine translation model.

[0110] To better implement the above methods, this application also provides a translation effectiveness evaluation device, which can be integrated into an electronic device, such as a terminal or server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster composed of multiple servers.

[0111] For example, in this embodiment, the method of this application embodiment will be described in detail by taking the translation effect evaluation device specifically integrated into an electronic device as an example.

[0112] For example, such as Figure 3 As shown, the translation effectiveness evaluation device may include: The candidate translation acquisition unit 301 is used to translate the original text to obtain multiple candidate translations and an accuracy score corresponding to each candidate translation. The accuracy score is a score determined by the translation process that reflects the translation accuracy of the corresponding candidate translation.

[0113] The first sorting unit 302 is used to sort the multiple candidate translations according to the accuracy score and the preset sorting rules to obtain a first sorting sequence.

[0114] The quality assessment unit 303 is used to evaluate each candidate translation according to the translation quality assessment index to obtain a quality assessment score corresponding to each candidate translation, wherein the translation quality assessment index is an indicator that measures the degree of matching between the candidate translation and the standard translation.

[0115] The second sorting unit 304 is used to sort the multiple candidate translations according to the quality assessment score and the preset sorting rules to obtain a second sorting sequence.

[0116] Error calculation unit 305 is used to calculate the error value between the first sorted sequence and the second sorted sequence, wherein the error value reflects the translation effect of the translation process.

[0117] The error value display unit 306 is used to display the error value so that the user can obtain the translation effect of the translation processing.

[0118] In some embodiments, the original text includes l The phrases, and the multiple candidate translations include k A candidate translation; the candidate translation acquisition unit 301 includes: Candidate phrase determination subunit, used to determine the subunit based on the original text and k The previously generated ones i -1 phrases, determine multiple candidate phrases, and each candidate phrase is selected as the first... i The probability values ​​of each word group, where... i greater than 1 and not greater than l Positive integers; The probability value selection subunit is used to select, from a plurality of probability values, the one that meets a preset requirement. k 1 probability value, and obtain k The probability values ​​corresponding to the th probability value are respectively the th i a phrase; Phrase combination subunit, used to combine k The first one mentioned i Each phrase and its preceding words i -1 word combination, resulting in k Before i a phrase; The steps above are repeated sequentially in the candidate phrase determination subunit, probability value selection subunit, and phrase combination subunit to obtain the following results: k One candidate translation; The scoring calculation subunit is used for... k For each candidate translation, an accuracy score is calculated based on the probability values ​​of the multiple word groups that make up the candidate translation.

[0119] In some embodiments, the candidate phrase determining subunit is specifically used for... k The already generated first i -1 words in each of the generated prefixes i -1 phrase, based on the original text and the already generated preceding text. i -1 phrases, counted in the vocabulary. a Each of the candidate word groups is used as the first... i The probability values ​​of each word group were obtained. k * aA probability value, wherein the vocabulary includes a One alternative phrase, a It is a positive integer; The probability value selection sub-unit is specifically used to select from... k * a Select from the probability values ​​that meet the preset requirements. k 1 probability value, and obtain k The probability values ​​corresponding to the th probability value are respectively the th i A phrase.

[0120] In some embodiments, the probability value selection subunit includes: The probability value sequence subunit is used for the... k * a The probability values ​​are sorted in descending order to obtain a probability value sequence. The probability value truncation subunit is used to truncate the probability value sequence. k A probability value.

[0121] In some embodiments, the probability value selection subunit includes: The probability value average subunit is used to calculate the... k * a The average of the probability values; Randomly select a secondary subunit for use from the k * a Randomly select from a probability value k A probability value greater than the average value.

[0122] In some embodiments, the scoring calculation subunit is specifically used to calculate the product of the probability values ​​of multiple word groups that make up the candidate translation, and the product is the accuracy score corresponding to the candidate translation.

[0123] In some embodiments, the scoring calculation subunit is specifically used to perform a logarithmic operation on the probability value of each word group in a plurality of word groups that make up the candidate translation, to obtain a plurality of logarithmic operation results; The sum of the results of the multiple logarithmic operations is calculated, and the sum is the accuracy score corresponding to the candidate translation.

[0124] In some embodiments, the candidate translation acquisition unit 301 includes: Global translation determines sub-units, used to determine all Each candidate translation and its corresponding probability value, wherein, for l Each phrase in the group except the last one has a The vocabulary includes 10 candidate phrases as the following phrases of the given phrase, and a probability value for each candidate phrase as the following phrase.a One alternative phrase, a It is a positive integer; Globally capture sub-units for use with The probability values ​​are sorted in descending order, and the first few are truncated. k Each probability value and the previous k Each probability value corresponds to a candidate translation.

[0125] In some embodiments, the quality assessment unit 303 includes: The hit subunit is used to calculate, for each candidate translation, the number of times a phrase consisting of n consecutive words in the candidate translation is hit in the standard translation, denoted as the target hit count, where n is a positive integer; The target number subunit is used to calculate the number of word groups consisting of n consecutive words in the candidate translation, denoted as the target number. The target ratio subunit is used to calculate the target ratio of the number of target hits to the number of targets, wherein the target ratio reflects the degree of matching between the candidate translation and the standard translation based on the word group composed of the n consecutive words; The evaluation score determination subunit is used to determine the quality evaluation score corresponding to the candidate translation based on the parameter reflecting the word count of the original text and the target ratio.

[0126] In some embodiments, the hit subunit is specifically used to: if the number of times the phrase consisting of n consecutive words appears in the candidate translation is greater than the number of times the phrase consisting of n consecutive words appears in the standard translation, then the number of times the phrase consisting of n consecutive words appears in the standard translation is taken as the hit count; If the number of times the phrase consisting of n consecutive words appears in the candidate translation is less than or equal to the number of times the phrase consisting of n consecutive words appears in the standard translation, then the number of times the phrase consisting of n consecutive words appears in the candidate translation is taken as the hit count.

[0127] In some embodiments, the error calculation unit 305 is specifically used to calculate a first value based on the order of each candidate translation in the first sorting sequence and the order of each candidate translation in the second sorting sequence, wherein the first value is used to reflect the sorting accuracy of the first sorting sequence for the plurality of candidate translations. A second value is calculated based on the order of each candidate translation in the second sorting sequence, wherein the second value is used to reflect the sorting accuracy of the second sorting sequence for the plurality of candidate translations; The error value is calculated based on the first value and the second value.

[0128] In some embodiments, the error calculation unit 305 is specifically used to calculate a third value based on the order of each candidate translation in the first sorting sequence and the quality evaluation score corresponding to each candidate translation, wherein the third value is used to reflect the sorting accuracy of the first sorting sequence for the plurality of candidate translations. A fourth value is calculated based on the order of each candidate translation in the second sorting sequence and the quality assessment score corresponding to each candidate translation, wherein the fourth value is used to reflect the sorting accuracy of the second sorting sequence for the multiple candidate translations. The error value is calculated based on the third and fourth values. In some embodiments, the error calculation unit 305 is specifically used to calculate a third value based on the order of each candidate translation in the first sorting sequence and the quality evaluation score corresponding to each candidate translation, wherein the third value is used to reflect the sorting accuracy of the first sorting sequence for the plurality of candidate translations. A fifth value is calculated based on the order of each candidate translation in the second sorting sequence and the upper limit of the quality assessment score corresponding to each candidate translation, wherein the fifth value is used to reflect the sorting accuracy of the second sorting sequence for the multiple candidate translations. The error value is calculated based on the third and fifth values.

[0129] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0130] As described above, the translation effect evaluation device in this embodiment comprises: a candidate translation acquisition unit for translating the original text to obtain multiple candidate translations and an accuracy score for each candidate translation; a first sorting unit for sorting the multiple candidate translations according to the accuracy scores and preset sorting rules to obtain a first sorting sequence; a quality evaluation unit for evaluating each candidate translation according to translation quality evaluation indicators to obtain a quality evaluation score for each candidate translation; a second sorting unit for sorting the multiple candidate translations according to the quality evaluation scores and preset sorting rules to obtain a second sorting sequence; an error calculation unit for calculating the error value between the first sorting sequence and the second sorting sequence; and an error value display unit for displaying the error value so that the user can obtain the translation effect of the translation process.

[0131] Therefore, the embodiments of this application can improve the comprehensiveness of the evaluation of translation effects.

[0132] This application also provides an electronic device, which can be a terminal, a server, or other similar device. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0133] In some embodiments, the translation effect evaluation device can also be integrated into multiple electronic devices. For example, the translation effect evaluation device can be integrated into multiple servers, and the translation effect evaluation method of this application can be implemented by multiple servers.

[0134] In this embodiment, the electronic device will be used as an example for detailed description, such as... Figure 4 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically: The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input module 404, and a communication module 405. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 401 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, it performs various functions and processes data, thereby performing overall detection of the electronic device. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 401.

[0135] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0136] The electronic device also includes a power supply 403 that supplies power to the various components. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0137] The electronic device may also include an input module 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0138] The electronic device may also include a communication module 405. In some embodiments, the communication module 405 may include a wireless module, through which the electronic device can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 405 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0139] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows: The original text is translated to obtain multiple candidate translations and an accuracy score for each candidate translation. The accuracy score is a score reflecting the translation accuracy of the corresponding candidate translation as determined by the translation process. Based on the accuracy scores and a preset sorting rule, the multiple candidate translations are sorted to obtain a first sorting sequence. Each candidate translation is evaluated according to a translation quality assessment index to obtain a quality assessment score for each candidate translation. The translation quality assessment index measures the degree of matching between the candidate translation and the standard translation. Based on the quality assessment score and the preset sorting rule, the multiple candidate translations are sorted to obtain a second sorting sequence. The error value between the first sorting sequence and the second sorting sequence is calculated, where the error value reflects the translation effect of the translation process. The error value is displayed so that users can understand the translation effect of the translation process.

[0140] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0141] As can be seen from the above, the embodiments of this application can more accurately obtain the difference between the score made by the translation processing step and the score made according to the translation quality evaluation index. This difference can reflect the accuracy of the accuracy score made by the translation processing step for multiple candidate translations. Compared with the prior art, which does not evaluate the accuracy score, the evaluation of the translation effect of this application is more comprehensive.

[0142] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0143] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the translation effect evaluation methods provided in embodiments of this application. For example, the instructions can execute the following steps: The original text is translated to obtain multiple candidate translations and an accuracy score for each candidate translation. The accuracy score is a score reflecting the translation accuracy of the corresponding candidate translation as determined by the translation process. Based on the accuracy scores and a preset sorting rule, the multiple candidate translations are sorted to obtain a first sorting sequence. Each candidate translation is evaluated according to a translation quality assessment index to obtain a quality assessment score for each candidate translation. The translation quality assessment index measures the degree of matching between the candidate translation and the standard translation. Based on the quality assessment score and the preset sorting rule, the multiple candidate translations are sorted to obtain a second sorting sequence. The error value between the first sorting sequence and the second sorting sequence is calculated, where the error value reflects the translation effect of the translation process. The error value is displayed so that users can understand the translation effect of the translation process.

[0144] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0145] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the machine translation or content generation aspects provided in the above embodiments.

[0146] Since the instructions stored in the storage medium can execute the steps in any of the translation effect evaluation methods provided in the embodiments of this application, the beneficial effects that any of the translation effect evaluation methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0147] The translation effect evaluation method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for evaluating translation effectiveness, characterized in that, include: The original text is translated to obtain multiple candidate translations and an accuracy score for each candidate translation. The accuracy score is a score determined by the translation process that reflects the translation accuracy of the corresponding candidate translation. The original text includes... l The phrases, and the multiple candidate translations include k One candidate translation, l It is a positive integer not less than 2. k For positive integers; the step includes: for k The already generated first i -1 words in each of the generated prefixes i -1 phrase, based on the original text and the already generated preceding text. i -1 phrases, counted in the vocabulary. a Each of the candidate word groups is used as the first... i The probability values ​​of each word group were obtained. k * a A probability value, wherein the vocabulary includes a One alternative phrase, a Let be a positive integer, where, i greater than 1 and not greater than l Positive integers; from k * a Select from the probability values ​​that meet the preset requirements. k 1 probability value, and obtain k The probability values ​​corresponding to the th probability value are respectively the th i a phrase; will k The first one mentioned i Each phrase and its preceding words i -1 word combination, resulting in k Before i 1 word group; repeat the above steps until you get 1 word group. k One candidate translation; for k For each candidate translation among the candidate translations, an accuracy score is calculated based on the probability values ​​of the multiple word groups that make up the candidate translation. Based on the accuracy score and the preset sorting rules, the multiple candidate translations are sorted to obtain a first sorting sequence; Each candidate translation is evaluated according to the translation quality assessment index to obtain a quality assessment score for each candidate translation. The translation quality assessment index is an indicator that measures the degree of matching between the candidate translation and the standard translation. Based on the quality assessment score and the preset sorting rules, the multiple candidate translations are sorted to obtain a second sorting sequence; Calculate the error value between the first sorted sequence and the second sorted sequence, wherein the error value reflects the translation effect of the translation process; The error values ​​are displayed so that users can obtain the translation effect of the translation process.

2. The method as described in claim 1, characterized in that, The from k * a Select from the probability values ​​that meet the preset requirements. k The probability values ​​include: Regarding the k * a The probability values ​​are sorted in descending order to obtain a probability value sequence. Before truncation from the probability value sequence k A probability value.

3. The method as described in claim 1, characterized in that, The from k * a Select from the probability values ​​that meet the preset requirements. k The probability values ​​include: Calculate the k * a The average of the probability values; From the above k * a Randomly select from a probability value k A probability value greater than the average value.

4. The method as described in claim 1, characterized in that, The step of calculating the accuracy score corresponding to the candidate translation based on the probability values ​​of multiple word groups that make up the candidate translation includes: Calculate the product of the probability values ​​of multiple word groups that make up the candidate translation, and the product is the accuracy score corresponding to the candidate translation.

5. The method as described in claim 1, characterized in that, The step of calculating the accuracy score corresponding to the candidate translation based on the probability values ​​of multiple word groups that make up the candidate translation includes: Logarithmic operations are performed on the probability values ​​of each word group in the multiple word groups that make up the candidate translations to obtain multiple logarithmic operation results; The sum of the results of the multiple logarithmic operations is calculated, and the sum is the accuracy score corresponding to the candidate translation.

6. The method as described in claim 1, characterized in that, The step of evaluating each candidate translation based on translation quality assessment indicators to obtain a quality assessment score for each candidate translation includes: For each candidate translation, the number of times a phrase consisting of n consecutive words in the candidate translation is matched in the standard translation is calculated, and denoted as the target number of matches, where n is a positive integer; Calculate the number of word groups consisting of n consecutive words in the candidate translations, and denote it as the target number; Calculate the target hit count to the target number, whereby the target ratio reflects the degree of matching between the candidate translation and the standard translation based on the word group composed of the n consecutive words; The quality assessment score corresponding to the candidate translation is determined based on the parameter reflecting the word count of the original text and the target ratio.

7. The method as described in claim 6, characterized in that, The calculation of the number of times a phrase consisting of n consecutive words in the candidate translation is matched in the standard translation includes: If the number of times the phrase consisting of n consecutive words appears in the candidate translation is greater than the number of times the phrase consisting of n consecutive words appears in the standard translation, then the number of times the phrase consisting of n consecutive words appears in the standard translation is taken as the hit count; If the number of times the phrase consisting of n consecutive words appears in the candidate translation is less than or equal to the number of times the phrase consisting of n consecutive words appears in the standard translation, then the number of times the phrase consisting of n consecutive words appears in the candidate translation is taken as the hit count.

8. The method as described in claim 1, characterized in that, The calculation of the error value between the first sorted sequence and the second sorted sequence includes: A first value is calculated based on the order of each candidate translation in the first sorting sequence and the order of each candidate translation in the second sorting sequence, wherein the first value is used to reflect the sorting accuracy of the first sorting sequence for the multiple candidate translations. A second value is calculated based on the order of each candidate translation in the second sorting sequence, wherein the second value is used to reflect the sorting accuracy of the second sorting sequence for the plurality of candidate translations; The error value is calculated based on the first value and the second value.

9. The method as described in claim 1, characterized in that, The calculation of the error value between the first sorted sequence and the second sorted sequence includes: A third value is calculated based on the order of each candidate translation in the first sorting sequence and the quality assessment score corresponding to each candidate translation, wherein the third value is used to reflect the sorting accuracy of the first sorting sequence for the multiple candidate translations. A fourth value is calculated based on the order of each candidate translation in the second sorting sequence and the quality assessment score corresponding to each candidate translation, wherein the fourth value is used to reflect the sorting accuracy of the second sorting sequence for the multiple candidate translations. The error value is calculated based on the third and fourth values.

10. The method as described in claim 1, characterized in that, The calculation of the error value between the first sorted sequence and the second sorted sequence includes: A third value is calculated based on the order of each candidate translation in the first sorting sequence and the quality assessment score corresponding to each candidate translation, wherein the third value is used to reflect the sorting accuracy of the first sorting sequence for the multiple candidate translations. A fifth value is calculated based on the order of each candidate translation in the second sorting sequence and the upper limit of the quality assessment score corresponding to each candidate translation, wherein the fifth value is used to reflect the sorting accuracy of the second sorting sequence for the multiple candidate translations. The error value is calculated based on the third and fifth values.

11. A translation effectiveness evaluation device, characterized in that, include: A candidate translation acquisition unit is used to translate the original text to obtain multiple candidate translations and an accuracy score for each candidate translation. The accuracy score is a score determined by the translation process reflecting the translation accuracy of the corresponding candidate translation. The original text includes... l The phrases, and the multiple candidate translations include k One candidate translation, l It is a positive integer not less than 2. k For positive integers; the candidate translation acquisition unit is specifically used for: for k The already generated first i -1 words in each of the generated prefixes i -1 phrase, based on the original text and the already generated preceding text. i -1 phrases, counted in the vocabulary. a Each of the candidate word groups is used as the first... i The probability values ​​of each word group were obtained. k * a A probability value, wherein the vocabulary includes a One alternative phrase, a Let be a positive integer, where, i greater than 1 and not greater than l Positive integers; from k * a Select from the probability values ​​that meet the preset requirements. k 1 probability value, and obtain k The probability values ​​corresponding to the th probability value are respectively the th i a phrase; will k The first one mentioned i Each phrase and its preceding words i -1 word combination, resulting in k Before i 1 word group; repeat the above steps until you get 1 word group. k One candidate translation; for k For each candidate translation among the candidate translations, an accuracy score is calculated based on the probability values ​​of the multiple word groups that make up the candidate translation. The first sorting unit is used to sort the multiple candidate translations according to the accuracy score and the preset sorting rules to obtain a first sorting sequence; A quality assessment unit is used to evaluate each candidate translation according to translation quality assessment indicators to obtain a quality assessment score for each candidate translation, wherein the translation quality assessment indicators are indicators that measure the degree of matching between the candidate translation and the standard translation. The second sorting unit is used to sort the multiple candidate translations according to the quality assessment score and the preset sorting rules to obtain a second sorting sequence; An error calculation unit is used to calculate the error value between the first sorted sequence and the second sorted sequence, wherein the error value reflects the translation effect of the translation process; An error value display unit is used to display the error value so that the user can obtain the translation effect of the translation processing.

12. An electronic device, characterized in that, It includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform the steps in the translation effect evaluation method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps in the translation effect evaluation method according to any one of claims 1 to 10.

Images