Automatic translation evaluation method and system based on multiple factors and semantics

Abstract

The invention provides an automatic translation evaluation method and system based on multiple factors and semantics. The automatic translation evaluation scheme based on multiple factors and semantics mainly relates to two stages, namely an offline training stage and an online evaluation stage. In the off-line training stage, annotation data is mainly utilized to train a sentence embedding network and a digital phrase embedding network, and in the on-line evaluation stage, translation pairs are mainly preprocessed. The translation quality of the translation pair is independently evaluated on multiple dimensions such as semantics, digital translation errors, unit symbol translation errors and spelling errors based on semantics and multiple factors, and finally evaluation results of the independent evaluation are combined to finally obtain comprehensive evaluation on the translation quality of the translation pair.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and more specifically, to a multi-factor and semantic-based automatic translation evaluation method and system. Background Art

[0002] For example, in the context of global vehicle manufacturing, professional documents such as operating manuals, instructions, internal specifications, design documents, engineering drawings, etc. need to be accurately translated between different languages ​​to avoid possible technical deviations. In order to translate such professional documents quickly and efficiently, a large number of multilingual translation text pairs are generated. These translation texts have great commercial value for subsequent reuse. However, since the translation pairs currently generated come from a variety of factors and most of them have uneven quality, the evaluation of translation quality has received widespread attention in the field of natural language processing. Manual quality evaluation of translated texts will be a huge workload, and multilingual translation involves professional usage of different languages, so professional personnel are needed to complete the evaluation work.

[0003] Traditional translation quality assessment mostly uses BLEU values ​​and requires a large number of human professional translators. In essence, a single BLEU quality assessment can only assess the approximate text similarity between the translation result and the reference translation, but ignores the semantic and grammatical features, so synonyms and the length of the translation will affect the effect of the translation assessment.

[0004] In addition, existing translation quality assessments only evaluate the quality score between the translated text and the reference text, without evaluating the error factors that occur during translation. There is a lack of detailed error factor analysis, and thus a lack of a comprehensive assessment method and standard.

[0005] Therefore, a solution for comprehensive and automatic evaluation of translations is needed. Summary of the invention

[0006] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0007] According to one aspect of the present invention, a method for automatic translation evaluation based on multiple factors and semantics is provided, comprising: receiving a translation pair including a source sentence and a target sentence, wherein the source sentence is a sentence to be translated written in a source language, and the target sentence is a sentence corresponding to the source sentence and translated into a target language; evaluating the translation pair at a semantic level to obtain a semantic evaluation score of the translation pair; evaluating the translation pair at multiple factor levels to obtain a multiple factor evaluation score of the translation pair, wherein the multiple factors include one or more of the following: a numerical translation error factor, a unit symbol translation error factor, and a spelling error factor; and combining the semantic evaluation score of the translation pair with the multiple factor evaluation score of the translation pair to obtain a comprehensive evaluation score of the translation pair.

[0008] According to a further embodiment of the present invention, the method further includes: performing data preprocessing on the translation pair to determine whether the target sentence in the translation pair has a predefined translation error, and the predefined translation error includes one of the following: the target sentence in the translation pair is empty; or the target sentence is completely equal to the source sentence.

[0009] According to a further embodiment of the present invention, obtaining the semantic evaluation score of the translation pair further includes: using a trained sentence embedding network in combination with a BLEU evaluation index to perform a semantic translation quality evaluation on the translation pair to obtain a semantic translation quality evaluation score, wherein the trained sentence embedding network can evaluate the semantic similarity between the two sentences based on the feature vectors of the two sentences; using a term translation library to perform a term translation quality evaluation on the translation pair to obtain a term translation quality evaluation score, wherein the term translation quality evaluation score indicates whether the term in the source sentence is correctly translated; and combining the semantic translation quality evaluation score and the term translation quality evaluation score to obtain the semantic evaluation score for the translation pair.

[0010] According to a further embodiment of the present invention, obtaining the semantic translation quality assessment score further includes: obtaining one or more reference translations for the source sentence in the translation pair via a translation application program interface; inputting the target sentence and the one or more reference translations into the trained sentence embedding network to obtain a sentence embedding network evaluation score; calculating BLEU evaluation index values ​​of the target sentence and the one or more reference translations; and combining the sentence embedding network evaluation score and the BLEU evaluation index value to obtain the semantic translation quality assessment score.

[0011] According to a further embodiment of the present invention, obtaining the multi-factor evaluation score of the translation pair further includes: extracting numeric phrases in the source sentence and the target sentence to obtain the extracted source sentence numeric phrases and the extracted target sentence numeric phrases, the numeric phrases consisting of the number itself and a number of characters before and / or after the number; identifying the numeric phrase category of the source sentence numeric phrase; formatting the numbers in the source sentence numeric phrase based on the identified numeric phrase category, the source language and the target language; matching the converted numbers with the numbers in the target sentence numeric phrase; and calculating the numeric translation error factor evaluation score based on the matching result.

[0012] According to a further embodiment of the present invention, obtaining the multi-factor evaluation score of the translation pair further includes: identifying unit symbol errors in the target sentence to calculate a unit symbol translation error factor evaluation score; identifying spelling errors in the target sentence to calculate a spelling error factor evaluation score; and calculating the multi-factor evaluation score based on the calculated digital translation error factor evaluation score, the unit symbol translation error factor evaluation score and the spelling error factor evaluation score.

[0013] According to a further embodiment of the present invention, identifying the digital phrase category of the source sentence digital phrase further includes: obtaining the source data digital phrase feature vector of the source sentence digital phrase through a trained digital phrase embedding network, wherein the trained digital phrase embedding network can evaluate the similarity between the two digital phrases based on the feature vectors of the two digital phrases; and matching the source data digital phrase feature vector with the digital phrase feature vector in a feature library to take the digital phrase category corresponding to the most similar digital phrase feature vector in the feature library as the digital phrase category of the source data digital phrase, wherein the feature library records the digital phrase feature vectors and corresponding digital phrase categories of one or more digital phrases.

[0014] According to another aspect of the present invention, a multi-factor and semantic-based automatic translation evaluation system is provided, comprising: a trained sentence embedding network, wherein the trained sentence embedding network can evaluate the semantic similarity between the two sentences based on the feature vectors of the two sentences; a trained digital phrase embedding network, wherein the trained digital phrase embedding network can evaluate the similarity between the two digital phrases based on the feature vectors of the two digital phrases; a semantic evaluation module, wherein the semantic evaluation module is configured to evaluate the translation pair at a semantic level to obtain a semantic evaluation score of the translation pair; a multi-factor evaluation module, wherein the multi-factor evaluation module is configured to evaluate the translation pair at multiple factor levels to obtain a multi-factor evaluation score of the translation pair, wherein the multiple factors include one or more of the following: a digital translation error factor, a unit symbol translation error factor, and a spelling error factor; and a comprehensive evaluation module, wherein the comprehensive evaluation module is configured to combine the semantic evaluation score of the translation pair with the multi-factor evaluation score of the translation pair to obtain a comprehensive evaluation score of the translation pair.

[0015] According to a further embodiment of the present invention, the system further includes: a data preprocessing module, which is configured to perform data preprocessing on the translation pair to determine whether the target sentence in the translation pair has a predefined translation error, wherein the predefined translation error includes one of the following: the target sentence in the translation pair is empty, or the target sentence is completely equal to the source sentence.

[0016] According to a further embodiment of the present invention, the semantic evaluation module is further configured to: use a trained sentence embedding network in combination with a BLEU evaluation index to perform a semantic translation quality evaluation on the translation pair to obtain a semantic translation quality evaluation score, wherein the trained sentence embedding network can evaluate the semantic similarity between the two sentences based on the feature vectors of the two sentences; use a term translation library to perform a term translation quality evaluation on the translation pair to obtain a term translation quality evaluation score, wherein the term translation quality evaluation score indicates whether the term in the source sentence is correctly translated; and combine the semantic translation quality evaluation score and the term translation quality evaluation score to obtain the semantic evaluation score for the translation pair.

[0017] According to a further embodiment of the present invention, obtaining the semantic translation quality assessment score further includes: obtaining one or more reference translations for the source sentence in the translation pair via a translation application program interface; inputting the target sentence and the one or more reference translations into the trained sentence embedding network to obtain a sentence embedding network evaluation score; calculating BLEU evaluation index values ​​of the target sentence and the one or more reference translations; and combining the sentence embedding network evaluation score and the BLEU evaluation index value to obtain the semantic translation quality assessment score.

[0018] According to a further embodiment of the present invention, the multi-factor evaluation module is further configured to: extract the numeric phrases in the source sentence and the target sentence to obtain the extracted source sentence numeric phrases and the extracted target sentence numeric phrases, wherein the numeric phrases are composed of the number itself and several characters before and / or after the number; identify the numeric phrase category of the source sentence numeric phrase; format-convert the numbers in the source sentence numeric phrases based on the identified numeric phrase category, the source language and the target language; match the converted numbers with the numbers in the target sentence numeric phrases; and calculate the numeric translation error factor evaluation score based on the matching result of the matching.

[0019] According to a further embodiment of the present invention, the multi-factor evaluation module is further configured to: identify unit symbol errors in the target sentence to calculate a unit symbol translation error factor evaluation score; identify spelling errors in the target sentence to calculate a spelling error factor evaluation score; and calculate the multi-factor evaluation score based on the calculated digital translation error factor evaluation score, the unit symbol translation error factor evaluation score and the spelling error factor evaluation score.

[0020] According to another aspect of the present invention, there is provided an automatic translation evaluation system based on multiple factors and semantics, comprising: an offline training subsystem, the offline training subsystem being configured to train a sentence embedding network at a semantic level using annotated sentence pairs in different languages ​​and to train a digital phrase embedding network using annotated digital phrase pairs in different languages, wherein the annotated sentence pairs include annotated sentence pairs with the same semantics and annotated sentence pairs with different semantics, and the annotated digital phrase pairs include annotated digital phrase pairs with the same category and annotated digital phrase pairs with different categories; and an online evaluation subsystem, the online evaluation subsystem comprising the semantic evaluation module, the multiple factor evaluation module and the comprehensive evaluation module as described above.

[0021] According to a further embodiment of the present invention, the trained sentence embedding network is capable of evaluating the semantic similarity between the two sentences based on the feature vectors of the two sentences, and the trained number phrase embedding network is capable of evaluating the similarity between the two number phrases based on the feature vectors of the two number phrases.

[0022] These and other features and advantages will become apparent by reading the following detailed description and by reference to the associated drawings.It is to be understood that the foregoing general description and the following detailed description are illustrative only and are not restrictive of the aspects of what is claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to understand the manner in which the above features of the present invention are used in detail, the above briefly summarized contents can be described in more detail with reference to various embodiments, some of which are shown in the accompanying drawings. However, it should be noted that the accompanying drawings only show some typical aspects of the present invention and should not be considered to limit its scope, because the description may allow for other equally effective aspects.

[0024] Figure 1 A schematic diagram of a system 100 for automatic translation evaluation based on multiple factors and semantics according to an embodiment of the present invention is shown;

[0025] Figure 2 The data flow of the semantic evaluation module 120 performing the above three processes to perform semantic evaluation according to one embodiment of the present invention is shown;

[0026] Figure 3 An embodiment of the present invention is shown. Figure 1 The digital translation error factor evaluation submodule 125-1 in the embodiment of the present invention is used to evaluate the data stream;

[0027] Figure 4 The data flow of the digital phrase category recognition process according to one embodiment of the present invention is shown;

[0028] Figure 5 A flowchart of a method 500 for automatic translation evaluation based on multiple factors and semantics according to an embodiment of the present invention is shown;

[0029] Figure 6 A flowchart 600 for step 520 is shown according to one embodiment of the present invention;

[0030] Figure 7 A flowchart 700 for step 525 is shown according to one embodiment of the present invention; and

[0031] Figure 8 A block diagram of an exemplary computing device according to one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0032] The present invention will be described in detail below in conjunction with the accompanying drawings, and the features of the present invention will be further revealed in the following specific description.

[0033] The following detailed description refers to the accompanying drawings showing exemplary embodiments of the present invention. However, the scope of the present invention is not limited to these embodiments, but is defined by the appended claims. Therefore, embodiments other than those shown in the drawings, such as modified versions of the illustrated embodiments, are still encompassed by the present invention.

[0034] References in this specification to "one embodiment," "an embodiment," "an example embodiment," etc., mean that the embodiment may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it should be understood that the particular feature, structure, or characteristic can be implemented in conjunction with other embodiments within the knowledge of a person skilled in the relevant art, whether or not explicitly described.

[0035] Unless otherwise stated, the term "A or B" used throughout this specification refers to "A and B" and "A or B" but does not mean that A and B are exclusive.

[0036] Related terms:

[0037] BLEU (Bilingual Evaluation Understudy): It is an indicator for machine translation evaluation proposed by IBM in 2002. It can be used to judge the accuracy and advantages of machine translation. BLEU measures the effect of translation by analyzing the number of times N-gram fragments co-occur in the source language text and the target language text. In other words, it checks whether the vocabulary in the source language text appears accurately in the target language text. If the keywords of the source language text appear accurately in the target language text, the BLEU value will be improved.

[0038] Source sentence and target sentence: In the context of this application, a source sentence is a sentence written in a source language to be translated. A target sentence is a sentence corresponding to the source sentence and translated into a target language. A source sentence and a corresponding target sentence constitute a translation pair.

[0039] Numeric phrase: In the context of the present application, a number itself and a number of characters before and / or after the number can form a numeric phrase, for example, "V 6.0", where "6.0" is the number itself, and the two characters "V" (i.e., V and a space) before the number can form a numeric phrase representing the version. Another example is "2.5", where "2.5" is the number itself, and a character "" (i.e., a space) before and after the number can form a numeric phrase, which can represent, for example, a chapter number, etc. Of course, depending on the different numeric phrase categories, a numeric phrase can be formed in a variety of different ways to fully represent the number itself and the numeric category.

[0040] The present invention proposes a translation automatic evaluation method and system based on multiple factors and semantics. The BLUE quality evaluation method lacks the evaluation of semantic information. At present, some translation quality evaluations lack more reference factors and directly evaluate the quality score. Therefore, the present invention proposes a translation evaluation method that utilizes multiple factors and introduces semantics, which not only evaluates the information at the semantic level, but also detects the quality of more evaluation factors.

[0041] Specifically, the multi-factor and semantic-based automatic translation evaluation scheme provided by the present invention mainly involves two stages, namely, an offline training stage and an online evaluation stage. The offline training stage mainly uses annotated data to train sentence embedding networks and digital phrase embedding networks, while the online evaluation stage mainly pre-processes the translation pairs, and based on semantics and multiple factors, the translation quality of the translation pairs is individually evaluated in multiple dimensions such as semantics, digital translation errors, unit symbol translation errors, spelling errors, etc., and finally the evaluation results of these individual evaluations are combined to finally obtain a comprehensive evaluation of the translation quality of the translation pairs.

[0042] Figure 1 FIG. 1 is a schematic diagram of a system 100 for automatic translation evaluation based on multiple factors and semantics according to an embodiment of the present invention. The system 100 may include an offline training subsystem 105 and an online evaluation subsystem 110 .

[0043] According to one embodiment of the present invention, the offline training subsystem 105 is mainly configured to train a sentence embedding network and a digital phrase embedding network at a semantic level. For the sake of clarity, Figure 1 The offline training subsystem 105 in FIG. 1 is shown as two parts, upper and lower, but this separate representation does not illustrate the time sequence of the training of the two embedding networks. In addition, according to actual needs, the sentence embedding network and the digital phrase embedding network can also be trained in different offline training subsystems.

[0044] First, the offline training of the sentence embedding network is discussed. According to one embodiment of the present invention, the offline training subsystem 105 can be configured to train the sentence embedding network at the semantic level, so that the distance between the vector encodings of semantically similar sentences is smaller, and the distance between the vector encodings of semantically different sentences is larger, thereby allowing the trained sentence embedding network to have the ability to evaluate the semantic similarity of multiple translations.

[0045] For example, referring to the offline training subsystem 105 in the upper part, the training data of the sentence embedding network is mainly divided into two categories, one is the annotated sentence pairs with the same semantics, and the other is the annotated sentence pairs with different semantics. Specifically, sentence 1 and sentence 2 can be regarded as a sentence pair. In the case of sentence pairs with the same semantics, sentence 1 can be, for example, "What's your age?", and sentence 2 can be, for example, "How old are you?", that is, sentence 1 and sentence 2 represent the same semantics (how old are you?). In the case of sentence pairs with different semantics, sentence 1 can be, for example, "Have you eaten today?", and sentence 2 can be, for example, "How is the weather today?", that is, sentence 1 and sentence 2 have different semantics. In practice, the training data can contain sentence pairs in multiple (e.g., 15 or more) languages, so that the sentence embedding network has the ability to evaluate multiple languages.

[0046] The loss function of the sentence embedding network adopts the CosineEmbeddingLoss (Cosine embedding loss) function, and the main formula is as follows:

[0047]

[0048] In the above formula, N represents the number of samples, a data pair is represented as (v1, v2, y), v1 and v2 represent the embedding vectors of sentence 1 and sentence 2 respectively, and y represents the label. If the two sentences have the same semantics, they are marked as 1, and if the two sentences have different semantics, they are marked as 0.

[0049] Thus, after training, the trained sentence embedding network can evaluate the semantic similarity between the two sentences based on their feature vectors.

[0050] Next, the training of the digital phrase embedding network is discussed. According to one embodiment of the present invention, the offline training subsystem 105 can be configured to train the digital phrase embedding network so that the vector encodings of digital phrase pairs of the same category have a smaller distance, and the vector encodings of digital phrase pairs of different categories have a larger distance, so that the trained digital phrase embedding network has the ability to evaluate the similarity between two digital phrases.

[0051] For example, referring to the lower half offline training subsystem 105, the training data of the digital phrase embedding network is mainly divided into two categories, one is the labeled digital phrase pairs with the same category, and the other is the labeled digital phrase pairs with different categories. For example, the digital phrase pairs with the same category may include phrase 1 "V 5.0" and phrase 2 "V 2.1", while the digital phrase pairs with different categories may include phrase 1 "V 5.0" and phrase 2 "$1".

[0052] In practice, the training data may contain number-word pairs in multiple (e.g., 15 or even more) languages, so that the number-word embedding network has the ability to evaluate number translations in multiple languages.

[0053] The loss function of the digital phrase embedding network adopts the CosineEmbeddingLoss (Cosine embedding loss) function, and the main formula is as follows:

[0054]

[0055] In the above formula, N represents the number of samples, a number-phrase pair is represented as (v1, v2, y), v1 and v2 represent the embedding vectors of phrase 1 and phrase 2 respectively, and y represents the label. If the two number-phrases have the same category, they are marked as 1, and if the two number-phrases have different categories, they are marked as 0.

[0056] Thus, after training, the trained number phrase embedding network can evaluate the similarity between the two number phrases based on their feature vectors.

[0057] In addition, during the training process, a feature library for digital phrases may be established, which records digital phrase feature vectors of one or more digital phrases and corresponding digital phrase categories. For example, a digital phrase feature vector of a digital phrase may be obtained through a digital phrase embedding network, and the digital phrase feature vector and the corresponding digital phrase category label (for example, the category label may be manually added) may be stored in the feature library to serve as a comparison basis for subsequent identification of the digital phrase category of the digital phrase in the source sentence.

[0058] Of course, those skilled in the art can fully understand that the feature library may not be constructed during the training process, but may be constructed offline at any time.

[0059] According to one embodiment of the present invention, the online evaluation subsystem 110 is configured to perform data preprocessing on a translation pair including a source sentence and a target sentence (e.g., a human translation text of the source sentence), and perform semantic evaluation and evaluation of multiple other factors on the translation pair after data preprocessing, and finally combine the evaluation scores of the semantic aspects and the evaluation scores of multiple other factors to obtain a final comprehensive evaluation score.

[0060] Specifically, the online evaluation subsystem 110 includes a data preprocessing module 115, a semantic evaluation module 120, a multi-factor evaluation module 125 and a comprehensive evaluation module 130. It is fully understood by those skilled in the art that the division of the above modules is only explained for the purpose of clarity. The functions of one or more of the above modules can be combined into a single module or split into more modules / sub-modules. Moreover, one or more of the above modules can be implemented in software, hardware or a combination thereof. In addition, the data flow between the modules can be implemented in a manner known in the art, which is not within the scope of the present invention.

[0061] According to an embodiment of the present invention, the data preprocessing module 115 is configured to detect whether a target sentence in a translation pair has a predefined translation error, so as to reduce excessive subsequent meaningless calculations.

[0062] For example, the data preprocessing module 115 may be configured to: when the target sentence of the translation pair is empty, it may be determined that there is no translation for the source sentence in the translation pair, and thus the translation quality problem may be directly output.

[0063] For another example, the data preprocessing module 115 may be configured to: when the source sentence and the target sentence in the translation pair are equal, it may be determined that the source sentence has not been translated, and thus a translation quality problem may be directly output.

[0064] Therefore, if the data pre-processing module 115 outputs a translation quality problem for a translation pair, the translation pair does not need to be transmitted to subsequent modules (eg, the semantic evaluation module 120 , the multi-factor evaluation module 125 ).

[0065] The following describes the semantic evaluation module 120 and the multi-factor evaluation module 125. In the context of the present application, there is no time limit for evaluating the translation pairs in the semantic evaluation module 120 and the multi-factor evaluation module 125, that is, the two modules can evaluate the translation pairs in parallel or in sequence.

[0066] According to one embodiment of the present invention, the semantic evaluation module 120 can be configured to perform three processes: (1) using the trained sentence embedding network and combining the BLEU evaluation index to perform semantic translation quality evaluation on the translation pair; (2) using the terminology translation library to perform terminology translation quality evaluation on the translation pair; (3) combining the semantic translation quality evaluation and terminology translation quality evaluation to obtain a semantic evaluation of the translation pair. Thus, the semantic evaluation module 120 not only considers the semantics expressed in the translated text, but also evaluates the translation quality of some domain-specific terms.

[0067] Specifically, refer to Figure 2 , which shows the data flow of the semantic evaluation module 120 according to one embodiment of the present invention when executing the above three processes to perform semantic evaluation.

[0068] According to one embodiment of the present invention, in the process of semantic translation quality assessment, since obtaining professional human reference translations is extremely time-consuming and labor-intensive, existing commercial translation application program interface APIs (e.g., Baidu Translate, Bing Translate, etc.) can translate the meaning of texts more accurately, so this solution uses the existing translation API interface to obtain reference translations, which are the translation results obtained by the commercial translation API interface for the source sentence in the translation pair. The reference translation can be flexibly configured, and multiple reference translations can also be used to obtain multiple reference translations to achieve better evaluation results.

[0069]

[0070] The above describes the formula for calculating the semantic translation quality assessment score. It can be seen from the above formula that the semantic translation quality assessment process is mainly divided into two parts. The first part is the sentence embedding network semantic assessment part, and the second part is the BLEU evaluation index. In the first part, the target sentence and one or more reference translations are input into the trained sentence embedding network for semantic evaluation. In the second part, the BLEU evaluation index value between the target sentence and the one or more reference translations is calculated for BLEU evaluation.

[0071] Among them, δ and ε are the weights assigned to the two parts respectively. r represents the number of reference translations, w i represents the weight of each reference translation, T represents the sentence embedding vector of the target sentence, R i Represents the sentence embedding vector of the reference translation. BP represents the length penalty factor in the BLEU evaluation index, N n Represents the number of n-gram language models used, w n is the corresponding weight, p nis the matching degree between the target sentence and the reference translation. That is, the calculation of the semantic translation quality evaluation of the present invention takes into account both the evaluation of the semantics of the reference translation and the target sentence in the trained sentence embedding network and the BLEU evaluation index of the reference translation and the target sentence. In practice, different weight values ​​δ and ε can be set according to different considerations.

[0072] It can be seen that the BLEU evaluation index only considers the matching degree at the word level, and does not take into account the accuracy of language expression. Therefore, the present invention introduces a measurement index of semantic similarity in the process of semantic translation quality evaluation, and makes the evaluation index more complete by measuring the semantic level scores of the target sentence and the reference translation.

[0073] According to an embodiment of the present invention, in the terminology translation quality evaluation stage, the present invention mainly evaluates whether some professional terms in the source sentence are translated into the correct usage of the target language in the target sentence.

[0074] The correct translation of terms comes from the long-term accumulation of translation libraries, forming a professional terminology translation library. In the terminology translation quality assessment, whether a professional term is translated into the preferred translation term in the terminology translation library is used as an evaluation method. Therefore, firstly, all the terms existing in the source sentence are searched according to the terminology translation library, and then it is determined whether there is a corresponding preferred translation term in the target sentence. The terminology translation quality assessment score is as follows:

[0075]

[0076] Where E t_ The number of terms with incorrect translations, N T_ The number of words that represent the target sentence.

[0077] According to one embodiment of the present invention, the semantic evaluation module 120 is configured to combine the semantic translation quality evaluation score and the terminology translation quality evaluation score to obtain a final semantic evaluation score, as follows:

[0078] S 语义评估得分 =μS 语义翻译 +ρS 术语翻译

[0079] Among them, μ+ρ=1, which are the weights corresponding to semantic translation quality assessment and terminology translation quality assessment respectively. In practice, the weights can be adjusted appropriately according to the proportion of different parts to achieve better evaluation results.

[0080] According to one embodiment of the present invention, the multi-factor evaluation module 125 is configured to evaluate multiple factors (e.g., digital translation errors, unit symbol translation errors, spelling errors, etc.) to obtain a multi-factor evaluation score. Specifically, the multi-factor evaluation module 125 may further include a digital translation error factor evaluation submodule 125-1, a unit symbol translation error factor evaluation submodule 125-2, and a spelling error factor evaluation submodule 125-3.

[0081] According to one embodiment of the present invention, the digital translation error factor evaluation submodule 125-1 is configured to evaluate one or more digital translations appearing in the target sentence. Specifically, the digital translation error factor evaluation submodule 125-1 can be configured to perform the following process: (1) extracting one or more digital phrases in the source sentence and the target sentence in the translation pair; (2) identifying the digital phrase category of the extracted source sentence digital phrase; (3) formatting the numbers in the extracted source sentence digital phrase based on the identified digital phrase category, source language and target language; (4) matching the converted numbers with the numbers in the extracted target sentence digital phrase; (5) calculating the digital translation quality evaluation score based on the digital matching result.

[0082] Specifically, refer to Figure 3 , which shows an embodiment of the present invention Figure 1 The digital translation error factor evaluation submodule 125-1 in the embodiment is used to perform the data flow for evaluation.

[0083] According to one embodiment of the present invention, in the process (1) of extracting the numeric phrases in the source sentence and the target sentence in the translation pair, the numeric phrases in the source sentence and the target sentence can be extracted by regularized matching. In the context of the present application, the numeric phrases extracted from the source sentence are referred to as the source sentence numeric phrases, and the numeric phrases extracted from the target sentence are referred to as the target sentence numeric phrases.

[0084] According to one embodiment of the present invention, in process (2) of identifying the category of the digital phrase of the extracted source sentence digital phrase, the feature library constructed above is used to identify the category of the digital phrase. Since the requirements for digital conversion of different categories are different, it is necessary to identify the category of the extracted digital phrase. If a classification network is trained, the classification network model has no scalability. If new categories appear, the model needs to be retrained, and the classification network model may have bottlenecks, resulting in the inability to quickly filter misclassified samples. Therefore, the present invention adopts the following Figure 4 The category of the number phrase is identified by using the number phrase feature matching method described in .

[0085] Specifically, Figure 4The data flow of the digit phrase category recognition process according to an embodiment of the present invention is shown. First, the source sentence digit phrase extracted from the source sentence is input into the trained digit phrase embedding network to obtain the feature vector of the source sentence digit phrase, and the source sentence digit phrase feature vector is matched with all the recorded digit phrase feature vectors in the feature library, so as to use the digit phrase category corresponding to the most similar digit phrase feature vector in the feature library (for example, the digit phrase feature vector in the feature library with the closest vector distance to the source sentence digit phrase feature vector) as the digit phrase category of the source sentence digit phrase.

[0086] The specific formula for feature matching is as follows:

[0087] C=L max(Cos(P1,PD))

[0088] C represents the final output digital phrase category, L represents the digital phrase category label corresponding to the digital phrase feature vector in the feature library. P1 represents the digital phrase feature vector of the input digital phrase (for example, the digital phrase feature vector of the source sentence), and PD represents all digital phrase feature vectors in the feature library.

[0089] Afterwards, the identified numeric phrase category is output for verification. If the current identification of the numeric phrase category is wrong, the corresponding entry in the feature library can be updated (e.g., manually) to modify the numeric phrase category label in the matched entry in the feature library to the correct category, thereby, the feature library can be updated / maintained in a timely manner.

[0090] According to one embodiment of the present invention, in process (3), the format conversion of the numbers in the extracted source sentence number phrases is performed based on the identified number phrase category, source language and target language. The number format conversion can be performed according to a predefined conversion rule, and the predefined conversion rule can define the conversion rule of the number format between different languages. For example, numbers such as chapter numbers and version numbers do not need to be converted, while decimals and thousandths need to be converted in different needs. For example, the decimal point and thousandths symbols in German are "," and "." respectively, while the decimal point and thousandths symbols in English are "." and "," respectively, so the numbers in the source sentence written in German need to be converted to the format in the target sentence translated into English.

[0091] According to one embodiment of the present invention, in the process (4) of matching the converted numbers with the numbers in the extracted target sentence number phrase, it is determined whether the numbers in the extracted target sentence number phrase are the same as the converted data obtained in the process (3). If they are different, it can be identified as a number translation error.

[0092] According to one embodiment of the present invention, in process (5) of calculating the digital translation quality evaluation score based on the digital matching result, the digital translation error factor evaluation score may be calculated based on the following formula:

[0093]

[0094] Among them, N 数字 represents the number of digits in the target sentence, E 数字 A number indicating the number of errors in the target sentence.

[0095] According to one embodiment of the present invention, after process (5), a list of digital translation errors may be further generated to indicate which digital translation errors occurred.

[0096] According to one embodiment of the present invention, the unit symbol translation error factor assessment submodule 125-2 is configured to perform quality assessment on one or more unit symbols in the target sentence. Generally speaking, unit symbols may be translated into different languages ​​in different forms. The present invention constructs an index library for unit symbols that may appear in different languages. In order to quickly retrieve the results, an AC automaton can be used to quickly build a dictionary tree to retrieve the unit symbols and find the corresponding target language unit symbol representation in the index library. Therefore, the unit symbol translation error factor assessment score can be calculated as follows:

[0097]

[0098] Among them, N 单位符号 represents the number of unit symbols in the target sentence, E 单位符号 The number of single-digit symbols representing errors in the target sentence.

[0099] According to an embodiment of the present invention, the unit symbol translation error factor evaluation submodule 125 - 2 may be further configured to generate a unit symbol translation error list to indicate which unit symbols have translation errors.

[0100] According to one embodiment of the present invention, the spelling error factor evaluation submodule 125-3 is configured to evaluate the spelling errors of words in the target sentence. In practice, due to human translation or writing errors, abnormal words may appear in the target sentence, so it is necessary to perform a spelling check on the target sentence to check whether there are spelling errors after translation. An existing corpus including the corpus of the target language can be used to perform spelling error checking. The spelling error factor evaluation score can be calculated as follows:

[0101]

[0102] Among them, N 拼写represents the number of words in the target sentence, E 拼写 Indicates the number of misspelled words in the target sentence.

[0103] According to an embodiment of the present invention, the spelling error factor evaluation submodule 125 - 3 may be further configured to generate a spelling error list to indicate which word or words are spelled incorrectly.

[0104] According to one embodiment of the present invention, the multi-factor evaluation module 125 may be configured to calculate a multi-factor evaluation score based on the numerical translation error factor evaluation score, the unit symbol translation error factor evaluation score, and the spelling error factor evaluation score obtained above, as shown in the following formula:

[0105] S 多因素评估得分 =αS 数字翻译错误因素评估 +βS 单位符号翻译错误因素评估 +γS 拼写错误因素评估

[0106] Among them, α, β, and γ are weights of different factors respectively, and α+β+γ=1. Each weight can be assigned according to actual needs to reflect the importance of each factor.

[0107] In summary, the multi-factor evaluation module 125 introduces the unit symbol error factor and the spelling error factor, which can be evaluated not only in daily text translation, but also in professional documents containing some unit symbols. In addition, if there are new evaluation factors in the future, this module can be easily introduced.

[0108] According to one embodiment of the present invention, the comprehensive evaluation module 130 may be configured to calculate a final comprehensive evaluation score for the translation pair based on the semantic translation quality evaluation score obtained by the semantic evaluation module 120 and the multi-factor evaluation score obtained by the multi-factor evaluation module 125, as shown in the following formula:

[0109]

[0110] in, ω are the weights of the semantic evaluation score and the multi-factor evaluation score, respectively, and Each weight can be assigned according to actual needs to reflect the importance of different factors. Among them, the comprehensive evaluation score can range from 0 to 1, and the higher the score, the better the translation quality.

[0111] According to an embodiment of the present invention, the comprehensive evaluation module 130 may also be configured to output the error lists generated by the different submodules in the multi-factor evaluation module 125 to list the specific details of digital translation errors, unit symbol translation errors and spelling errors respectively.

[0112] In summary, the comprehensive evaluation module 130 comprehensively considers all the factors to give a final reasonable score, making the overall translation quality evaluation more comprehensive. At this stage, the comprehensive evaluation module 130 can also generate corresponding detailed error factors and error results, thereby evaluating the overall translation error type distribution.

[0113] Figure 5 The flowchart of a method 500 for automatic translation evaluation based on multiple factors and semantics according to an embodiment of the present invention is shown. The method 500 may be executed by the online evaluation subsystem 110 .

[0114] At 505 , a translation pair is received, the translation pair comprising a source sentence and a target sentence, wherein the source sentence is a sentence to be translated written in a source language, and the target sentence is a sentence corresponding to the source sentence and translated into a target language.

[0115] At 510, data preprocessing is performed on the translation pair to determine whether the target sentence in the translation pair has a predefined translation error. If yes, the process proceeds to step 515, where a translation error reminder is directly output. If no, the process proceeds to step 520.

[0116] According to one embodiment of the present invention, step 510 may further include comparing the source language and the target language in the translation pair. According to one embodiment of the present invention, the predefined translation errors may include any of the following: the target sentence in the translation pair is empty; the target sentence is completely equal to the source sentence.

[0117] According to one embodiment of the present invention, step 510 may also be omitted, that is, the process may proceed directly from step 505 to step 520 .

[0118] At 520, the translation pair is evaluated at a semantic level to obtain a semantic evaluation score for the translation pair. A detailed description of step 520 is provided in Figure 6 Being explained.

[0119] At 525, the translation pair is evaluated at multiple factor levels to obtain a multi-factor evaluation score for the translation pair. According to one embodiment of the present invention, the multiple factors include one or more of the following: a number translation error factor, a unit symbol translation error factor, and a spelling error factor. A detailed description of step 525 is provided in Figure 7 Being explained.

[0120] At 530, the semantic evaluation score of the translation pair and the multi-factor evaluation score of the translation pair are combined to obtain a comprehensive evaluation score of the translation pair. According to one embodiment of the present invention, the semantic evaluation score of the translation pair and the multi-factor evaluation score of the translation pair may be weighted to obtain the comprehensive evaluation score of the translation pair.

[0121] Optionally, at 535, a list of errors for each of the plurality of factors is output.

[0122] Figure 6 A flowchart 600 for step 520 according to one embodiment of the present invention is shown.

[0123] At 605 , the trained sentence embedding network is used in combination with the BLEU evaluation index to perform semantic translation quality evaluation on the translation pair to obtain a semantic translation quality evaluation score.

[0124] According to one embodiment of the present invention, the trained sentence embedding network is capable of evaluating the semantic similarity between the two sentences based on the feature vectors of the two sentences.

[0125] According to an embodiment of the present invention, step 605 further includes obtaining one or more reference translations for the source sentence in the translation pair via a translation API.

[0126] According to one embodiment of the present invention, step 605 further includes inputting the target sentence and the one or more reference translations into a trained sentence embedding network to obtain a sentence embedding network evaluation score, calculating the BLEU evaluation index value of the target sentence and the one or more reference translations, and combining the sentence embedding network evaluation score and the BLEU evaluation index value to obtain a semantic translation quality evaluation score.

[0127] At 610 , a term translation library is used to perform a term translation quality assessment on the translation pair to obtain a term translation quality assessment score, which indicates whether the term in the source sentence is correctly translated.

[0128] According to one embodiment of the present invention, the term translation quality assessment score identifies whether the term in the source sentence in the translation pair is translated into a preferred translation term in one or more translation terms corresponding to the term in the term translation library in the target sentence in the translation pair.

[0129] At 615 , the semantic translation quality assessment score and the terminology translation quality assessment score are combined to obtain a semantic assessment score for the translation pair.

[0130] According to an embodiment of the present invention, the semantic translation quality assessment score and the terminology translation quality assessment score may be assigned different weights.

[0131] In summary, in step 520, the sentence semantic vector and term translation quality are introduced, which improves the disadvantage that the traditional BLEU index only evaluates the similarity of word text. In addition, the present invention introduces a sentence embedding network and adds a multilingual language corpus for training, so that the sentence embedding network has the ability to adapt to multiple languages ​​that may appear in the translation library.

[0132] Figure 7 A flowchart 700 for step 525 is shown according to one embodiment of the present invention.

[0133] At 705 , the number phrases in the source sentence and the target sentence in the translation pair are extracted to obtain the extracted source sentence number phrases and the extracted target sentence number phrases.

[0134] If the number of extracted target sentence number phrases is greater than 1, the following steps 710 - 720 may be iteratively performed for each target sentence number phrase.

[0135] At 710, the digital phrase category of the extracted source sentence digital phrase is identified. According to one embodiment of the present invention, step 710 further includes: obtaining a source data digital phrase feature vector of the source sentence digital phrase through a trained digital phrase embedding network, and matching the source data digital phrase feature vector with a digital phrase feature vector in a feature library, so as to use the digital phrase category corresponding to the most similar digital phrase feature vector in the feature library (for example, a digital phrase feature vector in the feature library having the closest vector distance to the feature vector) as the digital phrase category of the source data digital phrase.

[0136] According to one embodiment of the present invention, the feature library may be updated to a correct category based on the identification category error.

[0137] At 715 , the numbers of the extracted source sentence number phrases are formatted based on the identified number phrase category, the source language, and the target language.

[0138] At 720, the converted numbers are matched with the numbers in the extracted target sentence number phrases. According to one embodiment of the present invention, if the converted numbers do not match the numbers in the extracted target sentence number phrases, it can be determined that the number translation is wrong.

[0139] At 725 , a number translation error factor evaluation score is calculated based on the number matching result.

[0140] At 730 , unit symbol errors in the target sentence are identified to calculate a unit symbol translation error factor evaluation score.

[0141] At 735 , spelling errors in the target sentence are identified to calculate a spelling error factor evaluation score.

[0142] At 740, a multi-factor evaluation score is calculated based on the calculated numerical translation error factor evaluation score, unit symbol translation error factor evaluation score, and spelling error factor evaluation score. According to one embodiment of the present invention, the numerical translation error factor evaluation score, the unit symbol translation error factor evaluation score, and the spelling error factor evaluation score may be respectively assigned weights.

[0143] Therefore, the present invention introduces a feature matching scheme in the digital translation quality assessment, making the matching method more flexible. It is not necessary to redesign and train the network, and only the feature library needs to be maintained to dynamically identify different types of numbers. Therefore, there is no restriction on the identified categories, and new categories can be added more conveniently.

[0144] Figure 8 A block diagram of an exemplary computing device according to one embodiment of the present invention is shown, which is one example of a hardware device applicable to various aspects of the present invention.

[0145] refer to Figure 8 , a computing device 800 will now be described, which is an example of a hardware device applicable to various aspects of the present invention. The computing device 800 can be any machine that can be configured to perform processing and / or computing, and can be but is not limited to a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal digital assistant, a smart phone, a vehicle-mounted computer, or any combination thereof. The various methods / apparatus / server / client devices described above can be implemented in whole or in part by the computing device 800 or a similar device or system.

[0146] The computing device 800 may include components that can be connected or communicated via one or more interfaces and a bus 802. For example, the computing device 800 may include a bus 802, one or more processors 804, one or more input devices 806, and one or more output devices 808. The one or more processors 804 may be any type of processor and may include, but are not limited to, one or more general-purpose processors and / or one or more special-purpose processors (e.g., a dedicated processing chip). The input device 806 may be any type of device capable of inputting information to the computing device and may include, but are not limited to, a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. The output device 808 may be any type of device capable of presenting information and may include, but are not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The computing device 800 may also include or be connected to a non-transient storage device 810, which may be any storage device that is non-transient and capable of data storage, and may include, but is not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a floppy disk, a hard disk, a tape or any other magnetic medium, an optical disk or any other optical medium, a ROM (read-only memory), a RAM (random access memory), a cache memory, and / or any memory chip or cassette, and / or any other medium from which a computer can read data, instructions, and / or code. The non-transient storage device 810 may be detachable from the interface. The non-transient storage device 810 may have data / instructions / code for implementing the above methods and steps. The computing device 800 may also include a communication device 812. The communication device 812 can be any type of device or system that can communicate with internal devices and / or with a network and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device and / or a chipset, such as a Bluetooth device, an IEEE 1302.11 device, a WiFi device, a WiMax device, a cellular communication device and / or the like.

[0147] When the computing device 800 is used as an in-vehicle device, it can also be connected to external devices (e.g., a GPS receiver, sensors for sensing different environmental data (such as an accelerometer, a wheel speed sensor, a gyroscope, etc.)). In this way, the computing device 800 can, for example, receive positioning data and sensor data indicating the vehicle's form condition. When the computing device 800 is used as an in-vehicle device, it can also be connected to other devices (e.g., an engine system, wipers, an anti-lock brake system, a vehicle climate setting system, etc.) for controlling the travel and operation of the vehicle (e.g., vehicle climate settings).

[0148] In addition, the non-transitory storage device 810 may have map information and software components so that the processor 804 can implement route guidance processing. In addition, the output device 806 may include a display for displaying a map, a positioning mark of the vehicle, and an image indicating the driving status of the vehicle. The output device 806 may also include a speaker or a headphone interface for audio guidance.

[0149] The bus 802 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. In particular, for vehicle-mounted devices, the bus 802 may also include a Controller Area Network (CAN) bus or other structures designed for automotive applications.

[0150] The computing device 800 may also include a working memory 814 , which may be any type of working memory capable of storing instructions and / or data that facilitate the operation of the processor 804 and may include, but is not limited to, a random access memory and / or a read-only storage device.

[0151] Software components may be located in the working memory 814, including but not limited to an operating system 816, one or more application programs 818, drivers and / or other data and codes. Instructions for implementing the above methods and steps may be included in the one or more application programs 818, and the modules / units / components of the aforementioned various devices / servers / client devices may be implemented by the processor 804 reading and executing the instructions of the one or more application programs 818.

[0152] It should also be recognized that changes may be made according to specific needs. For example, custom hardware may also be used, and / or specific components may be implemented in hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. In addition, connections to other computing devices, such as network input / output devices, etc. may be employed. For example, programming hardware (e.g., programmable logic circuits including field programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs)) with assembly language or hardware programming languages ​​(e.g., VERILOG, VHDL, C++) may be used to implement part or all of the disclosed methods and devices using logic and algorithms according to the present invention.

[0153] Although various aspects of the present invention have been described so far with reference to the accompanying drawings, the above-described methods, systems and devices are merely examples, and the scope of the present invention is not limited to these aspects, but is limited only by the appended claims and their equivalents. Various components may be omitted or replaced by equivalent components. In addition, the steps may be implemented in an order different from the order described in the present invention. In addition, various components may be combined in various ways. It is also important that, as technology develops, many of the components described may be replaced by equivalent components that appear later.

Claims

1. A multi-factor and semantic-based automatic translation evaluation method, comprising: receiving a translation pair including a source sentence and a target sentence, wherein the source sentence is a sentence to be translated written in a source language, and the target sentence is a sentence corresponding to the source sentence and translated into a target language; Evaluating the translation pair at a semantic level to obtain a semantic evaluation score of the translation pair; Evaluating the translation pair at multiple factor levels to obtain a multi-factor evaluation score for the translation pair, wherein the multiple factors include one or more of the following: a number translation error factor, a unit symbol translation error factor, and a spelling error factor; as well as The semantic evaluation score of the translation pair and the multi-factor evaluation score of the translation pair are combined to obtain a comprehensive evaluation score of the translation pair.

2. The method of claim 1, further comprising: Performing data preprocessing on the translation pair to determine whether the target sentence in the translation pair has a predefined translation error, wherein the predefined translation error includes one of the following: the target sentence in the translation pair is empty; Or the target sentence is completely equal to the source sentence.

3. The method according to claim 1, characterized in that Obtaining the semantic evaluation score of the translation pair further comprises: Using a trained sentence embedding network in combination with a BLEU evaluation index to perform a semantic translation quality assessment on the translation pair to obtain a semantic translation quality assessment score, wherein the trained sentence embedding network is capable of evaluating the semantic similarity between the two sentences based on feature vectors of the two sentences; Using a term translation library to perform term translation quality assessment on the translation pair to obtain a term translation quality assessment score, wherein the term translation quality assessment score indicates whether the term in the source sentence is correctly translated; and The semantic translation quality assessment score and the terminology translation quality assessment score are combined to obtain the semantic assessment score for the translation pair.

4. The method according to claim 3, characterized in that Obtaining the semantic translation quality assessment score further includes: obtaining, via a translation application program interface, one or more reference translations for the source sentence in the translation pair; Inputting the target sentence and the one or more reference translations into the trained sentence embedding network to obtain a sentence embedding network evaluation score; Calculating BLEU evaluation index values ​​of the target sentence and the one or more reference translations; and The sentence embedding network evaluation score and the BLEU evaluation index value are combined to obtain the semantic translation quality evaluation score.

5. The method according to claim 1, characterized in that Obtaining the multi-factor evaluation score of the translation pair further comprises: Extracting the numeric phrases in the source sentence and the target sentence to obtain the extracted source sentence numeric phrase and the extracted target sentence numeric phrase, wherein the numeric phrase is composed of the number itself and a number of characters before and / or after the number; Identify the numeral phrase category of the numeral phrase of the source sentence; converting the format of the numbers in the source sentence number phrase based on the identified number phrase category, the source language and the target language; Matching the converted numbers with the numbers in the target sentence number phrase; A numerical translation error factor evaluation score is calculated based on the matching result of the matching.

6. The method according to claim 5, characterized in that Obtaining the multi-factor evaluation score of the translation pair further comprises: identifying unit symbol errors in the target sentence to calculate a unit symbol translation error factor assessment score; identifying spelling errors in the target sentence to calculate a spelling error factor evaluation score; and The multi-factor evaluation score is calculated based on the calculated number translation error factor evaluation score, unit symbol translation error factor evaluation score, and spelling error factor evaluation score.

7. The method according to claim 5, characterized in that Identifying the digit phrase category of the digit phrase in the source sentence further includes: Obtaining a source data digital phrase feature vector of the source sentence digital phrase through a trained digital phrase embedding network, wherein the trained digital phrase embedding network is capable of evaluating the similarity between two digital phrases based on their feature vectors; and The digital phrase feature vector of the source data is matched with the digital phrase feature vector in the feature library, so that the digital phrase category corresponding to the most similar digital phrase feature vector in the feature library is used as the digital phrase category of the digital phrase of the source data, wherein the feature library records the digital phrase feature vectors of one or more digital phrases and the corresponding digital phrase categories.

8. A multi-factor and semantic-based automatic translation evaluation system, comprising: A trained sentence embedding network capable of evaluating semantic similarity between two sentences based on feature vectors of the two sentences; a trained number phrase embedding network capable of evaluating the similarity between the two number phrases based on their feature vectors; A semantic evaluation module, wherein the semantic evaluation module is configured to evaluate the translation pair at a semantic level to obtain a semantic evaluation score of the translation pair; a multi-factor evaluation module configured to evaluate the translation pair at multiple factor levels to obtain a multi-factor evaluation score for the translation pair, wherein the multiple factors include one or more of the following: a number translation error factor, a unit symbol translation error factor, and a spelling error factor; as well as A comprehensive evaluation module is configured to combine the semantic evaluation score of the translation pair and the multi-factor evaluation score of the translation pair to obtain a comprehensive evaluation score of the translation pair.

9. The system of claim 8, further comprising: A data preprocessing module is configured to perform data preprocessing on the translation pair to determine whether the target sentence in the translation pair has a predefined translation error, wherein the predefined translation error includes one of the following: the target sentence in the translation pair is empty, or the target sentence is completely equal to the source sentence.

10. The system according to claim 8, characterized in that The semantic evaluation module is further configured to: Using a trained sentence embedding network in combination with a BLEU evaluation index to perform a semantic translation quality assessment on the translation pair to obtain a semantic translation quality assessment score, wherein the trained sentence embedding network is capable of evaluating the semantic similarity between the two sentences based on feature vectors of the two sentences; Using a term translation library to perform term translation quality assessment on the translation pair to obtain a term translation quality assessment score, wherein the term translation quality assessment score indicates whether the term in the source sentence is correctly translated; as well as The semantic translation quality assessment score and the terminology translation quality assessment score are combined to obtain the semantic assessment score for the translation pair.

11. The system according to claim 10, characterized in that Obtaining the semantic translation quality assessment score further includes: obtaining, via a translation application program interface, one or more reference translations for the source sentence in the translation pair; Inputting the target sentence and the one or more reference translations into the trained sentence embedding network to obtain a sentence embedding network evaluation score; Calculating BLEU evaluation index values ​​of the target sentence and the one or more reference translations; and The sentence embedding network evaluation score and the BLEU evaluation index value are combined to obtain the semantic translation quality evaluation score.

12. The system of claim 8, wherein: The multi-factor assessment module is further configured to: Extracting the numeric phrases in the source sentence and the target sentence to obtain the extracted source sentence numeric phrase and the extracted target sentence numeric phrase, wherein the numeric phrase is composed of the number itself and a number of characters before and / or after the number; Identify the numeral phrase category of the numeral phrase of the source sentence; converting the format of the numbers in the source sentence number phrase based on the identified number phrase category, the source language and the target language; Matching the converted numbers with the numbers in the target sentence number phrase; A numerical translation error factor evaluation score is calculated based on the matching result of the matching.

13. The system of claim 12, wherein: The multi-factor assessment module is further configured to: identifying unit symbol errors in the target sentence to calculate a unit symbol translation error factor assessment score; identifying spelling errors in the target sentence to calculate a spelling error factor assessment score; as well as The multi-factor evaluation score is calculated based on the calculated number translation error factor evaluation score, unit symbol translation error factor evaluation score, and spelling error factor evaluation score.

14. An automatic translation evaluation system based on multiple factors and semantics, comprising: An offline training subsystem configured to train a sentence embedding network at a semantic level using annotated sentence pairs in different languages ​​and to train a digital phrase embedding network using annotated digital phrase pairs in different languages, wherein the annotated sentence pairs include annotated sentence pairs with the same semantics and annotated sentence pairs with different semantics, and the annotated digital phrase pairs include annotated digital phrase pairs with the same category and annotated digital phrase pairs with different categories; as well as An online evaluation subsystem, wherein the online evaluation subsystem comprises a semantic evaluation module, a multi-factor evaluation module and a comprehensive evaluation module as described in any one of claims 8 to 13.

15. The system of claim 14, wherein: The trained sentence embedding network can evaluate the semantic similarity between the two sentences based on the feature vectors of the two sentences, and the trained number phrase embedding network can evaluate the similarity between the two number phrases based on the feature vectors of the two number phrases.

Images