Translation semantic quality evaluation method and system

Abstract

The invention provides a translation semantic quality evaluation method and system. The method mainly relates to two stages, namely an offline training stage and an online evaluation stage. In the offline training stage, annotation data is mainly utilized to train a sentence embedding network, and in the online evaluation stage, the trained sentence embedding network is mainly utilized to extract semantic-level sentence vectors, and sentence semantic information is evaluated in combination with a BLEU value.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and more specifically, to a translation semantic quality assessment method and system. Background Art

[0002] For example, in the context of global vehicle manufacturing, professional documents such as operating manuals, instructions, internal specifications, design documents, engineering drawings, etc. need to be accurately translated between different languages ​​to avoid possible technical deviations. In order to translate such professional documents quickly and efficiently, a large number of multilingual translation text pairs are generated. These translation texts have great commercial value for subsequent reuse. However, since the translation pairs currently generated come from a variety of factors and most of them have uneven quality, the evaluation of translation quality has received widespread attention in the field of natural language processing. Manual quality evaluation of translated texts will be a huge workload, and multilingual translation involves professional usage of different languages, so professional personnel are needed to complete the evaluation work.

[0003] Traditional translation quality assessment mostly uses BLEU values ​​and requires a large number of human professional translators. In essence, a single BLEU quality assessment can only assess the approximate text similarity between the translation result and the reference translation, but ignores the semantic and grammatical features, so synonyms and the length of the translation will affect the effect of the translation assessment.

[0004] Therefore, a solution is needed to improve the shortcoming that the BLEU value only evaluates the text similarity of words. Summary of the invention

[0005] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0006] According to one aspect of the present invention, a semantic evaluation method is provided, comprising: receiving a translation pair, the translation pair comprising a source sentence and a target sentence; using a trained sentence embedding network in combination with a BLEU evaluation index to perform a semantic translation quality evaluation on the translation pair to obtain a semantic translation quality evaluation score; using a term translation library to perform a term translation quality evaluation on the translation pair to obtain a term translation quality evaluation score, the term translation quality evaluation score indicating whether a term in the source sentence is correctly translated; combining the semantic translation quality evaluation score and the term translation quality evaluation score to obtain a semantic evaluation score for the translation pair, the semantic evaluation score evaluating the translation quality of the target sentence in the translation pair.

[0007] According to a further embodiment of the present invention, the method further comprises: performing data preprocessing on the translation pair to determine whether the target sentence in the translation pair has a predefined translation error.

[0008] According to a further embodiment of the present invention, the predefined translation error includes one of the following: the target sentence in the translation pair is empty; or the target sentence is completely equal to the source sentence.

[0009] According to a further embodiment of the present invention, the trained sentence embedding network is capable of evaluating the semantic similarity between the two sentences based on the feature vectors of the two sentences.

[0010] According to a further embodiment of the present invention, obtaining the semantic translation quality assessment score further comprises: obtaining one or more reference translations for the source sentence in the translation pair via a translation application program interface.

[0011] According to a further embodiment of the present invention, obtaining the semantic translation quality assessment score further includes: inputting the target sentence and the one or more reference translations into the trained sentence embedding network to obtain a sentence embedding network evaluation score; calculating the BLEU evaluation index values ​​of the target sentence and the one or more reference translations; and combining the sentence embedding network evaluation score and the BLEU evaluation index value to obtain the semantic translation quality assessment score.

[0012] According to a further embodiment of the present invention, the term translation quality assessment score identifies whether the term in the source sentence is translated into a preferred translation term in the target sentence among one or more translation terms corresponding to the term in the term translation library.

[0013] According to another aspect of the present invention, a semantic evaluation system is provided, comprising: a trained sentence embedding network and a semantic evaluation module; the trained sentence embedding network can evaluate the semantic similarity between the two sentences based on the feature vectors of the two sentences; the semantic evaluation module is configured to: use the trained sentence embedding network in combination with the BLEU evaluation index to perform semantic translation quality evaluation on the translation pair to obtain a semantic translation quality evaluation score; use a term translation library to perform term translation quality evaluation on the translation pair to obtain a term translation quality evaluation score, the term translation quality evaluation score indicating whether the term in the source sentence is correctly translated; combine the semantic translation quality evaluation score and the term translation quality evaluation score to obtain a semantic evaluation score for the translation pair, the semantic evaluation score evaluating the translation quality of the target sentence in the translation pair.

[0014] According to a further embodiment of the present invention, the system further comprises: a data preprocessing module, wherein the data preprocessing module is configured to perform data preprocessing on the translation pair to determine whether the target sentence in the translation pair has a predefined translation error.

[0015] According to a further embodiment of the present invention, the predefined translation error includes one of the following: the target sentence in the translation pair is empty, or the target sentence is completely equal to the source sentence.

[0016] According to a further embodiment of the present invention, obtaining the semantic translation quality assessment score further comprises: obtaining one or more reference translations for the source sentence in the translation pair via a translation application program interface.

[0017] According to a further embodiment of the present invention, obtaining the semantic translation quality assessment score further includes: inputting the target sentence and the one or more reference translations into the trained sentence embedding network to obtain a sentence embedding network evaluation score; calculating the BLEU evaluation index values ​​of the target sentence and the one or more reference translations; and combining the sentence embedding network evaluation score and the BLEU evaluation index value to obtain the semantic translation quality assessment score.

[0018] According to a further embodiment of the present invention, the term translation quality assessment score identifies whether the term in the source sentence is translated into a preferred translation term in the target sentence among one or more translation terms corresponding to the term in the term translation library.

[0019] According to another aspect of the present invention, there is provided a semantic evaluation system, comprising: an offline training subsystem, the offline training subsystem being configured to train a semantic-level sentence embedding network using annotated sentence pairs in different languages, wherein the annotated sentence pairs include annotated sentence pairs with the same semantics and annotated sentence pairs with different semantics; and an online evaluation subsystem, the online evaluation subsystem comprising the semantic evaluation module as described above.

[0020] According to a further embodiment of the present invention, the trained sentence embedding network is capable of evaluating the semantic similarity between the two sentences based on the feature vectors of the two sentences.

[0021] These and other features and advantages will become apparent by reading the following detailed description and by reference to the associated drawings.It is to be understood that the foregoing general description and the following detailed description are illustrative only and are not restrictive of the aspects of what is claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to understand the manner in which the above features of the present invention are used in detail, the above briefly summarized contents can be described in more detail with reference to various embodiments, some of which are shown in the accompanying drawings. However, it should be noted that the accompanying drawings only show some typical aspects of the present invention and should not be considered to limit its scope, because the description may allow for other equally effective aspects.

[0023] Figure 1 A schematic diagram of a system 100 for semantic evaluation according to an embodiment of the present invention is shown;

[0024] Figure 2 An embodiment of the present invention is shown. Figure 1 The semantic evaluation module 120 in the data stream for performing semantic evaluation;

[0025] Figure 3 A flowchart of a semantic evaluation method 300 according to an embodiment of the present invention is shown;

[0026] Figure 4 A block diagram of an exemplary computing device according to one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] The present invention will be described in detail below in conjunction with the accompanying drawings, and the features of the present invention will be further revealed in the following specific description.

[0028] The following detailed description refers to the accompanying drawings showing exemplary embodiments of the present invention. However, the scope of the present invention is not limited to these embodiments, but is defined by the appended claims. Therefore, embodiments other than those shown in the drawings, such as modified versions of the illustrated embodiments, are still encompassed by the present invention.

[0029] References in this specification to "one embodiment," "an embodiment," "an example embodiment," etc., mean that the embodiment may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it should be understood that the particular feature, structure, or characteristic can be implemented in conjunction with other embodiments within the knowledge of a person skilled in the relevant art, whether or not explicitly described.

[0030] Unless otherwise stated, the term "A or B" used throughout this specification refers to "A and B" and "A or B" but does not mean that A and B are exclusive.

[0031] Related terms:

[0032] BLEU (Bilingual Evaluation Understudy): It is an indicator for machine translation evaluation proposed by IBM in 2002. It can be used to judge the accuracy and advantages of machine translation. BLEU measures the effect of translation by analyzing the number of times N-gram fragments co-occur in the source language text and the target language text. In other words, it checks whether the vocabulary in the source language text appears accurately in the target language text. If the keywords of the source language text appear accurately in the target language text, the BLEU value will be improved.

[0033] Source sentence and target sentence: In the context of this application, a source sentence is a sentence written in a source language to be translated. A target sentence is a sentence corresponding to the source sentence and translated into a target language. A source sentence and a corresponding target sentence constitute a translation pair.

[0034] The present invention provides a translation semantic quality assessment scheme. The scheme mainly involves two stages, namely, an offline training stage and an online assessment stage. The offline training stage mainly uses labeled data to train a sentence embedding network, and the online assessment stage mainly uses the trained sentence embedding network to extract sentence vectors at the semantic level and combines the BLEU value to evaluate the sentence semantic information.

[0035] Figure 1 FIG. 1 is a schematic diagram of a system 100 for semantic evaluation according to an embodiment of the present invention. The system 100 may include an offline training subsystem 105 and an online evaluation subsystem 110 .

[0036] According to one embodiment of the present invention, the offline training subsystem 105 is mainly configured to train a sentence embedding network at a semantic level, so that there is a smaller distance between the vector encodings of semantically similar sentence pairs, and a larger distance between the vector encodings of semantically different sentence pairs, thereby allowing the trained sentence embedding network to have the ability to evaluate the semantic similarity of multiple translations.

[0037] For example, referring to the offline training subsystem 105, the training data of the sentence embedding network is mainly divided into two categories, one is the annotated sentence pairs with the same semantics, and the other is the annotated sentence pairs with different semantics. Specifically, sentence 1 and sentence 2 can be regarded as a sentence pair. In the case of sentence pairs with the same semantics, sentence 1 can be, for example, "What's your age?", and sentence 2 can be, for example, "How old are you?", that is, sentence 1 and sentence 2 represent the same semantics (how old are you?). In the case of sentence pairs with different semantics, sentence 1 can be, for example, "Have you eaten today?", and sentence 2 can be, for example, "How is the weather today?", that is, sentence 1 and sentence 2 have different semantics. In practice, the training data can contain sentence pairs in multiple (e.g., 15 or more) languages, so that the sentence embedding network has the ability to evaluate multiple languages.

[0038] The loss function of the sentence embedding network adopts the CosineEmbeddingLoss (Cosine embedding loss) function, and the main formula is as follows:

[0039]

[0040] In the above formula, N represents the number of samples, a data pair is represented as (v1, v2, y), v1 and v2 represent the embedding vectors of sentence 1 and sentence 2 respectively, and y represents the label. If the two sentences have the same semantics, they are marked as 1, and if the two sentences have different semantics, they are marked as 0.

[0041] Thus, after training, the trained sentence embedding network can evaluate the semantic similarity between the two sentences based on their feature vectors.

[0042] According to one embodiment of the present invention, the online evaluation subsystem 110 is configured to perform data preprocessing on a translation pair including a source sentence and a target sentence (e.g., a human translation text of the source sentence), and perform semantic evaluation on the data preprocessed translation pair, thereby performing translation semantic quality evaluation on the target sentence.

[0043] Specifically, the online evaluation subsystem 110 includes a data preprocessing module 115 and a semantic evaluation module 120. It is fully understood by those skilled in the art that the division of the above modules is only explained for the purpose of clarity. The functions of one or more of the above modules can be combined into a single module or split into more modules / submodules. Moreover, one or more of the above modules can be implemented in software, hardware or a combination thereof. In addition, the data flow between the modules can be implemented in a manner known in the art, which is not within the scope of the present invention.

[0044] According to an embodiment of the present invention, the data preprocessing module 115 is configured to detect whether the target sentence has predefined translation errors, so as to reduce excessive subsequent meaningless calculations.

[0045] For example, the data preprocessing module 115 may be configured to: when the target sentence of the translation pair is empty, it may be determined that there is no translation for the source sentence in the translation pair, and thus the translation quality problem may be directly output.

[0046] For another example, the data preprocessing module 115 may be configured to: when the source sentence and the target sentence in the translation pair are equal, it may be determined that the source sentence has not been translated, and thus a translation quality problem may be directly output.

[0047] Thus, if the data pre-processing module 115 outputs a translation quality problem for a translation pair, the translation pair does not need to be transmitted to the semantic evaluation module 120 .

[0048] According to one embodiment of the present invention, the semantic evaluation module 120 can be configured to perform three processes: (1) using the trained sentence embedding network and combining the BLEU evaluation index to perform semantic translation quality evaluation on the translation pair; (2) using the terminology translation library to perform terminology translation quality evaluation on the translation pair; (3) combining the semantic translation quality evaluation and terminology translation quality evaluation to obtain a semantic evaluation of the translation pair. Thus, the semantic evaluation module 120 not only considers the semantics expressed in the translated text, but also evaluates the translation quality of some domain-specific terms.

[0049] Specifically, refer to Figure 2 , which shows the data flow of the semantic evaluation module 120 according to one embodiment of the present invention when executing the above three processes to perform semantic evaluation.

[0050] According to one embodiment of the present invention, in the process of semantic translation quality assessment, since obtaining professional human reference translations is extremely time-consuming and labor-intensive, existing commercial translation application program interface APIs (e.g., Baidu Translate, Bing Translate, etc.) can translate the meaning of texts more accurately, so this solution uses the existing translation API interface to obtain reference translations, which are the translation results obtained by the commercial translation API interface for the source sentence in the translation pair. The reference translation can be flexibly configured, and multiple reference translations can also be used to obtain multiple reference translations to achieve better evaluation results.

[0051]

[0052] The above describes the formula for calculating the semantic translation quality assessment score. It can be seen from the above formula that the semantic translation quality assessment process is mainly divided into two parts. The first part is the sentence embedding network semantic assessment part, and the second part is the BLEU evaluation index. In the first part, the target sentence and one or more reference translations are input into the trained sentence embedding network for semantic evaluation. In the second part, the BLEU evaluation index value between the target sentence and the one or more reference translations is calculated for BLEU evaluation.

[0053] Among them, δ and ε are the weights assigned to the two parts respectively. r represents the number of reference translations, w i represents the weight of each reference translation, T represents the sentence embedding vector of the target sentence, R i Represents the sentence embedding vector of the reference translation. BP represents the length penalty factor in the BLEU evaluation index, N n Represents the number of n-gram language models used, w n is the corresponding weight, p n is the matching degree between the target sentence and the reference translation. That is, the calculation of the semantic translation quality evaluation of the present invention takes into account both the evaluation of the semantics of the reference translation and the target sentence in the trained sentence embedding network and the BLEU evaluation index of the reference translation and the target sentence. In practice, different weight values ​​δ and ε can be set according to different considerations.

[0054] It can be seen that the BLEU evaluation index only considers the matching degree at the word level, and does not take into account the accuracy of language expression. Therefore, the present invention introduces a measurement index of semantic similarity in the process of semantic translation quality evaluation, and makes the evaluation index more complete by measuring the semantic level scores of the target sentence and the reference translation.

[0055] According to one embodiment of the present invention, in the terminology translation quality assessment stage, the present invention mainly assesses whether some professional terms in the source sentence are translated into the correct usage of the target language in the target sentence. The correct translation of terms comes from the long-term accumulation of translation libraries to form a professional terminology translation library. In the terminology translation quality assessment, whether a professional term is translated into the preferred translation term in the terminology translation library is used as an evaluation method. Therefore, firstly, all existing terms in the source sentence are searched based on the terminology translation library, and then it is determined whether there are corresponding preferred translation terms in the target sentence. The terminology translation quality assessment score is as follows:

[0056]

[0057] Where E t_ The number of terms with incorrect translations, N T_ The number of words that represent the target sentence.

[0058] According to one embodiment of the present invention, the semantic evaluation module 120 is configured to integrate the semantic translation quality evaluation score and the terminology translation quality evaluation score to obtain a final semantic evaluation score, as follows:

[0059] S s =μS 语义 +ρS 术语

[0060] Among them, μ+ρ=1, which are the weights corresponding to semantic translation quality assessment and terminology translation quality assessment respectively. In practice, the weights can be adjusted appropriately according to the proportion of different parts to achieve better evaluation results.

[0061] Figure 3 FIG. 3 is a flowchart of a semantic evaluation method 300 according to an embodiment of the present invention. The method 300 may be executed by the online evaluation subsystem 110 .

[0062] At 305 , a translation pair is received, the translation pair comprising a source sentence and a target sentence, wherein the source sentence is a sentence to be translated written in a source language, and the target sentence is a sentence corresponding to the source sentence and translated into a target language.

[0063] At 310, data preprocessing is performed on the translation pair to determine whether the target sentence in the translation pair has a predefined translation error. If yes, the process proceeds to step 315, where a translation error reminder is directly output. If no, the process proceeds to step 320.

[0064] According to one embodiment of the present invention, step 310 may further include comparing the source language and the target language in the translation pair. According to one embodiment of the present invention, the predefined translation errors may include any of the following: the target sentence in the translation pair is empty; the target sentence is completely equal to the source sentence.

[0065] According to one embodiment of the present invention, step 310 may also be omitted, that is, the process may proceed directly from step 305 to step 320 .

[0066] At 320 , the trained sentence embedding network is used in combination with the BLEU evaluation index to perform semantic translation quality evaluation on the translation pair to obtain a semantic translation quality evaluation score.

[0067] According to one embodiment of the present invention, the trained sentence embedding network is capable of evaluating the semantic similarity between the two sentences based on the feature vectors of the two sentences.

[0068] According to an embodiment of the present invention, step 320 further includes obtaining one or more reference translations for the source sentence in the translation pair via a translation API.

[0069] According to one embodiment of the present invention, step 320 further includes inputting the target sentence and the one or more reference translations into a trained sentence embedding network to obtain a sentence embedding network evaluation score, calculating a BLEU evaluation index value of the target sentence and the one or more reference translations, and combining the sentence embedding network evaluation score and the BLEU evaluation index value to obtain a semantic translation quality evaluation score.

[0070] At 325 , a term translation database is used to perform a term translation quality assessment on the translation pair to obtain a term translation quality assessment score, which indicates whether the term in the source sentence is correctly translated.

[0071] According to one embodiment of the present invention, the term translation quality assessment score identifies whether the term in the source sentence in the translation pair is translated into a preferred translation term in one or more translation terms corresponding to the term in the term translation library in the target sentence in the translation pair.

[0072] At 330, the semantic translation quality assessment score and the terminology translation quality assessment score are combined to obtain a semantic assessment score for the translation pair. In summary, the semantic assessment score evaluates the translation quality of the target sentence in the translation pair.

[0073] According to an embodiment of the present invention, the semantic translation quality assessment score and the terminology translation quality assessment score may be assigned different weights.

[0074] In summary, the translation semantic quality assessment method proposed in the present invention introduces sentence semantic vectors and term translation quality, improving the shortcoming of the traditional BLEU index that only evaluates word text similarity. In addition, the present invention introduces a sentence embedding network and adds a multilingual language corpus for training, so that the sentence embedding network has the ability to adapt to multiple languages ​​that may appear in the translation library.

[0075] Figure 4 A block diagram of an exemplary computing device according to one embodiment of the present invention is shown, which is one example of a hardware device applicable to various aspects of the present invention.

[0076] refer to Figure 4, a computing device 400 will now be described, which is an example of a hardware device applicable to various aspects of the present invention. The computing device 400 can be any machine that can be configured to perform processing and / or computing, and can be but is not limited to a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal digital assistant, a smart phone, a vehicle-mounted computer, or any combination thereof. The various methods / apparatus / server / client devices described above can be implemented in whole or in part by the computing device 400 or a similar device or system.

[0077] The computing device 400 may include components that may be connected or communicated via one or more interfaces and a bus 402. For example, the computing device 400 may include a bus 402, one or more processors 404, one or more input devices 406, and one or more output devices 408. The one or more processors 404 may be any type of processor and may include, but are not limited to, one or more general purpose processors and / or one or more special purpose processors (e.g., a dedicated processing chip). The input device 406 may be any type of device capable of inputting information to the computing device and may include, but are not limited to, a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. The output device 408 may be any type of device capable of presenting information and may include, but are not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The computing device 400 may also include or be connected to a non-transient storage device 410, which may be any storage device that is non-transient and capable of data storage, and may include, but is not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a floppy disk, a hard disk, a tape or any other magnetic medium, an optical disk or any other optical medium, a ROM (read-only memory), a RAM (random access memory), a cache memory, and / or any memory chip or cassette, and / or any other medium from which a computer can read data, instructions, and / or code. The non-transient storage device 410 may be detachable from the interface. The non-transient storage device 410 may have data / instructions / code for implementing the above methods and steps. The computing device 400 may also include a communication device 412. The communication device 412 can be any type of device or system that can communicate with internal devices and / or with a network and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device and / or a chipset, such as a Bluetooth device, an IEEE 1302.11 device, a WiFi device, a WiMax device, a cellular communication device and / or the like.

[0078] When the computing device 400 is used as an in-vehicle device, it can also be connected to external devices (e.g., a GPS receiver, sensors for sensing different environmental data (such as an accelerometer, a wheel speed sensor, a gyroscope, etc.)). In this way, the computing device 400 can, for example, receive positioning data and sensor data indicating the vehicle's form condition. When the computing device 400 is used as an in-vehicle device, it can also be connected to other devices (e.g., an engine system, wipers, an anti-lock brake system, a vehicle climate setting system, etc.) for controlling the travel and operation of the vehicle (e.g., vehicle climate settings).

[0079] In addition, the non-transitory storage device 410 may have map information and software components so that the processor 404 can implement route guidance processing. In addition, the output device 406 may include a display for displaying a map, displaying a positioning mark of the vehicle, and displaying an image indicating the driving status of the vehicle. The output device 406 may also include a speaker or a headphone interface for audio guidance.

[0080] The bus 402 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. In particular, for in-vehicle devices, the bus 402 may also include a Controller Area Network (CAN) bus or other structures designed for automotive applications.

[0081] The computing device 400 may also include a working memory 414 , which may be any type of working memory capable of storing instructions and / or data that facilitate the operation of the processor 404 and may include, but is not limited to, a random access memory and / or a read-only storage device.

[0082] Software components may be located in the working memory 414, including but not limited to an operating system 416, one or more application programs 418, drivers and / or other data and codes. Instructions for implementing the above methods and steps may be included in the one or more application programs 418, and the modules / units / components of the aforementioned various devices / servers / client devices may be implemented by the processor 404 reading and executing the instructions of the one or more application programs 418.

[0083] It should also be recognized that changes may be made according to specific needs. For example, custom hardware may also be used, and / or specific components may be implemented in hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. In addition, connections to other computing devices, such as network input / output devices, etc. may be employed. For example, programming hardware (e.g., programmable logic circuits including field programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs)) with assembly language or hardware programming languages ​​(e.g., VERILOG, VHDL, C++) may be used to implement part or all of the disclosed methods and devices using logic and algorithms according to the present invention.

[0084] Although various aspects of the present invention have been described so far with reference to the accompanying drawings, the above-described methods, systems and devices are merely examples, and the scope of the present invention is not limited to these aspects, but is limited only by the appended claims and their equivalents. Various components may be omitted or replaced by equivalent components. In addition, the steps may be implemented in an order different from the order described in the present invention. In addition, various components may be combined in various ways. It is also important that, as technology develops, many of the components described may be replaced by equivalent components that appear later.

Claims

1. A semantic evaluation method, comprising: receiving a translation pair, the translation pair comprising a source sentence and a target sentence; Using the trained sentence embedding network and the BLEU evaluation index to perform semantic translation quality assessment on the translation pair to obtain a semantic translation quality assessment score; Using a term translation library to perform term translation quality assessment on the translation pair to obtain a term translation quality assessment score, wherein the term translation quality assessment score indicates whether the term in the source sentence is correctly translated; as well as The semantic translation quality evaluation score and the terminology translation quality evaluation score are combined to obtain a semantic evaluation score for the translation pair, wherein the semantic evaluation score evaluates the translation quality of the target sentence in the translation pair.

2. The method of claim 1, further comprising: Data preprocessing is performed on the translation pair to determine whether the target sentence in the translation pair has a predefined translation error.

3. The method according to claim 2, characterized in that The predefined translation errors include one of the following: The target sentence in the translation pair is empty; or The target sentence is completely equal to the source sentence.

4. The method according to claim 1, characterized in that The trained sentence embedding network is capable of evaluating the semantic similarity between the two sentences based on their feature vectors.

5. The method according to claim 4, characterized in that Obtaining the semantic translation quality assessment score further includes: One or more reference translations for the source sentence in the translation pair are obtained via a translation application program interface.

6. The method according to claim 5, characterized in that Obtaining the semantic translation quality assessment score further includes: Inputting the target sentence and the one or more reference translations into the trained sentence embedding network to obtain a sentence embedding network evaluation score; Calculating BLEU evaluation index values ​​of the target sentence and the one or more reference translations; and The sentence embedding network evaluation score and the BLEU evaluation index value are combined to obtain the semantic translation quality evaluation score.

7. The method according to claim 1, characterized in that The term translation quality assessment score identifies whether a term in the source sentence is translated in the target sentence into a preferred translation term among one or more translation terms corresponding to the term in the term translation library.

8. A semantic evaluation system, comprising: A trained sentence embedding network capable of evaluating semantic similarity between two sentences based on feature vectors of the two sentences; as well as A semantic evaluation module, wherein the semantic evaluation module is configured to: Using the trained sentence embedding network and combining it with the BLEU evaluation index to perform semantic translation quality evaluation on the translation pair to obtain a semantic translation quality evaluation score; Using a term translation library to perform term translation quality assessment on the translation pair to obtain a term translation quality assessment score, wherein the term translation quality assessment score indicates whether the term in the source sentence is correctly translated; The semantic translation quality evaluation score and the terminology translation quality evaluation score are combined to obtain a semantic evaluation score for the translation pair, wherein the semantic evaluation score evaluates the translation quality of the target sentence in the translation pair.

9. The system of claim 8, further comprising: A data preprocessing module is configured to perform data preprocessing on the translation pair to determine whether the target sentence in the translation pair has a predefined translation error.

10. The system according to claim 8, characterized in that The predefined translation error includes one of the following: the target sentence in the translation pair is empty, or the target sentence is completely equal to the source sentence.

11. The system according to claim 8, characterized in that Obtaining the semantic translation quality assessment score further includes: One or more reference translations for the source sentence in the translation pair are obtained via a translation application program interface.

12. The system according to claim 11, characterized in that Obtaining the semantic translation quality assessment score further includes: Inputting the target sentence and the one or more reference translations into the trained sentence embedding network to obtain a sentence embedding network evaluation score; Calculating BLEU evaluation index values ​​of the target sentence and the one or more reference translations; and The sentence embedding network evaluation score and the BLEU evaluation index value are combined to obtain the semantic translation quality evaluation score.

13. The system of claim 8, wherein: The term translation quality assessment score identifies whether a term in the source sentence is translated in the target sentence into a preferred translation term among one or more translation terms corresponding to the term in the term translation library.

14. A semantic evaluation system, comprising: An offline training subsystem configured to train a semantic-level sentence embedding network using annotated sentence pairs in different languages, wherein the annotated sentence pairs include annotated sentence pairs with the same semantics and annotated sentence pairs with different semantics; as well as An online evaluation subsystem, wherein the online evaluation subsystem comprises a semantic evaluation module as claimed in any one of claims 8 to 13.

15. The system of claim 14, wherein: The trained sentence embedding network is able to evaluate the semantic similarity between two sentences based on their feature vectors.

Images