Translation quality assessment methods, related devices and computer program products

Abstract

This disclosure provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for determining translation quality, relating to artificial intelligence technologies such as natural language processing, machine translation, and deep learning. One specific implementation of the method includes: acquiring a first corpus and a second corpus with different languages ​​but consistent semantics; replacing key information in the second corpus with interrogative words to construct a third corpus with interrogative sentence structures; using a translation model to generate a fourth corpus that semantically corresponds to the first corpus and is in the same language as the second corpus; determining first result information in the fourth corpus corresponding to the interrogative points in the third corpus; and generating first evaluation information for evaluating the translation quality of the translation model based on the similarity between the key information and the first result information. This implementation provides a method for determining translation quality, enabling the evaluation of translation model translation quality based on semantic-level understanding.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to the fields of artificial intelligence technology such as natural language processing, machine translation, and deep learning, and particularly to methods, apparatus, electronic devices, computer-readable storage media, and computer program products for determining translation quality. Background Technology

[0002] Simultaneous interpreting and real-time translation systems have high requirements for translation quality. Therefore, evaluating these interpreting and translation systems has always been a challenge. The evaluation system needs to take into account issues such as contextual consistency, coherence, and fidelity to the original text to produce a reasonable evaluation of each simultaneous interpreting result.

[0003] In existing technologies, most studies use the classic text translation evaluation method—Bilingual Evaluation Understudy (BLEU)—to evaluate translations. It calculates the similarity between the translation result and the manually annotated standard translation result, returns a value between 0 and 1, and uses this value to evaluate the translation result. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for determining translation quality.

[0005] In a first aspect, embodiments of this disclosure propose a method for determining translation quality, comprising: acquiring a first corpus and a second corpus of different languages ​​but consistent semantics; replacing key information in the second corpus with interrogative words to construct a third corpus of interrogative sentences; processing the first corpus using a translation model to generate a fourth corpus corresponding to the first corpus, wherein the translation model is used to transform the corpus between different languages, and the fourth corpus is in the same language as the second corpus; determining first result information in the fourth corpus corresponding to the interrogative points in the third corpus; and generating first evaluation information for evaluating the translation quality of the translation model based on the similarity between the key information and the first result information.

[0006] Secondly, embodiments of this disclosure propose a translation quality determination apparatus, comprising: a corpus acquisition unit configured to acquire a first corpus and a second corpus of different languages ​​but consistent semantics; an interrogative corpus construction unit configured to replace key information in the second corpus with interrogative words to construct a third corpus of interrogative sentences; a corpus translation unit configured to process the first corpus using a translation model to generate a fourth corpus corresponding to the first corpus, wherein the translation model is used to transform the corpus between different languages, and the fourth corpus is in the same language as the second corpus; a first result information generation unit configured to determine first result information in the fourth corpus corresponding to the interrogative points in the third corpus; and a first evaluation information generation unit configured to generate first evaluation information for evaluating the translation quality of the translation model based on the similarity between the key information and the first result information.

[0007] Thirdly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the translation quality determination method as described in any implementation of the first aspect.

[0008] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement the translation quality determination method as described in any implementation of the first aspect.

[0009] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, can implement the translation quality determination method as described in any implementation of the first aspect.

[0010] The translation quality determination method, apparatus, electronic device, computer-readable storage medium, and computer program product provided in this disclosure acquire a first corpus and a second corpus with different languages ​​but consistent semantics. After replacing key information in the second corpus with interrogative words to construct a third corpus with interrogative sentences, a translation model is used to generate a fourth corpus that corresponds semantically to the first corpus and is in the same language as the second corpus. First result information corresponding to the interrogative points in the third corpus is determined in the fourth corpus. Based on the similarity between the key information and the first result information, first evaluation information is generated to evaluate the translation quality of the translation model.

[0011] This disclosure constructs a third corpus of interrogative sentences by replacing key information with interrogative words in a second corpus that corresponds semantically to the first corpus. After processing the first corpus using a translation model to obtain a fourth corpus of the same language type as the second corpus, the translation quality of the translation model is evaluated based on the similarity between the aforementioned key information and the first result information in the fourth corpus that corresponds to the interrogative points in the third corpus. This achieves semantic-level translation quality evaluation of the translation model.

[0012] The descriptions in this section are not intended to identify key or essential features of the embodiments of this disclosure, nor are they intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0014] Figure 1 This is an exemplary system architecture to which this disclosure can be applied;

[0015] Figure 2 A flowchart of a translation quality determination method provided in this disclosure embodiment;

[0016] Figure 3 A flowchart of another translation quality determination method provided in this disclosure embodiment;

[0017] Figure 4 A flowchart illustrating a method for determining translation quality in an application scenario, as provided in an embodiment of this disclosure;

[0018] Figure 5 A structural block diagram of a translation quality determination device provided in this disclosure embodiment;

[0019] Figure 6 This is a schematic diagram of the structure of an electronic device suitable for performing a translation quality determination method, provided as an embodiment of this disclosure. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding; these should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0021] Furthermore, the acquisition, storage, use, processing, transportation, provision, and disclosure of user personal information involved in the technical solutions disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0022] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the translation quality determination methods, apparatus, electronic devices, and computer-readable storage media of this disclosure can be applied.

[0023] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0024] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications for enabling information communication between the terminal devices 101, 102, and 103 and server 105 can be installed. These applications include translation quality testing applications, remote model debugging applications, and instant messaging applications.

[0025] Terminal devices 101, 102, and 103 and server 105 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices, and can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here. When server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When server 105 is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here.

[0026] Server 105 can provide various services through its built-in applications. Taking a translation quality testing application that provides translation capabilities and quality detection as an example, server 105 can achieve the following effects when running this application: First, it obtains a first corpus and a second corpus with different languages ​​but consistent semantics from terminal devices 101, 102, and 103 via network 104; then, server 105 replaces key information in the second corpus with interrogative words to construct a third corpus with interrogative sentences; next, server 105 processes the first corpus using a translation model to generate a fourth corpus corresponding to the first corpus, wherein the translation model is used to transform the corpus between different languages, and the fourth corpus is in the same language as the second corpus; further, server 105 determines the first result information in the fourth corpus corresponding to the interrogative points in the third corpus; finally, server 105 generates first evaluation information based on the similarity between the key information and the first result information to evaluate the translation quality of the translation model.

[0027] It should be noted that the first and second corpora, which are different in language but consistent in meaning, can be obtained from terminal devices 101, 102, and 103 via network 104, or they can be pre-stored locally on server 105 in various ways. Therefore, when server 105 detects that this data is already stored locally (for example, when starting to process previously retained tasks for evaluating the translation quality of the translation model), it can choose to directly obtain this data from locally. In this case, the exemplary system architecture 100 may also exclude terminal devices 101, 102, and 103 and network 104.

[0028] Since processing corpora using translation models requires significant computational resources and power, the translation quality determination methods provided in the subsequent embodiments of this disclosure are generally executed by a server 105 with strong computational power and abundant resources. Correspondingly, the translation quality determination device is also generally located within the server 105. However, it should also be noted that when terminal devices 101, 102, and 103 also possess sufficient computational power and resources, they can also perform the aforementioned calculations performed by the server 105 through translation quality testing applications installed on them, thereby outputting the same results as the server 105. Especially when multiple terminal devices with different computational capabilities exist simultaneously, but the translation quality testing application determines that the terminal device has strong computational power and abundant remaining resources, the terminal device can perform the aforementioned calculations, thereby appropriately reducing the computational burden on the server 105. Accordingly, the translation quality determination device can also be located within terminal devices 101, 102, and 103. In this case, the exemplary system architecture 100 may also exclude the server 105 and the network 104.

[0029] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0030] Please refer to Figure 2 , Figure 2 A flowchart of a translation quality determination method provided for embodiments of this disclosure is provided, wherein process 200 includes the following steps:

[0031] Step 201: Obtain the first and second corpora, which are in different languages ​​but have the same semantics.

[0032] In this embodiment, the entity executing the translation quality determination method (e.g., Figure 1 The server 105 shown obtains a first corpus and a second corpus with different languages ​​but consistent semantics. For example, when the language of the first corpus is Chinese and the language of the second corpus is English, the first corpus can be "There is an apple on the table", and the corresponding second corpus can be "There is an apple on the table". It should be understood that the content corresponding to the second corpus is usually generated by humans or by translation models, translators, etc. that meet the credibility requirements, and can be used as standard corpus for training the model.

[0033] It should be noted that the first and second corpora can be obtained directly from local storage devices by the aforementioned executing entity, or from non-local storage devices (e.g., Figure 1 The data is obtained from the terminal devices 101, 102, and 103 shown. The local storage device can be a data storage module located within the aforementioned execution entity, such as a server hard drive. In this case, the first and second corpora can be quickly read locally. The non-local storage device can also be any other electronic device configured to store data, such as some user terminals. In this case, the aforementioned execution entity can obtain the required first and second corpora by sending an acquisition command to the electronic device.

[0034] Step 202: Replace the key information in the second corpus with interrogative words to construct a third corpus with interrogative sentences.

[0035] In this embodiment, after obtaining the second corpus, determining at least one key piece of information from the second corpus, and correspondingly replacing the key piece of information with an interrogative word, the included "." (period) is replaced with "?" (question mark) to construct a third corpus with the sentence structure of an interrogative sentence. That is, in the constructed third corpus, except for the interrogative words included and replaced based on the key information in the second corpus, the rest of the content is consistent with the second corpus.

[0036] In the case where the second corpus is a set of sentences consisting of multiple sentences, key information can be determined from each sentence, and then interrogative sentences can be constructed separately to obtain a third corpus containing multiple interrogative sentences (represented as a set of interrogative sentences). Alternatively, some sentences used to construct interrogative sentences can be determined from each sentence. This disclosure does not impose any restrictions on this.

[0037] Furthermore, the content in the second corpus can be split according to sentence components, that is, the content in the second corpus can be split into subjects, predicates, objects, nouns, verbs, etc., so as to quickly construct the third corpus by pre-configuring interrogative words corresponding to different sentence components. For example, when the sentence component of the key information in the second corpus is determined to be a noun, the key information can be replaced according to the pre-configured interrogative word "What" corresponding to the noun, and a third corpus with interrogative sentences can be constructed.

[0038] Step 203: Process the first corpus using a translation model to generate a fourth corpus corresponding to the first corpus.

[0039] In this embodiment, a translation model to be determined and evaluated for translation quality is obtained, and the first corpus is processed using the translation model to generate a result that translates the first corpus into the language corresponding to the second corpus, and the result is determined as the fourth corpus.

[0040] The translation model is used to translate between at least two languages, for example, translating the first corpus into the language corresponding to the second corpus.

[0041] Step 204: Determine the first result information in the fourth corpus that corresponds to the question point in the third corpus.

[0042] In this embodiment, the fourth corpus is used as the answer or basis to find the question points in the third corpus, that is, the content referred to or asked by the question words, and the content referred to or asked by the question words is used as the first result information corresponding to the question points in the third corpus. It should be understood that the first result is usually semantically related (or the same) and in the same language as the key information.

[0043] In practice, a pre-configured question-answering model can be used, with the content in the fourth corpus as a reference for the question-answering model, the third corpus as the input for the question, and the question-answering result obtained through the question-answering model as the first result information in the fourth corpus to determine the question point corresponding to the question point in the third corpus.

[0044] Step 205: Generate first evaluation information based on the similarity between key information and first result information to evaluate the translation quality of the translation model.

[0045] In this embodiment, after obtaining the key information determined in step 202 and the first result information obtained in step 204, first evaluation information for evaluating the translation quality of the translation model is generated based on the similarity between the key information and the first result information. This allows for understanding the gap between the translation result made by the translation model and the standard result based on the similarity included in the first evaluation information. The similarity can be generated by directly comparing the literal similarity between the key information and the first result information, or by converting the key information and the first result information into corresponding feature vectors and then using the feature distance and cosine distance between the feature vectors to generate the similarity.

[0046] Furthermore, the information length that allows the above similarity to be obtained through literal similarity can be pre-configured. This allows the similarity between the key information and the first result information to be obtained using literal similarity when the literal lengths of both the key information and the first result information meet the information length requirement. If the literal length of at least one of the key information and the first result information does not meet the information length requirement, the similarity between the key information and the first result information can be obtained using feature vectors. This balances the efficiency and quality of obtaining the similarity between the key information and the first result information in different application scenarios.

[0047] In practice, multiple evaluation levels with different similarity threshold ranges can be set. This allows for determining the corresponding evaluation level based on the similarity threshold range into which the key information and the first result information fall after obtaining the similarity. The first evaluation information is then generated based on this evaluation level, thus more intuitively reflecting the translation quality of the translation model.

[0048] The translation quality determination method provided in this embodiment of the present disclosure is based on replacing key information with interrogative words in a second corpus that corresponds semantically to the first corpus to construct a third corpus with interrogative sentences. After processing the first corpus using a translation model to obtain a fourth corpus with the same language type as the second corpus, the translation quality of the translation model is evaluated based on the similarity between the aforementioned key information and the first result information in the fourth corpus that corresponds to the interrogative points in the third corpus. This achieves translation quality evaluation of the translation model at the semantic level.

[0049] In some optional implementations of this embodiment, the translation quality determination method further includes: in response to the similarity between the key information and the first result information being lower than a preset similarity threshold, obtaining component information of the sentence components that the key information performs in the second corpus; and generating fourth evaluation information based on the component information.

[0050] Specifically, when the similarity between the key information and the first result information is lower than a preset similarity threshold, the component information of the sentence components that the key information performs in the second corpus is obtained. The similarity threshold is usually determined based on different application scenarios. Usually, when it is determined that the similarity between the key information and the first result information is lower than the preset similarity threshold, the translation quality of the translation model cannot meet the expected requirements. The component information is used to indicate the sentence components that the key information performs in the second corpus, and a fourth evaluation information is generated based on the component information. This allows the translation model to be adjusted and trained in a targeted manner after receiving feedback on its translation capabilities for each specific sentence component.

[0051] Please refer to Figure 3 , Figure 3 A flowchart of another translation quality determination method provided in this disclosure embodiment, wherein process 300 includes the following steps:

[0052] Step 301: Obtain the first and second corpora, which are in different languages ​​but have the same semantics.

[0053] Step 302: Determine at least one first key piece of information from the first corpus.

[0054] In this embodiment, at least one key piece of information can be determined from the first corpus, so that the second key piece of information corresponding to the first key piece of information in the second corpus can be determined based on the first key piece of information through sentence alignment, sentence component decomposition, etc., so as to avoid problems such as improper determination of key information and low quality caused by the inability of sentence components to strictly correspond due to different languages, and improve the quality of the determined key information.

[0055] In practice, punctuation and sentence segmentation rules for the first and second corpora can be pre-configured to better align sentences between them. After sentence splitting and alignment, sentence sets can be formed from the split sentences to test the translation model's ability to translate each split sentence, avoiding information omissions caused by differences in punctuation and sentence segmentation rules that could affect the accuracy of translation quality determination.

[0056] When determining the second key information in the second corpus corresponding to the first key information based on the first key information through methods such as sentence alignment and sentence component decomposition,

[0057] Step 303: Replace the second key information in the second corpus that corresponds to the first key information with interrogative words to construct a third corpus with interrogative sentences.

[0058] In this embodiment, after determining the first key information in the first corpus based on the above step 302, the second key information corresponding to the first key information and located in the second corpus is determined from the second corpus, and a third corpus with the sentence structure of an interrogative sentence is constructed based on the second key information.

[0059] Step 304: Process the first corpus using a translation model to generate a fourth corpus corresponding to the first corpus.

[0060] Step 305: Determine the first result information in the fourth corpus that corresponds to the question point in the third corpus.

[0061] Step 306: Generate first evaluation information based on the similarity between the key information and the first result information to evaluate the translation quality of the translation model.

[0062] The above steps 301, 304-306 and as follows Figure 2 The steps 201-205 shown correspond to each other. For the same parts, please refer to the corresponding parts of the previous embodiment. They will not be repeated here. This embodiment is based on the above. Figure 2 Based on the illustrated embodiment, after determining the first key information based on the first corpus, the second key information is determined in the second corpus in accordance with the first key information. This avoids problems such as improper determination of key information and low quality caused by the inability of sentence components to strictly correspond due to different languages, thereby improving the quality of the determined key information.

[0063] In some optional implementations of this embodiment, the translation quality determination method further includes: generating second result information that is semantically identical to the first result information and is in the language of the first corpus; and generating second evaluation information for evaluating the translation quality of the translation model based on the similarity between the first key information and the second result information.

[0064] Specifically, the first result can be processed by human or a translation model or translator that meets the credibility requirements to generate a second result that is semantically identical to the first result and is in the language of the first corpus. The similarity between the first key information and the second result information is used to form a second evaluation information for evaluating the translation quality of the translation model, so as to use the second evaluation information to provide feedback on the translation quality of the translation model corresponding to the language of the first corpus.

[0065] In some optional implementations of this embodiment, the translation quality determination method further includes: weighting the first evaluation information and the second evaluation information to generate third evaluation information.

[0066] Specifically, after obtaining the first and second evaluation information, i.e., after determining the translation quality of the translation model in the languages ​​of the first and second corpora respectively, the first and second evaluation information can be weighted based on the pre-configured value ratio to generate third evaluation information for feedback on the overall translation capability of the translation model, so as to provide a more complete and comprehensive feedback on the translation quality of the translation model.

[0067] Furthermore, corresponding evaluation information (such as fifth evaluation information) can be generated based on the difference between the first and second evaluation information, so as to provide feedback on the differences in translation quality between different languages ​​through the fifth evaluation information.

[0068] Based on any of the above embodiments, in some embodiments of this disclosure, the translation quality determination method further includes: extracting an interrogative sentence format template corresponding to the interrogative word from a pre-configured interrogative sentence database, wherein the interrogative sentence format template is used to indicate the order of each sentence component in the interrogative sentence; adjusting the order of each sentence component in the third corpus based on the interrogative sentence format template to generate an optimized third corpus.

[0069] Specifically, after replacing the key information in the second corpus with interrogative words to construct a third corpus with interrogative sentences, an interrogative sentence format template corresponding to the interrogative word can be extracted from a pre-configured interrogative sentence database based on the determined interrogative word. This database contains various interrogative sentence format templates for different interrogative words (the specific content of the interrogative word and the sentence component it constitutes). These templates are determined based on language usage habits in real-world scenarios. By using these templates, the order of sentence components in the third corpus can be adjusted to generate an optimized third corpus with a more realistic arrangement of sentence components. For example, after replacing the key information and sentence component "table" in "This is atable" with "What," the resulting third corpus is "This is a what?". Based on the interrogative sentence format template corresponding to "What," the third corpus can be optimized to "What is this?". This adjustment of the generated third corpus using interrogative sentence format templates results in interrogative sentences that are closer to real-world scenarios, reducing translation quality interference caused by differences in the arrangement of sentence components compared to real-world scenarios and improving the accuracy of the determined translation quality.

[0070] To deepen understanding, this disclosure also provides a specific implementation scheme based on a concrete application scenario. Taking the first corpus, "At the recognition end, you can pass through the gate through your own face," and the second corpus, "At therecognition end, you can pass through the gate through your own face," as examples, the processing procedure after obtaining the first and second corpora is described in detail below. Figure 4 The process shown in step 400 is as follows:

[0071] The first key information, “recognition end” and “gate”, is determined from the first corpus. The second key information, “the recognition end” and “the gate”, corresponding to the first key information, is determined from the second corpus.

[0072] Then, by replacing the second key information "the recognition end" and "the gate" in the second corpus with the interrogative word "What", a third corpus of interrogative sentences was constructed: Q1 "At what, you can pass through the gate through your own face?" and Q2 "At the recognition end, you can pass through what through your own face?". The first corpus was then processed using a translation model to obtain a fourth corpus corresponding to the first corpus and in the same language as the second corpus: "In the recognition end, you can do it by your face."

[0073] Next, in the fourth corpus, the first result information corresponding to the question points in the third corpus is determined, namely M1 "the recognition end" corresponding to Q1 and M2 "it" corresponding to Q2.

[0074] Finally, based on the similarity between the first result information and the corresponding key information (M1-Ans1, M2-Ans2) (M1-Ans1: 0.99, M2-Ans2: 0.15), first evaluation information is generated to evaluate the translation quality of the translation model.

[0075] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a translation quality determination apparatus, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0076] like Figure 5 As shown, the translation quality determination device 500 of this embodiment may include: a corpus acquisition unit 501, an interrogative corpus construction unit 502, a corpus translation unit 503, a first result information generation unit 504, and a first evaluation information generation unit 505. The system includes: a corpus acquisition unit 501, configured to acquire a first corpus and a second corpus, which are in different languages ​​but semantically consistent; an interrogative corpus construction unit 502, configured to replace key information in the second corpus with interrogative words to construct a third corpus with interrogative sentences; a corpus translation unit 503, configured to process the first corpus using a translation model to generate a fourth corpus corresponding to the first corpus, wherein the translation model is used to transform the corpus between different languages, and the fourth corpus is in the same language as the second corpus; a first result information generation unit 504, configured to determine the first result information in the fourth corpus corresponding to the interrogative points in the third corpus; and a first evaluation information generation unit 505, configured to generate first evaluation information based on the similarity between the key information and the first result information to evaluate the translation quality of the translation model.

[0077] In this embodiment, the specific processing and technical effects of the following components in the translation quality determination device 500—including the corpus acquisition unit 501, the interrogative corpus construction unit 502, the corpus translation unit 503, the first result information generation unit 504, and the first evaluation information generation unit 505—can be referred to separately. Figure 2 The relevant descriptions of steps 201-205 in the corresponding embodiments will not be repeated here.

[0078] In some optional implementations of this embodiment, the interrogative corpus construction unit 502 includes: a first key information determination subunit, configured to determine at least one first key information from the first corpus; and an interrogative corpus construction subunit, configured to replace the second key information in the second corpus corresponding to the first key information with an interrogative word to construct a third corpus with an interrogative sentence structure.

[0079] In some optional implementations of this embodiment, the translation quality determination device 500 further includes: a second result information generation unit configured to generate second result information that is semantically identical to the first result information and is in the language of the first corpus; and a second evaluation information generation unit configured to generate second evaluation information for evaluating the translation quality of the translation model based on the similarity between the first key information and the second result information.

[0080] In some optional implementations of this embodiment, the translation quality determination device 500 further includes: a third evaluation information generation unit, configured to weight the first evaluation information and the second evaluation information to generate third evaluation information.

[0081] In some optional implementations of this embodiment, the translation quality determination device 500 further includes: a component information acquisition unit configured to acquire component information of the key information as a sentence component in the second corpus in response to the similarity between the key information and the first result information being lower than a preset similarity threshold; and a fourth evaluation information generation unit to generate fourth evaluation information based on the component information.

[0082] In some optional implementations of this embodiment, the translation quality determination device 500 further includes: a sentence component sorting acquisition unit, configured to extract an interrogative sentence format template corresponding to the interrogative word from a pre-configured interrogative sentence library, wherein the interrogative sentence format template is used to indicate the arrangement order of each sentence component in the interrogative sentence; and a third corpus optimization unit, configured to adjust the arrangement order of each sentence component in the third corpus based on the interrogative sentence format template to generate an optimized third corpus.

[0083] This embodiment exists as a device embodiment corresponding to the above method embodiment. The translation quality determination device provided in this embodiment constructs a third corpus with interrogative sentences by replacing key information with interrogative words in a second corpus that corresponds to the semantics of the first corpus. After processing the first corpus using a translation model to obtain a fourth corpus with the same language type as the second corpus, the translation quality of the translation model is evaluated based on the similarity between the aforementioned key information and the first result information in the fourth corpus that corresponds to the interrogative points in the third corpus. This realizes the translation quality evaluation of the semantic-level translation model.

[0084] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0085] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0086] like Figure 6As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0087] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0088] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the translation quality determination method. For example, in some embodiments, the translation quality determination method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the translation quality determination method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the translation quality determination method by any other suitable means (e.g., by means of firmware).

[0089] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0090] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0091] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0092] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0093] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0094] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are hosting products within the cloud computing service ecosystem to address the management difficulties and weak business scalability inherent in traditional physical hosts and Virtual Private Servers (VPS) services. Servers can also be categorized as distributed system servers or servers incorporating blockchain technology.

[0095] According to the technical solution of this disclosure, a third corpus with interrogative sentences is constructed by replacing key information with interrogative words in a second corpus that corresponds semantically to the first corpus. After processing the first corpus using a translation model to obtain a fourth corpus with the same language type as the second corpus, the translation quality of the translation model is evaluated based on the similarity between the aforementioned key information and the first result information in the fourth corpus that corresponds to the interrogative point in the third corpus. This achieves semantic-level translation quality evaluation of the translation model.

[0096] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0097] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining translation quality, comprising: Obtain first and second corpora with different languages ​​but consistent semantics; The process involves replacing key information in the second corpus with interrogative words to construct a third corpus with interrogative sentences, including: determining at least one first key piece of information from the first corpus; replacing the second key piece of information in the second corpus corresponding to the first key piece of information with interrogative words to construct a third corpus with interrogative sentences. The first corpus is processed using a translation model to generate a fourth corpus corresponding to the first corpus. The translation model is used to transform the corpus between different languages, and the fourth corpus is in the same language as the second corpus. In the fourth corpus, a first result information corresponding to the question point in the third corpus is determined, wherein the first result information includes the content in the fourth corpus referred to or inquired about by the question point; First evaluation information is generated based on the similarity between the second key information and the first result information to evaluate the translation quality of the translation model.

2. The method according to claim 1, further comprising: Generate second result information that has the same semantics as the first result information and is in the language of the first corpus; Based on the similarity between the first key information and the second result information, a second evaluation information is generated to evaluate the translation quality of the translation model.

3. The method according to claim 2, further comprising: The first evaluation information and the second evaluation information are weighted together to generate the third evaluation information.

4. The method according to claim 1, further comprising: In response to the fact that the similarity between the key information and the first result information is lower than a preset similarity threshold, the component information of the key information as a sentence component in the second corpus is obtained; A fourth evaluation message is generated based on the component information.

5. The method according to any one of claims 1-4, further comprising: Extract the question sentence format template corresponding to the question word from the pre-configured question sentence statement library, wherein the question sentence format template is used to indicate the order of the sentence components in the question sentence; Based on the interrogative sentence format template, the order of each sentence component in the third corpus is adjusted to generate an optimized third corpus.

6. A translation quality determination device, comprising: The corpus acquisition unit is configured to acquire a first corpus and a second corpus that are different in language but have the same semantics; The interrogative corpus construction unit is configured to replace key information in the second corpus with interrogative words to construct a third corpus with interrogative sentences, including: a first key information determination subunit, configured to determine at least one first key information from the first corpus; and an interrogative corpus construction subunit, configured to replace second key information in the second corpus corresponding to the first key information with interrogative words to construct a third corpus with interrogative sentences. The corpus translation unit is configured to process the first corpus using a translation model to generate a fourth corpus corresponding to the first corpus, wherein the translation model is used to transform the corpus between different languages, and the fourth corpus is in the same language as the second corpus; The first result information generation unit is configured to determine first result information in the fourth corpus corresponding to the question point in the third corpus, wherein the first result information includes the content in the fourth corpus referred to and inquired by the question point; The first evaluation information generation unit is configured to generate first evaluation information for evaluating the translation quality of the translation model based on the similarity between the second key information and the first result information.

7. The apparatus according to claim 6, further comprising: The second result information generation unit is configured to generate second result information in the same language as the first result information but in the same language as the first corpus. The second evaluation information generation unit is configured to generate second evaluation information for evaluating the translation quality of the translation model based on the similarity between the first key information and the second result information.

8. The apparatus according to claim 7, further comprising: The third evaluation information generation unit is configured to weight the first evaluation information and the second evaluation information to generate the third evaluation information.

9. The apparatus according to claim 6, further comprising: The component information acquisition unit is configured to acquire the component information of the key information as a sentence component in the second corpus in response to the similarity between the key information and the first result information being lower than a preset similarity threshold; The fourth evaluation information generation unit generates fourth evaluation information based on the component information.

10. The apparatus according to any one of claims 6-9, further comprising: The statement component sorting and acquisition unit is configured to extract the question sentence format template corresponding to the question word from a pre-configured question sentence statement library, wherein the question sentence format template is used to indicate the arrangement order of each statement component in the question sentence; The third corpus optimization unit is configured to adjust the order of sentence components in the third corpus based on the interrogative sentence format template to generate an optimized third corpus.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the translation quality determination method according to any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the translation quality determination method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the translation quality determination method according to any one of claims 1-5.

Images