Translation Quality Assessment Method, Apparatus, Device, and Computer Storage Medium

Abstract

The present application discloses a translation quality evaluation method, device, equipment and computer storage medium, which can be applied to scenarios such as cloud technology. Specifically, the translation quality evaluation method can obtain the source sentence and the corresponding candidate translation sentence; mark the zero pronouns hidden in the source sentence, and identify the target pronoun positions of the zero pronouns in the source sentence to be evaluated after marking; determine the alignment information between the source sentence to be evaluated and the candidate translation sentence; determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence according to the alignment information and the target pronoun positions; obtain the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculate the quality evaluation score corresponding to the candidate translation sentence according to the number of correct translation pronouns and the total number of zero pronouns. In this way, it is possible to evaluate the translation quality of the candidate translation sentence according to the correct translation of the zero pronouns, so as to evaluate whether the translation is consistent with the semantics and syntactic structure of the source text, and improve the accuracy of translation quality evaluation.

Description

Technical Field

[0001] The present application relates to the field of computer technologies, and particularly to a method, apparatus, device, and computer storage medium for translation quality evaluation. Background Art

[0002] Deep neural networks are widely applied in the field of artificial intelligence, such as fields of semantic recognition, computer vision, natural language processing, machine translation, etc. Among them, machine translation, as a way of translation between different languages, has great significance at the application level, and at the same time, the accuracy of machine translation has also received much attention. In related technologies, when performing machine translation, the performance of a machine translation model on training set data is mainly evaluated by a bilingual evaluation understudy (BLEU) score, and a high-performance machine translation model is used to translate a text to be translated to obtain a target translation after translation.

[0003] In the process of researching and practicing the existing technologies, the inventors of the present application found that when using machine translation in the existing technologies, although the BLEU score can evaluate the overall performance of a machine translation model, due to the phenomenon that some languages may have omitted pronouns in expressions, the BLEU score cannot evaluate the quality of the model in translating the omitted pronouns, which may lead to differences in semantics, syntactic structures, etc. between the translation obtained by the model and the source text, and the errors in the translation cannot be recognized, affecting the accuracy in translation quality evaluation. Summary of the Invention

[0004] Embodiments of the present application provide a method, apparatus, device, and computer storage medium for translation quality evaluation, which can evaluate whether the translation obtained is consistent with the semantics and syntactic structure of the source text, and improve the accuracy in translation quality evaluation.

[0005] Embodiments of the present application provide a method for translation quality evaluation, including:

[0006] Obtaining a source sentence and a corresponding candidate translation sentence;

[0007] Marking zero pronouns hidden in the source sentence, and identifying target pronoun positions of the zero pronouns in the marked source sentence;

[0008] Determining alignment information between the source sentence to be evaluated and the candidate translation sentence;

[0009] Determining the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence according to the alignment information and the target pronoun positions;

[0010] Obtain the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculate the quality evaluation score corresponding to the candidate translation sentence according to the number of correctly translated pronouns and the total number of zero pronouns.

[0011] Correspondingly, an embodiment of the present application provides a translation quality evaluation device, including:

[0012] An acquisition unit, configured to acquire a source sentence and a corresponding candidate translation sentence;

[0013] A marking unit, configured to mark zero pronouns hidden in the source sentence, and identify the target pronoun positions of the zero pronouns in the marked source sentence;

[0014] A first determination unit, configured to determine the alignment information between the source sentence to be evaluated and the candidate translation sentence;

[0015] A second determination unit, configured to determine the number of correctly translated pronouns corresponding to the zero pronouns in the candidate translation sentence according to the alignment information and the target pronoun positions;

[0016] A calculation unit, configured to obtain the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculate the quality evaluation score corresponding to the candidate translation sentence according to the number of correctly translated pronouns and the total number of zero pronouns.

[0017] In some embodiments, the second determination unit is further configured to:

[0018] Determine candidate translation pronouns in the candidate translation sentence that are aligned with the zero pronouns according to the alignment information and the target pronoun positions;

[0019] Based on the candidate translation pronouns aligned with the zero pronouns, determine correctly translated pronouns equivalent to the zero pronouns;

[0020] Obtain the number of correctly translated pronouns.

[0021] In some embodiments, the second determination unit is further configured to:

[0022] Determine the target pronoun positions of the zero pronouns in the source sentence to be evaluated according to the zero pronoun marking;

[0023] According to the alignment information and the target pronoun positions, search for candidate translation words in the candidate translation sentence that are aligned with the zero pronouns to obtain a search result;

[0024] Based on the search result, determine candidate translation pronouns aligned with the zero pronouns.

[0025] In some embodiments, the second determination unit is further configured to:

[0026] If it is detected that the search result does not contain the candidate translation words aligned with the zero pronoun, then according to the alignment information and the position of the target pronoun, a translation word region corresponding to the zero pronoun in the candidate translation sentence is selected;

[0027] If it is recognized that the translation word region contains translation words, the translation words are determined as candidate translation pronouns aligned with the zero pronoun.

[0028] In some embodiments, the second determination unit is further configured to:

[0029] Determine the position of the word adjacent to the position of the target pronoun in the source sentence to be evaluated;

[0030] According to the position of the target pronoun and the position of the word, determine the pronoun region adjacent to the zero pronoun in the source sentence to be evaluated;

[0031] According to the alignment information, determine the corresponding translation word region of the pronoun region in the candidate translation sentence.

[0032] In some embodiments, the second determination unit is further configured to:

[0033] Match the candidate translation pronouns aligned with the zero pronoun from a preset bilingual dictionary to obtain a matching result;

[0034] If the matching result contains a preset pronoun that matches the candidate translation pronoun, the candidate translation pronoun is determined as the correct translation pronoun equivalent to the zero pronoun.

[0035] In some embodiments, the second determination unit is further configured to:

[0036] If it is detected that the search result contains the candidate translation words aligned with the zero pronoun, the candidate translation words are determined as the candidate translation pronouns aligned with the zero pronoun.

[0037] In some embodiments, the obtaining unit is further configured to:

[0038] Obtain the source text to be translated, and extract a plurality of source sentences from the source text to be translated;

[0039] Perform translation processing on the source text to be translated through a machine translation model to obtain a candidate translation sentence corresponding to each source sentence.

[0040] In addition, an embodiment of the present application further provides a computer device, including a processor and a memory, where the memory stores a computer program, and the processor is configured to run the computer program in the memory to implement the steps in any one of the translation quality evaluation methods provided by the embodiments of the present application.

[0041] In addition, an embodiment of the present application further provides a computer-readable storage medium storing multiple instructions, which are suitable for being loaded by a processor to execute the steps in any one of the translation quality assessment methods provided by the embodiments of the present application.

[0042] In addition, an embodiment of the present application further provides a computer program product including computer instructions, which when executed implement the steps in any one of the translation quality assessment methods provided by the embodiments of the present application.

[0043] The embodiments of the present application can obtain a source sentence and a corresponding candidate translation sentence; mark the zero pronouns hidden in the source sentence, and identify the target pronoun positions of the zero pronouns in the marked source sentence; determine the alignment information between the source sentence to be evaluated and the candidate translation sentence; according to the alignment information and the target pronoun positions, determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence; obtain the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculate the quality assessment score corresponding to the candidate translation sentence according to the number of correct translation pronouns and the total number of zero pronouns. Thus, it can be seen that this solution can pre-mark the zero pronouns in the source sentence to be translated to determine the alignment information between the candidate translation sentence and the marked source sentence to be evaluated, determine the zero pronouns in the source sentence to be evaluated that are correctly translated into the candidate translation sentence according to the alignment information, so as to determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence. Furthermore, calculate the quality assessment score of the candidate translation sentence according to the total number of zero pronouns corresponding to the source sentence to be evaluated and the number of correct translation pronouns; in this way, after obtaining the candidate translation sentence, it is possible to evaluate the translation quality of the candidate translation sentence according to the number of correct translations of zero pronouns, so as to evaluate whether the translated text is consistent with the source text in semantics and syntactic structure, and improve the accuracy of translation quality assessment. Description of the Drawings

[0044] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0045] Figure 1 is a schematic diagram of the scenario of the translation quality assessment system provided by the embodiments of the present application;

[0046] Figure 2 is a schematic flowchart of the steps of the translation quality assessment method provided by the embodiments of the present application;

[0047] Figure 3 is another schematic flowchart of the steps of the translation quality assessment method provided by the embodiments of the present application;

[0048] Figure 4 It is an example diagram of the machine translation quality scenario provided by the embodiments of the present application;

[0049] Figure 5 It is a schematic diagram of the scenario of the translation quality evaluation method provided by the embodiments of the application;

[0050] Figure 6 It is a schematic structural diagram of the translation quality evaluation device provided by the embodiments of the present application;

[0051] Figure 7 It is a schematic structural diagram of the computer device provided by the embodiments of the present application. Detailed implementation manners

[0052] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0053] The embodiments of the present application provide a translation quality evaluation method, device, equipment, and computer storage medium. Specifically, the embodiments of the present application will be described from the perspective of the translation quality evaluation device. The translation quality evaluation device can be specifically integrated in a computer device, and the computer device can be a server or a user terminal or other devices. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Among them, the user terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart home appliance, a vehicle terminal, a smart voice interaction device, an aircraft, etc., but is not limited thereto.

[0054] The translation quality assessment method provided by the embodiments of the present application can be applied to various scenarios involving image clarity recognition, such as cloud technology, artificial intelligence, intelligent transportation, assisted driving, etc., and these scenarios can be implemented through cloud service methods. For example, artificial intelligence cloud services, generally also known as AIaaS (AI as a Service, Chinese for "AI as a service"). This is a current mainstream service method for artificial intelligence platforms. Specifically, the AIaaS platform will split several common AI services and provide independent or packaged services in the cloud. This service model is similar to opening an AI-themed mall: all developers can access and use one or more artificial intelligence services provided by the platform through the API interface, and some senior developers can also use the AI frameworks and AI infrastructure provided by the platform to deploy and operate their own exclusive cloud artificial intelligence services. The specific description is as follows through the following embodiments:

[0055] For example, refer to Figure 1 , which is a schematic diagram of the scenario of the translation quality assessment system provided by the embodiments of the present application. This scenario includes a terminal or a server.

[0056] The terminal or the server can obtain the source sentence and the corresponding candidate translation sentence; mark the zero pronouns hidden in the source sentence and identify the target pronoun positions of the zero pronouns in the marked source sentence; determine the alignment information between the source sentence to be evaluated and the candidate translation sentence; determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence according to the alignment information and the target pronoun positions; obtain the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculate the quality assessment score corresponding to the candidate translation sentence according to the number of correct translation pronouns and the total number of zero pronouns.

[0057] Among them, the process of translation quality assessment can include: obtaining the source sentence to be translated and the corresponding candidate translation sentence, marking the zero pronouns in the source sentence, determining the alignment information between the source sentence and the candidate translation sentence, determining the number of correct translations of the zero pronouns, calculating the quality assessment score, etc.

[0058] The following will be described in detail respectively. It should be noted that the order of the following embodiments does not limit the preferred order of the embodiments.

[0059] In the embodiments of the present application, the description will be made from the perspective of the translation quality assessment device, and the translation quality assessment device can be specifically integrated in a computer device such as a terminal or a server. Refer to Figure 2 , Figure 2 , which is a schematic diagram of the step flow of a translation quality assessment method provided by the embodiments of the present application. Taking the translation quality assessment device specifically integrated on the server or the terminal as an example in the embodiments of the present application, when the processor on the server or the terminal executes the program instructions corresponding to the translation quality assessment method, the specific process is as follows:

[0060] 101. Obtain the source sentence and the corresponding candidate translation sentence.

[0061] Among them, the source sentence can be the original sentence in the target language. In the embodiments of the present application, the source sentence refers to the original sentence in a certain language to be translated, which is usually a semantically complete sentence. For example, taking Chinese as the target language as an example, the source sentence in Chinese can be translated into translation sentences in other languages. For example, "I love China" belongs to the source sentence, and "Do you like to eat cakes?" and "Very much" also respectively belong to a source sentence; the above Chinese language is only an example, and it can also be languages such as Japanese and English.

[0062] Among them, the candidate translation sentence can be the translation result in other languages corresponding to the source sentence. For example, taking the source sentence in Chinese as an example, the source sentence in Chinese can be translated into candidate translation sentences in other languages, such as translation sentences in languages such as English, French, and German. The above is only an example, and the embodiments of the present application do not limit the languages.

[0063] In order to evaluate the translation quality, in the embodiments of the present application, taking the terminal or the server as an example of a computer device, the computer device can be applied to the translation quality evaluation scenarios of commercial / platform / personal in the embodiments of the present application. Specifically, the computer device can obtain the source sentence and the candidate translation sentence corresponding to the source sentence in real time, so as to facilitate the subsequent evaluation of the translation quality of the candidate translation sentence in combination with the sentence conditions of the source sentence and the candidate translation sentence.

[0064] In some embodiments, in order to evaluate the accuracy of machine translation, the candidate translation sentence can be obtained by means of machine translation. For example, the source sentence is translated through a pre-prepared machine translation model to obtain the candidate translation sentence corresponding to the source sentence. Then the step of "obtaining the source sentence and the corresponding candidate translation sentence" may include: obtaining the source text to be translated, and extracting a plurality of source sentences from the source text to be translated; translating and processing the plurality of source sentences through a machine translation model to obtain the candidate translation sentence corresponding to each source sentence.

[0065] Among them, the source text to be translated can be an article / passage containing one or more source sentences, which belongs to the original text that needs to be translated into a certain language according to the will of the user or the requester.

[0066] Specifically, the computer device receives in real time the source text to be translated requested by the target object, and extracts source sentences from the source text to be translated to extract multiple source sentences; furthermore, the multiple source sentences are translated through a pre-trained machine translation model. During the translation process of the model, each source sentence can be translated separately, or multiple source sentences can be translated simultaneously. This is not limited here. Thus, candidate translation sentences corresponding to each source sentence are obtained to facilitate subsequent translation quality evaluation for each source sentence and its corresponding candidate translation sentence.

[0067] Through the above method, the source sentences before translation and the candidate translation sentences obtained by translation can be obtained to facilitate subsequent translation quality evaluation of the source sentences and the candidate translation sentences to determine the accuracy of the candidate translation sentences.

[0068] It should be noted that in specific language expressions (such as Chinese and Japanese), some pronouns are often omitted to achieve the effect of concise expression, that is, some pronouns are omitted literally or descriptively, but the user can infer the missing pronoun information through the context. The missing pronoun information is the "zero pronoun" or "elliptical pronoun" described in this application. The absence of the zero pronoun or elliptical pronoun does not affect the user's understanding of the content. For example, in languages such as Chinese and Japanese, pronoun ellipsis is relatively common. On the contrary, in languages such as English and French, pronouns are essential sentence components. Therefore, when translating between different languages, attention needs to be paid to the problem of pronoun omission to determine whether the translation result (translation) is affected by the zero pronoun (zero anaphora) situation in the source sentence, so as to learn about the expression of chapter information in the translated text by zero anaphora and ensure the coherence of the text.

[0069] Therefore, in the embodiments of this application, when subsequently evaluating the translation quality of the translation result (i.e., the candidate translation sentence), it can be determined according to the situation where the zero pronoun / elliptical pronoun (Zero Pronoun, ZP) in the source sentence is correctly translated into the candidate translation sentence, such as whether the candidate translation sentence contains a candidate translation pronoun corresponding to the zero pronoun, and whether the position and quantity of the candidate translation pronoun are consistent with the source sentence, etc., so as to determine the quality of the candidate translation sentence according to the above translation situation of the zero pronoun, and effectively avoid the phenomenon that the translation quality is affected due to semantic errors caused by pronoun omission when translating between different languages.

[0070] 102. Mark the hidden zero pronouns in the source sentence and identify the target pronoun positions of the zero pronouns in the marked source sentence.

[0071] Among them, the zero pronoun can be a word / pronoun with a referential relationship in a sentence (source sentence), which is usually omitted in specific language expressions (such as Chinese and Japanese) to achieve a concise expression effect. It should be noted that the omission of this pronoun does not affect the semantics of the corresponding specific language expression, and the user can infer the missing pronoun information through its context.

[0072] Among them, the target pronoun position can be the position information where the corresponding zero pronoun needs to have a referential relationship in the source sentence, that is, the position information where the zero pronoun can appear in the sentence.

[0073] To facilitate subsequent evaluation of the quality of the translation result, the embodiments of this application need to first mark the missing zero pronouns in the source sentence to obtain the source sentence to be evaluated after marking, where the source sentence to be evaluated contains zero pronoun marks; furthermore, according to the zero pronoun marks, identify the position information of the corresponding zero pronoun in the source sentence to be evaluated after marking, that is, the target pronoun position, so as to facilitate subsequent evaluation of the translation quality of the translation result in combination with the target pronoun position.

[0074] In some embodiments, the process of marking the hidden zero pronouns in the source sentence can be: performing part-of-speech tagging on the source sentence according to a preset pronoun dataset to mark the hidden zero pronouns in the source sentence and obtain the source sentence to be evaluated after marking. It should be noted that the preset pronoun dataset can be a preset dataset. For example, taking the OntoNotes5.0 dataset as an example, divide the OntoNotes5.0 dataset according to the dataset division method to obtain a preset pronoun dataset for zero pronoun marking; furthermore, mark the zero pronouns in the source sentence. Since the source sentence usually consists of multiple words, that is, the source sentence contains multiple words, such as the source sentence S = {w, w,..., w}, there may be zero pronouns in the regions adjacent to each word w. The adjacent positions of each word w in the source sentence can also be marked for the zero pronoun position to obtain a source sentence containing zero pronoun (position) marks, that is, the source sentence to be evaluated after marking.

[0075] In some embodiments, the process of identifying the target pronoun position of the zero pronoun in the source sentence to be evaluated after marking can be: identifying the zero pronoun marks in the source sentence to be evaluated after marking and determining the target pronoun position of the zero pronoun in the source sentence according to the zero pronoun marks. In addition, when there are multiple zero pronoun marks in the source sentence to be evaluated, a pre-trained pronoun position recognition model can also be used to identify the zero pronouns in the source sentence to be evaluated to output the probabilities of the zero pronouns at each marked position. Thus, determine the target pronoun position of the zero pronoun in the source sentence to be evaluated according to the probabilities at each marked position.

[0076] In the above manner, zero pronoun marking can be performed on the source sentence, and the position of the zero pronoun in the source sentence to be evaluated after marking can be identified, so as to obtain the source sentence to be evaluated with zero pronoun marking and the target pronoun position of the zero pronoun, which is convenient for subsequent participation of the source sentence to be evaluated, the target pronoun position, etc. in the translation quality evaluation of the candidate translation sentence.

[0077] 103. Determine the alignment information between the source sentence to be evaluated and the candidate translation sentence.

[0078] Among them, the alignment information may be the alignment relationship information between sentence components such as the subject, predicate, object, and modifier between two sentences. In addition, the alignment information may also be the alignment relationship information in the form of "entity - predicate - entity", "entity - relationship - entity", etc. In the embodiments of the present application, taking sentence components such as the subject, predicate, object, and modifier as examples, alignment is performed according to any one or more of the sentence components such as the subject, predicate, object, and modifier between the source sentence to be evaluated and the candidate translation sentence to obtain the above alignment information. In the embodiments of the present application, the alignment information includes the alignment relationship of multiple sentence components between the source sentence to be evaluated and the candidate translation sentence. Among them, the number of each type of sentence component depends on the structure of the actual sentence (such as the source sentence to be evaluated and the candidate translation sentence), that is, the alignment information may include multiple alignment relationships of the same type but different sentence component semantics / references. For example, for the source sentence to be evaluated and the candidate translation sentence, the predicate components included in both are "like" and "eat", then the alignment information includes the alignment relationship information of "like" and "eat" between the source sentence to be evaluated and the candidate translation sentence; the above are only partial examples, and the alignment information of other sentence components can be inferred with reference to this example, and will not be listed one by one here.

[0079] In order to evaluate the translation quality of the candidate translation sentence, the embodiments of the present application need to obtain the alignment information between the source sentence to be evaluated and the candidate translation sentence. Specifically, the correlation relationship in the context of the source sentence to be evaluated can be determined first, and the correlation relationship in the context of the candidate translation sentence can be determined. The correlation relationship may include semantic information such as the semantic relationship within the sentence, semantic dependency, and sentence pattern structure relationship; furthermore, according to the correlation relationship in the source sentence to be evaluated and the correlation relationship in the candidate translation sentence, alignment processing is performed on the source sentence to be evaluated and the candidate translation sentence to obtain the alignment information between the source sentence to be evaluated and the candidate translation sentence, which is used for subsequent participation in the translation quality evaluation of the candidate translation sentence.

[0080] In the above manner, the alignment information between the source sentence to be evaluated and the candidate translation sentence can be obtained, which is convenient for subsequent combination of the alignment information and the target pronoun position of the zero pronoun to evaluate the translation quality of the candidate translation sentence.

[0081] 104. Determine the number of correct translation pronouns corresponding to the zero pronoun in the candidate translation sentence according to the alignment information and the position of the target pronoun.

[0082] It should be noted that the literal translation performance of machine translation is relatively mature, but the phenomenon of omitted pronouns in some languages will affect the translation quality. Therefore, when evaluating the translation quality of candidate translation sentences in the embodiments of the present application, the factor of pronoun omission in the source sentence is mainly considered to further evaluate the translation quality of the candidate translation sentences.

[0083] In order to evaluate the translation quality of candidate translation sentences, the embodiments of the present application need to determine the number of zero pronouns in the source sentence to be evaluated that are correctly translated into the candidate translation sentences. Specifically, according to the alignment information between the source sentence to be evaluated and the candidate translation sentence and the position of the target pronoun of the zero pronoun in the source sentence to be evaluated, determine the number of correct translation pronouns corresponding to the zero pronoun in the candidate translation sentence, so as to facilitate subsequent evaluation of the quality of the candidate translation sentence based on the number of correct translation pronouns.

[0084] In some embodiments, the step of "determining the number of correct translation pronouns corresponding to the zero pronoun in the candidate translation sentence according to the alignment information and the position of the target pronoun" may include:

[0085] (104.1) Determine the candidate translation pronouns aligned with the zero pronoun in the candidate translation sentence according to the alignment information and the position of the target pronoun;

[0086] (104.2) Based on the candidate translation pronouns aligned with the zero pronoun, determine the correct translation pronouns equivalent to the zero pronoun;

[0087] (104.3) Obtain the number of correct translation pronouns.

[0088] Among them, the candidate translation pronoun may be a demonstrative pronoun that is aligned with the zero pronoun in the source sentence to be evaluated after translation. It should be noted that due to factors such as translation differences or alignment deviations, the candidate translation pronoun may not be equivalent to the aligned zero pronoun, that is, the equivalence of the candidate translation pronoun needs to be further confirmed.

[0089] Specifically, in order to determine the number of correct translated pronouns corresponding to the zero pronouns in the candidate translation sentences, the embodiments of the present application need to first determine the candidate translated pronouns aligned with the zero pronouns in the candidate translation sentences according to the alignment information and the target pronoun positions, so as to select the translated pronouns aligned with the zero pronouns in the source sentence to be evaluated in the candidate translation sentences. Furthermore, based on the candidate translated pronouns aligned with the zero pronouns, the correct translated pronouns equivalent to the zero pronouns are determined to filter the candidate translated pronouns aligned with the zero pronouns to obtain the correct translated pronouns equivalent to the zero pronouns. This equivalence can be semantic equivalence of pronouns, equivalent reference relationships, etc. For example, there is semantic equivalence, equivalent reference relationships, and / or equivalent sentence structures between the zero pronouns and the correct translated pronouns. It should be noted that after filtering the candidate translated pronouns aligned with the zero pronouns, there may be a phenomenon that some or all of the zero pronouns are not equivalent to the aligned candidate translated pronouns. Specifically, there is no correct translated pronoun equivalent to the zero pronoun in the candidate translation sentence, that is, the zero pronoun in the source sentence to be evaluated is not correctly translated into the candidate translation sentence. Finally, after screening out the correct translated pronouns in the candidate translation sentences according to the equivalence, the number of correct translated pronouns can be obtained to determine the number of zero pronouns in the source sentence that are correctly translated into the candidate translation sentence for subsequent translation quality evaluation.

[0090] In some embodiments, the step of "determining the candidate translated pronouns aligned with the zero pronouns in the candidate translation sentence according to the alignment information and the target pronoun position" may include:

[0091] (104.1.1) Extract the pronoun alignment information corresponding to the target pronoun position from the alignment information;

[0092] (104.1.2) According to the pronoun alignment information, search for the candidate translated words aligned with the zero pronouns in the candidate translation sentence to obtain the search result;

[0093] (104.1.3) Based on the search result, determine the candidate translated pronouns aligned with the zero pronouns.

[0094] In the embodiments of the present application, the process of determining the candidate translated pronouns is a process of searching for the words aligned with the zero pronouns in the candidate translation sentence. Among them, when searching for the target words in the translation sentence, the target words can be specifically determined according to the relevant words (or the position information of the relevant words) of the source sentence and the alignment information. It should be noted that the alignment information includes the alignment relationship of each word or character between the source sentence to be evaluated and the candidate translation sentence, that is, it includes the alignment sub-information of each word between these two sentences. Therefore, in order to obtain the candidate translated pronouns in the candidate translation sentence, they can be determined according to the position information of the zero pronouns in the source sentence to be evaluated and the relevant alignment information.

[0095] To determine candidate translated pronouns aligned with zero pronouns, after obtaining the alignment information and the target pronoun position of the zero pronoun in the embodiments of this application, the pronoun alignment information of the zero pronoun can be determined first; specifically, since there may be many words in the candidate translated sentence and the source sentence to be evaluated, this may affect the processing efficiency of subsequent judgment of the correctness of translated pronouns. Therefore, the above phenomenon can be avoided by improving the purposefulness of word processing. For example, the pronoun alignment information corresponding to the target pronoun position can be extracted from the alignment information to facilitate subsequent targeted processing of the target words in the candidate translated sentence. Then, after obtaining the pronoun alignment information, the words aligned with the zero pronoun, that is, candidate translated words, are searched for in the candidate translated sentence to obtain the search result. Finally, the candidate translated pronouns aligned with the zero pronoun can be determined according to this search result.

[0096] In some embodiments, the search result includes two cases: one is that candidate translated words aligned with the zero pronoun are found, and the other is that candidate translated words aligned with the zero pronoun are not found. Specifically, the step of "determining candidate translated pronouns aligned with the zero pronoun based on the search result" may include:

[0097] (104.1.3.1) If it is detected that the search result does not include candidate translated words aligned with the zero pronoun, then according to the alignment information and the target pronoun position, the corresponding translation word region associated with the zero pronoun in the candidate translated sentence is selected;

[0098] Among them, the translation word region may be: the translation word region aligned with the zero pronoun and other adjacent words in the source sentence to be evaluated in the candidate translated sentence. For example, taking the source sentence to be evaluated S = {w1, w2, w3, w4} as an example, assuming that w2 is the zero pronoun, the translation word region associated with the zero pronoun in the candidate translated sentence can be determined according to w2 and the adjacent words w1 and w3.

[0099] In some embodiments, the step of "selecting the corresponding translation word region associated with the zero pronoun in the candidate translated sentence according to the alignment information and the target pronoun position" may include: determining the word positions adjacent to the target pronoun position in the source sentence to be evaluated; determining the pronoun region adjacent to the zero pronoun in the source sentence to be evaluated according to the target pronoun position and the word positions; and determining the corresponding translation word region of the pronoun region in the candidate translated sentence according to the alignment information.

[0100] It should be noted that when it is detected that the search result does not contain candidate translation words with zero pronoun alignment, other words adjacent to the zero pronoun in the source sentence to be evaluated can be determined first, and a pronoun region corresponding to the zero pronoun and other words can be constructed to determine the candidate word region aligned with the pronoun region according to the alignment information. Specifically, since the target pronoun position represents the referential relationship position of the zero pronoun in the source sentence to be evaluated, the position information of the corresponding zero pronoun in the source sentence to be evaluated can be determined according to the target pronoun position. Thus, the word positions of other words adjacent to the zero pronoun in the source sentence to be evaluated can be determined; furthermore, the pronoun region adjacent to the zero pronoun determined according to the target pronoun position and the adjacent word positions, that is, the pronoun region associated with the zero pronoun in the source sentence to be evaluated is determined; finally, the translation word region corresponding to the pronoun region in the candidate translation sentence can be determined according to the alignment information. Specifically, the region alignment information corresponding to the pronoun region is obtained from the alignment information to determine the translation word region associated with the zero pronoun in the candidate translation sentence according to this region alignment information. In this way, when no candidate translation word directly aligned with the zero pronoun can be found, the pronoun region can be constructed through other target words adjacent to the zero pronoun in the source sentence to be evaluated, and the translation word region aligned with the pronoun region can be determined, so that the candidate can determine the candidate translation pronoun according to this translation word region, expanding the search scope of the candidate translation pronoun corresponding to the zero pronoun, effectively reducing the probability of missing the candidate translation pronoun, and improving the accuracy of subsequent translation quality evaluation.

[0101] (104.1.3.2) If it is recognized that the translation word region contains a translation word, the translation word is determined as the candidate translation pronoun aligned with the zero pronoun.

[0102] Specifically, after determining the translation word region associated with the zero pronoun in the candidate translation sentence, it is necessary to identify this translation word region to determine whether the translation word region contains a translation word. Furthermore, when it is recognized that the translation word region contains a translation word, the translation word contained in this translation word region can be determined as the candidate translation pronoun aligned with the zero pronoun. It can be understood that since the translation word region may include one or more translation words, the candidate translation pronouns aligned with the zero pronoun may include multiple. It should be noted that when it is recognized that this translation word region does not contain a translation word, it means that no candidate translation pronoun aligned with the zero pronoun is found in the candidate translation sentence, that is, the zero pronoun is not correctly translated into the candidate translation sentence.

[0103] On the other hand, if it is detected that the search result contains a candidate translation word with zero pronoun alignment, the candidate translation word is determined as the candidate translation pronoun aligned with the zero pronoun. Specifically, when it is detected that the candidate translation sentence contains a candidate translation word directly aligned with the zero pronoun, this candidate translation word can be determined as the candidate translation pronoun to facilitate subsequent determination of whether it is equivalent to the zero pronoun, thereby evaluating the translation quality of the pronoun.

[0104] Furthermore, the step of "determining a correct translation pronoun equivalent to the zero pronoun based on the candidate translation pronoun aligned with the zero pronoun" may include: matching the candidate translation pronoun aligned with the zero pronoun in a preset bilingual dictionary to obtain a matching result; if the matching result includes a preset pronoun that matches the candidate translation pronoun, then determining the candidate translation pronoun as the correct translation pronoun equivalent to the zero pronoun.

[0105] Among them, the preset bilingual dictionary may be a predefined bilingual dictionary, which may contain information such as the semantics and referential relationships of bilingual pronouns.

[0106] To determine whether the zero pronoun in the source sentence to be evaluated is correctly translated into the candidate translation sentence, an embodiment of the present application may match the candidate translation pronoun aligned with the zero pronoun in the preset bilingual dictionary to obtain a matching result, and the matching result may or may not include a preset pronoun corresponding to the candidate translation pronoun; furthermore, identify the matching result. If it is identified that the matching result includes a preset pronoun that matches the candidate translation pronoun, it means that the zero pronoun in the source sentence to be evaluated is correctly translated into the candidate translation sentence. At this time, the candidate translation pronoun may be determined as the correct translation pronoun equivalent to the zero pronoun; otherwise, it means that the zero pronoun in the source sentence to be evaluated is not correctly translated into the candidate translation sentence. Furthermore, it is convenient to evaluate the translation quality according to the correct translation pronoun subsequently.

[0107] In the above manner, it is possible to determine whether the zero pronoun in the source sentence to be evaluated is correctly translated into the candidate translation sentence according to the alignment information between the source sentence to be evaluated and the candidate translation sentence and the position of the target pronoun of the zero pronoun in the source sentence to be evaluated, and to determine the number of correct translation pronouns, so as to evaluate the translation quality according to the number of correct translation pronouns subsequently.

[0108] 105. Obtain the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculate the quality evaluation score corresponding to the candidate translation sentence according to the number of correct translation pronouns and the total number of zero pronouns.

[0109] Among them, the quality evaluation score may be a score calculated according to the ratio of the number of correct translation pronouns to the total number of zero pronouns in the source sentence to be evaluated, and the quality evaluation score can reflect the situation where the zero pronouns in the source sentence to be evaluated are correctly translated.

[0110] To evaluate the quality of a candidate translation sentence, after obtaining the number of correctly translated pronouns in an embodiment of the present application, the total number of zero pronouns corresponding to the source sentence to be evaluated can be obtained, that is, the total amount of zero pronouns included in the source sentence to be evaluated is counted. Furthermore, by dividing the number of correctly translated pronouns by the total number of zero pronouns, the ratio between the number of correctly translated pronouns and the total number of zero pronouns is calculated to obtain a quality evaluation score. It can be understood that the larger the quality evaluation score, the higher the quality of the corresponding candidate translation sentence. Through this quality evaluation score, the situation where the zero pronouns in the source sentence to be evaluated are correctly translated can be accurately reflected, effectively avoiding the inconsistency between the semantics and syntactic structure of the candidate translation sentence and the source text semantics due to the omission of pronouns, and improving the accuracy in evaluating the translation quality of the candidate translation sentence.

[0111] In some embodiments, when the source text to be translated contains multiple source sentences, the quality evaluation score of the target translation corresponding to the source text to be translated can be calculated for the multiple source sentences to evaluate the quality of the target translation. Among them, the calculation process of the quality evaluation score corresponding to the source text to be translated is as follows: If it is recognized that the source text to be translated contains multiple source sentences, for each source sentence, perform the following steps: "Obtain the candidate translation sentence corresponding to the source sentence; mark the zero pronouns hidden in the source sentence, and identify the target pronoun positions of the zero pronouns in the marked source sentence; determine the alignment information between the source sentence to be evaluated and the candidate translation sentence; and determine the number of correctly translated pronouns corresponding to the zero pronouns in the candidate translation sentence according to the alignment information and the target pronoun positions". Furthermore, based on the number of correctly translated pronouns included in the candidate translation sentences corresponding to each source sentence, calculate the total number of correctly translated pronouns corresponding to all candidate translation sentences; finally, obtain the total number of zero pronouns included in the multiple (to be evaluated) source sentences, that is, the total amount of zero pronouns included in the source text to be translated, and divide the total number of correctly translated pronouns by the total amount of zero pronouns included in the source text to be translated to obtain the quality evaluation score of the target translation.

[0112] An embodiment of the present application mainly proposes a quality evaluation score to evaluate the accuracy of the translation of zero pronouns in the translation (candidate translation sentence) with respect to the source text (source sentence). On the one hand, it considers the presence or absence of zero pronouns in the translation, and on the other hand, it indirectly characterizes the evaluation of the candidate translation sentence in terms of semantics and structure by considering the correctness of the translation of zero pronouns; thereby, it is convenient to subsequently improve the learning and training of the neural network machine translation model regarding omitted pronouns and improve the effect of the translation system. It should be noted that the method provided in the embodiment of the present application is applicable to all translation tasks from pronoun-omitting languages to non-pronoun-omitting languages; in addition, it is also applicable to the evaluation of the translation quality of other omitted words in the original text, such as the evaluation of the translation quality of omitted conjunctions in the source text into other languages.

[0113] As can be seen from the above, the embodiments of the present application can obtain the source sentence and the corresponding candidate translation sentence; mark the zero pronouns hidden in the source sentence, and identify the target pronoun positions of the zero pronouns in the marked source sentence; determine the alignment information between the source sentence to be evaluated and the candidate translation sentence; according to the alignment information and the target pronoun positions, determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence; obtain the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculate the quality evaluation score corresponding to the candidate translation sentence according to the number of correct translation pronouns and the total number of zero pronouns. Thus, it can be obtained that this solution can pre-mark the zero pronouns in the source sentence to be translated to determine the alignment information between the candidate translation sentence and the marked source sentence to be evaluated, determine the zero pronouns in the source sentence to be evaluated that are correctly translated into the candidate translation sentence according to the alignment information, so as to determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence. Furthermore, calculate the quality evaluation score of the candidate translation sentence according to the total number of zero pronouns corresponding to the source sentence to be evaluated and the number of correct translation pronouns; in this way, after obtaining the candidate translation sentence, it is possible to evaluate the translation quality of the candidate translation sentence according to the number of correct translations of the zero pronouns, so as to evaluate whether the translated text is consistent with the source text in semantics and syntactic structure, and improve the accuracy of translation quality evaluation.

[0114] According to the method described in the above embodiments, the following will give examples for further detailed description.

[0115] Taking the translation quality evaluation as an example, the embodiments of the present application will further describe the translation quality evaluation method provided by the embodiments of the present application.

[0116] Figure 3 It is another step process schematic diagram of the translation quality evaluation method provided by the embodiments of the present application. Figure 4 It is an example diagram of the machine translation scenario provided by the embodiments of the application. Figure 5 It is a scenario schematic diagram of the translation quality evaluation method provided by the embodiments of the application. For the sake of easy understanding, the embodiments of the present application are described in combination with Figure 3-4 for description.

[0117] In the embodiments of the present application, the description will be made from the perspective of a translation quality evaluation device, which can be specifically integrated in computer devices such as terminals and / or servers. For example, when the processor on the computer device executes the program corresponding to the translation quality evaluation method, the specific process of the translation quality evaluation method is as follows:

[0118] 201. Obtain multiple source sentences.

[0119] Among them, the multiple source sentences can be multiple source sentences in the same source text to be translated, or sentences in different source texts, which is not limited here.

[0120] Among them, the source sentence can be the original sentence corresponding to the target language. The source sentence contains ellipsis pronouns (zero pronouns), which conform to the concise expression of the target language. For example, in specific languages (such as Chinese and Japanese), some pronouns are often omitted to achieve the effect of concise expression, that is, some pronouns are omitted literally or descriptively, but users can infer the missing pronoun information through the context; and the existence of the zero pronoun or ellipsis pronoun does not affect the user's understanding of the content.

[0121] In the embodiments of the present application, multiple source sentences are used as the data set to describe the subsequent implementation manner of translation quality evaluation.

[0122] 202. Each source sentence is translated through a machine translation model to obtain a candidate translation sentence corresponding to each source sentence.

[0123] It should be noted that since the existing machine translation does not pay attention to the ellipsis pronouns in some languages, the obtained translation results (translation or translation sentence) are inaccurate in semantics and syntactic structure, affecting the accuracy of the translation results; therefore, the embodiments of the present application can evaluate the translation quality of machine translation. It is worth noting that the solution of the embodiments of the present application is not limited to the instruction evaluation scenario of machine translation, but can also be applied to the translation quality evaluation scenario between the translation result obtained by any method and its source sentence.

[0124] In order to evaluate the translation quality of the translation obtained by machine translation, in the embodiments of the present application, each source sentence is translated through a trained machine translation model to obtain the translation corresponding to each source sentence, that is, the candidate translation sentence. Furthermore, each source sentence and its corresponding candidate translation sentence can be used as a group of evaluation data, and thus, a data set is constructed according to multiple groups of data for translation quality evaluation.

[0125] 203. The zero pronouns in the source sentence are marked to obtain a source sentence to be evaluated containing zero pronoun marks, and the target pronoun positions of the zero pronouns in the source sentence to be evaluated are identified.

[0126] Among them, the zero pronoun can be a word / pronoun with a referential relationship in a sentence (source sentence), which is usually omitted in specific languages (such as Chinese and Japanese) to achieve the effect of concise expression.

[0127] Among them, the target pronoun position can be the position information where the corresponding zero pronoun needs to have a referential relationship in the source sentence, that is, the position information where the zero pronoun can appear in the sentence.

[0128] In the embodiments of the present application, for each source sentence, it is necessary to mark the hidden zero pronouns in the source sentence. The process can be as follows: perform part-of-speech tagging on the source sentence according to a preset pronoun dataset to mark the hidden zero pronouns in the source sentence, and obtain the marked source sentence to be evaluated. It should be noted that the preset pronoun dataset can be a pre-set dataset. For example, taking the OntoNotes5.0 dataset as an example, divide the OntoNotes5.0 dataset according to the dataset division method to obtain a preset pronoun dataset for zero pronoun marking; furthermore, perform zero pronoun marking on the source sentence. Since a source sentence usually consists of multiple words, that is, the source sentence contains multiple words, such as the source sentence S = {w, w,..., w}, zero pronouns may exist in the regions adjacent to each word w. The positions of zero pronouns can also be marked at the adjacent positions of each word w in the source sentence to obtain a source sentence with zero pronoun (position) markings, and obtain the marked source sentence to be evaluated.

[0129] Furthermore, identify the zero pronoun markings in the marked source sentence, and determine the target pronoun position of the zero pronoun in the source sentence according to the zero pronoun markings.

[0130] 204. Determine the alignment information between the source sentence to be evaluated and the candidate translation sentence.

[0131] Among them, the alignment information can be the alignment relationship information between sentence components such as the subject, predicate, object, and modifier between two sentences.

[0132] In the embodiments of the present application, first determine the contextual association relationship in the source sentence to be evaluated and the contextual association relationship in the candidate translation sentence. The association relationship can be semantic information including intra-sentence semantic relationships, semantic dependencies, syntactic structure relationships, etc.; furthermore, according to the association relationship in the source sentence to be evaluated and the association relationship in the candidate translation sentence, perform alignment processing on the source sentence to be evaluated and the candidate translation sentence to obtain the alignment information between the source sentence to be evaluated and the candidate translation sentence.

[0133] 205. Determine the candidate translation pronoun in the candidate translation sentence that is aligned with the zero pronoun according to the alignment information and the target pronoun position.

[0134] Among them, the candidate translation pronoun can be a demonstrative pronoun that is aligned with the zero pronoun in the source sentence to be evaluated after translation; it should be noted that due to factors such as translation differences or alignment deviations, the candidate translation pronoun may not be equivalent to the aligned zero pronoun, that is, it is necessary to further confirm the equivalence of the candidate translation pronoun.

[0135] In the embodiment of the present application, the process of determining the candidate translation pronoun is a process of finding the word aligned with the zero pronoun from the candidate translation sentence, and specifically, the candidate translation pronoun can be determined according to the position information of the zero pronoun in the source sentence to be evaluated and the relevant alignment information. Specifically, the pronoun alignment information corresponding to the position of the target pronoun is extracted from the alignment information, so as to facilitate the subsequent targeted processing of the target word in the candidate translation sentence according to the pronoun alignment information. Furthermore, after obtaining the pronoun alignment information, the word aligned with the zero pronoun, that is, the candidate translation word, is found from the candidate translation sentence to obtain the search result. Finally, the candidate translation pronoun aligned with the zero pronoun can be determined according to the search result.

[0136] Among them, determining the candidate translation pronoun aligned with the zero pronoun according to the search result includes two cases. One is that a candidate translation word aligned with the zero pronoun is found, and the other is that no candidate translation word aligned with the zero pronoun is found. Specifically as follows:

[0137] On the one hand, when it is detected that the search result does not include the candidate translation word aligned with the zero pronoun, the position of the word adjacent to the position of the target pronoun in the source sentence to be evaluated is determined; the pronoun region adjacent to the zero pronoun in the source sentence to be evaluated is determined according to the position of the target pronoun and the word position; according to the alignment information, the corresponding translation word region of the pronoun region in the candidate translation sentence is determined; when it is recognized that the translation word region contains a translation word, the translation word contained in the translation word region can be determined as the candidate translation pronoun aligned with the zero pronoun. On the contrary, when it is recognized that the translation word region does not contain a translation word, it means that no candidate translation pronoun aligned with the zero pronoun is found in the candidate translation sentence, that is, the zero pronoun is not correctly translated into the candidate translation sentence.

[0138] On the other hand, when it is detected that the search result includes the candidate translation word aligned with the zero pronoun, the candidate translation word can be determined as the candidate translation pronoun, so as to facilitate the subsequent determination of whether it is equivalent to the zero pronoun, thereby evaluating the translation quality of the pronoun.

[0139] 206. Based on the candidate translation pronoun aligned with the zero pronoun, determine the correct translation pronoun equivalent to the zero pronoun.

[0140] In the embodiment of the present application, when determining the correct translation pronoun, the equivalence between the candidate translation pronoun and the zero pronoun can be determined according to a preset bilingual dictionary; among them, the preset bilingual dictionary can be a pre-defined bilingual dictionary, which can include information such as the semantics and referential relationships of bilingual pronouns.

[0141] Specifically, the candidate translation pronouns for zero pronoun alignment are matched in a preset bilingual dictionary to obtain a matching result, which may or may not include the preset pronouns corresponding to the candidate translation pronouns; furthermore, the matching result is identified. If it is identified that the matching result includes the preset pronouns matching the candidate translation pronouns, it means that the zero pronoun in the source sentence to be evaluated is correctly translated into the candidate translation sentence. At this time, the candidate translation pronoun can be determined as the correct translation pronoun equivalent to the zero pronoun; conversely, if it is identified that the matching result does not include the preset pronouns matching the candidate translation pronouns, it means that the zero pronoun in the source sentence to be evaluated is not correctly translated into the candidate translation sentence. Furthermore, it is convenient to evaluate the translation quality based on the correct translation pronouns subsequently.

[0142] 207. Obtain the number of correct translation pronouns in each candidate translation sentence.

[0143] Specifically, for each candidate translation sentence, count the number of zero pronouns in the corresponding source sentence that are correctly translated into this candidate translation sentence, that is, the number of correct translation pronouns.

[0144] 208. Based on the number of correct translation pronouns in each candidate translation sentence, determine the total number of correct translation pronouns corresponding to all candidate translation sentences.

[0145] In the embodiments of the present application, since the data set contains multiple groups of data, that is, multiple (source sentence, candidate translation sentence) combinations, therefore, when calculating the translation quality evaluation score of this data set subsequently, it is necessary to combine the number of correct translation pronouns in each candidate translation sentence to obtain the total number of correct translation pronouns corresponding to all candidate translation sentences.

[0146] Specifically, sum up the number of correct translation pronouns in all candidate translation sentences to obtain the total number of correct translation pronouns.

[0147] 209. Obtain the total number of zero pronouns corresponding to all source sentences to be evaluated, and calculate the ratio between the total number of correct translation pronouns and the total number of zero pronouns to obtain the quality evaluation score.

[0148] Among them, the quality evaluation score can be a score calculated according to the ratio of the number of correct translation pronouns to the total number of zero pronouns, and this quality evaluation score can reflect the situation where the zero pronouns in all source sentences in the data set are correctly translated.

[0149] In the embodiments of the present application, for multiple groups of data (source sentences, candidate translation sentences) in a dataset, the total number of zero pronouns in all source sentences to be evaluated in the dataset can be counted according to the number of zero pronouns included in the source sentences to be evaluated corresponding to the source sentences in each group of data. Furthermore, the total number of correctly translated pronouns is divided by the total number of zero pronouns to obtain a quality evaluation score. Through this quality evaluation score, the situation where the zero pronouns in the source sentences to be evaluated are correctly translated can be accurately reflected, effectively avoiding the inconsistency between the semantics and syntactic structures of the candidate translation sentences and the source text semantics due to the omission of pronouns, and improving the accuracy in evaluating the translation quality of candidate translation sentences.

[0150] For the convenience of understanding the embodiments of the present application, the embodiments of the present application will be described with specific application scenario examples. Specifically, by executing the above steps 201-209, and in combination with Figure 3 、 Figure 4 、and Figure 5 , the application scenario example will be described.

[0151] It should be noted that in the Chinese language, the omission of pronouns is a very common phenomenon. On the contrary, in languages such as English and French, pronouns are essential as sentence components. For example, as Figure 4 shown, the nominative pronouns "you", "I" and the accusative pronoun "it" in the Chinese brackets are omitted, but the corresponding pronouns on the English side cannot be omitted. Therefore, taking the translation quality evaluation scenario from Chinese (Mandarin) to English as an example in this application scenario, it directly targets the pronoun translation situation on the translation side to ensure that the translation is not affected by the zero anaphora (pronoun omission) on the source side (source sentence) and is correctly translated.

[0152] The anaphoric relationship is an important means to ensure the coherence of a text, and pronouns imply rich text information (such as the anaphoric relationship with the antecedent). Therefore, zero pronoun modeling plays a very important role in the model's understanding of language. For machine translation, the omission of pronouns on the source side will lead to incomplete translations, chaotic syntactic structures, and semantic errors, thus seriously affecting the translation quality and the coherence degree of the text. For example, as shown in Table 1 below, the neural machine translation (NMT) system not only fails to recall the omitted pronouns but also translates a translation with a large semantic difference from the original text. Therefore, when translating from a pronoun-omitting language (such as Chinese) to a non-pronoun-omitting language (such as English), it is necessary to first recall the implicit omitted pronouns and then translate them correctly.

[0153]

Table 1

[0154]

[0155]

[0156] Among them, the Chinese (Mandarin)-English translation quality assessment scenario is as follows: a ZPT algorithm is proposed, and source sentences with zero pronoun (ZP) markings, candidate translation results, and word-level alignment information between them are prepared in advance. The number of zero pronouns correctly translated from the source side to the target side is calculated by aZPT, and the rationality of the position is considered.

[0157] Among them, the calculation of the aZPT score is as shown in (1) below:

[0158]

[0159] where i is the position of each word in the first input sequence of n, is the zero pronoun ZP with a marker, represents the zero pronoun being checked and the weight of. Among them, the weight can be calculated through the alignment matrix as follows:

[0160]

[0161] where t i represents the translation corresponding to the zero pronoun .

[0162] Regarding this Chinese (Mandarin)-English translation quality assessment scenario, specifically, the source input (source sentence to be evaluated) with ZP markings and the candidate translation sentence (i.e., NMT output) are automatically aligned through GIZA++. To ensure accuracy, a large parallel corpus is used, and the growth-diag-final heuristic from Moses is employed. For example, if the ZP in the source input is aligned with any target word in the candidate translation sentence, it is checked whether the ZP and the target word are equal to each other; otherwise, the corresponding alignment between the words before and after the ZP (position +-1) in the source sentence to be evaluated and the candidate translation sentence is used to define the neighbor range of the zero pronoun in the candidate translation sentence. Furthermore, it is detected whether this range contains the equivalent source ZP. It should be noted that the equivalence is determined by a pre-defined bilingual pronoun dictionary. Further, after traversing all the sentences related to zero pronouns in the test set, the score is calculated as the number of correctly translated zero pronouns divided by the total number of zero pronouns ZP to obtain a quality assessment score for the translation of omitted pronouns, which is used for the translation quality assessment of the translation.

[0163] See Figure 5 , which is a schematic diagram of the scenario of the translation quality assessment method provided in the embodiments of this application. It should be noted that since this translation quality assessment method is mainly implemented by the aZPT algorithm in the Chinese (Mandarin)-English translation quality assessment scenario, for ease of understanding, it can be combined with Figure 5For a further understanding of this application scenario example. Specifically, let S be the source input with ZP markings (i.e., the source sentence to be evaluated), denote the preset pronoun corresponding to the zero pronoun (ZP) in the preset pronoun bilingual dictionary D, t denote the number of correct cases (i.e., the number of correctly translated pronouns), and n denote the number of test cases (i.e., the number of zero pronouns ZP); then the specific process of this algorithm is as follows:

[0164] (1) The aZPT algorithm reads in the source dataset (including multiple source sentences to be evaluated) and the translation results (the candidate translation sentences corresponding to each source sentence to be evaluated), and outputs the final quality assessment score of the source dataset.

[0165] (1.1) Regarding each source sentence to be evaluated and its corresponding candidate translation sentence as a detection internal round, for each internal round, specifically: for each zero pronoun (ZP) in each source sentence to be evaluated, determine the translated sentence region in the candidate translation sentence that is aligned with it.

[0166] (1.2) Judge whether the translated sentence region is non-empty. If it is non-empty (i.e., it contains the translated sentence region), then identify whether the candidate translated pronoun w in this translated sentence region contains the preset pronoun in the preset pronoun bilingual dictionary If it contains then t + 1.

[0167] (1.3) For the number of test cases n = n + 1.

[0168] (2) After completing the process traversal of all internal rounds (1.1)-(1.3) in the source dataset, the final number of correct cases t and the number of test cases n are obtained. At this time, the translation quality assessment score O = t / n.

[0169] Through the above scenario example, the following effects can be achieved: on the one hand, consider the presence or absence of the translation of zero pronouns (the omission at the source end may lead to the omission at the target end), and on the other hand, consider the correctness of its translation (including position and translation) to indirectly represent the evaluation of its semantics and structure; furthermore, it is possible to more specifically evaluate the attention and learning of the NMT system to the omitted pronouns, so that the subsequent translations are more complete, conform to the grammatical structure of the target language, and follow the original semantics, thus bringing an overall improvement in the translation quality and further improving the translation system effect.

[0170] As can be seen from the above, the embodiments of the present application can pre-mark zero pronouns in the source sentence to be translated, so as to determine the alignment information between the candidate translation sentence and the marked source sentence to be evaluated. According to the alignment information, the zero pronouns in the source sentence to be evaluated that are correctly translated into the candidate translation sentence are determined, so as to determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence. Furthermore, according to the total number of zero pronouns corresponding to the source sentence to be evaluated and the number of correct translation pronouns, the quality evaluation score of the candidate translation sentence is calculated. In this way, after obtaining the candidate translation sentence, the translation quality of the candidate translation sentence can be evaluated according to the number of correct translations of zero pronouns, so as to evaluate whether the translated text is consistent with the semantics and syntactic structure of the source text, and improve the accuracy of translation quality evaluation.

[0171] To better implement the above method, the embodiments of the present application also provide a translation quality evaluation device, and this translation quality evaluation device can be integrated into a server.

[0172] For example, as Figure 6 shown, this translation quality evaluation device may include an acquisition unit 401, a marking unit 402, a first determination unit 403, a second determination unit 404, and a calculation unit 405.

[0173] The acquisition unit 401 is used to acquire the source sentence and the corresponding candidate translation sentence;

[0174] The marking unit 402 is used to mark the hidden zero pronouns in the source sentence and identify the target pronoun positions of the zero pronouns in the marked source sentence;

[0175] The first determination unit 403 is used to determine the alignment information between the source sentence to be evaluated and the candidate translation sentence;

[0176] The second determination unit 404 is used to determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence according to the alignment information and the target pronoun positions;

[0177] The calculation unit 405 is used to obtain the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculate the quality evaluation score corresponding to the candidate translation sentence according to the number of correct translation pronouns and the total number of zero pronouns.

[0178] In some embodiments, the second determination unit 404 is further used to: determine the candidate translation pronouns aligned with each zero pronoun in the candidate translation sentence according to the alignment information and the target pronoun positions; determine the correct translation pronouns equivalent to the zero pronouns based on the candidate translation pronouns aligned with each zero pronoun; and obtain the number of correct translation pronouns.

[0179] In some embodiments, the second determination unit 404 is further configured to: determine the target pronoun position of the zero pronoun in the source sentence to be evaluated according to the zero pronoun marker; find the candidate translation words aligned with each zero pronoun in the candidate translation sentence according to the alignment information and the target pronoun position, so as to obtain a search result; and determine the candidate translation pronouns aligned with each zero pronoun based on the search result.

[0180] In some embodiments, the second determination unit 404 is further configured to: if it is detected that the search result does not include candidate translation words aligned with the zero pronoun, select the corresponding translation word region associated with the zero pronoun in the candidate translation sentence according to the alignment information and the target pronoun position; and if it is recognized that the translation word region includes a translation word, determine the translation word as the candidate translation pronoun aligned with the zero pronoun.

[0181] In some embodiments, the second determination unit 404 is further configured to: determine the word position adjacent to the target pronoun position in the source sentence to be evaluated; determine the pronoun region adjacent to the zero pronoun in the source sentence to be evaluated according to the target pronoun position and the word position; and determine the corresponding translation word region of the pronoun region in the candidate translation sentence according to the alignment information.

[0182] In some embodiments, the second determination unit 404 is further configured to: match the candidate translation pronouns aligned with each zero pronoun from a preset bilingual dictionary to obtain a matching result; and if the matching result includes a preset pronoun matching the candidate translation pronoun, determine the candidate translation pronoun as the correct translation pronoun equivalent to the zero pronoun.

[0183] In some embodiments, the second determination unit 404 is further configured to: if it is detected that the search result includes candidate translation words aligned with the zero pronoun, determine the candidate translation words as the candidate translation pronouns aligned with the zero pronoun.

[0184] In some embodiments, the acquisition unit 401 is further configured to: acquire the source text to be translated, and extract a plurality of source sentences from the source text to be translated; and perform translation processing on the source text to be translated through a machine translation model to obtain a candidate translation sentence corresponding to each source sentence.

[0185] As can be seen from the above, in the embodiment of the present application, the acquisition unit 401 can acquire the source sentence and the corresponding candidate translation sentence; the marking unit 402 marks the zero pronouns hidden in the source sentence and identifies the target pronoun positions of the zero pronouns in the marked source sentence; the first determination unit 403 determines the alignment information between the source sentence to be evaluated and the candidate translation sentence; the second determination unit 404 determines the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence according to the alignment information and the target pronoun positions; the calculation unit 405 acquires the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculates the quality evaluation score corresponding to the candidate translation sentence according to the number of correct translation pronouns and the total number of zero pronouns. Thus, in this solution, the zero pronouns in the source sentence to be translated can be marked in advance to determine the alignment information between the candidate translation sentence and the marked source sentence to be evaluated, determine the zero pronouns correctly translated from the source sentence to be evaluated into the candidate translation sentence according to the alignment information, so as to determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence. Furthermore, the quality evaluation score of the candidate translation sentence is calculated according to the total number of zero pronouns corresponding to the source sentence to be evaluated and the number of correct translation pronouns. In this way, after obtaining the candidate translation sentence, the translation quality of the candidate translation sentence can be evaluated according to the number of correct translations of the zero pronouns, so as to evaluate whether the translated text is consistent with the source text in semantics and syntactic structure, and improve the accuracy of translation quality evaluation.

[0186] The embodiment of the present application also provides a computer device, as Figure 7 shown, which shows a schematic structural diagram of the computer device involved in the embodiment of the present application. Specifically:

[0187] The computer device may include a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a power supply 503, an input unit 504 and other components. Those skilled in the art can understand that Figure 7 the computer device structure shown in

[0188] The processor 501 is the control center of the computer device, connecting various parts of the entire computer device through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 502, and by invoking the data stored in the memory 502, it executes various functions of the computer device and processes data. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 501 either.

[0189] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and translation quality evaluations by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.); the data storage area can store the data created according to the use of the computer device. In addition, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0190] The computer device also includes a power supply 503 that powers each component. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 503 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0191] The computer device may also include an input unit 504, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0192] Although not shown, the computer device may also include a display unit, etc., which will not be elaborated here. Specifically, in the embodiment of this application, the processor 501 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 502 according to the following instructions, and the processor 501 will run the application programs stored in the memory 502 to realize various functions as follows:

[0193] Obtain the source sentence and its corresponding candidate translation sentence; mark the hidden zero pronouns in the source sentence, and identify the target pronoun positions of the zero pronouns in the marked source sentence; determine the alignment information between the source sentence to be evaluated and the candidate translation sentence; according to the alignment information and the target pronoun positions, determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence; obtain the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculate the quality evaluation score corresponding to the candidate translation sentence based on the number of correct translation pronouns and the total number of zero pronouns.

[0194] For the specific implementation of each of the above operations, reference may be made to the previous embodiments, which will not be elaborated here.

[0195] Thus, it can be obtained that this solution can pre-mark the zero pronouns in the source sentence to be translated to determine the alignment information between the candidate translation sentence and the marked source sentence to be evaluated, determine the zero pronouns in the source sentence to be evaluated that are correctly translated into the candidate translation sentence according to the alignment information to determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence. Furthermore, calculate the quality evaluation score corresponding to the candidate translation sentence based on the total number of zero pronouns corresponding to the source sentence to be evaluated and the number of correct translation pronouns; in this way, after obtaining the candidate translation sentence, it is possible to evaluate the translation quality of the candidate translation sentence according to the number of correct translations of zero pronouns, so as to evaluate whether the translated text is consistent with the source text in semantics and syntactic structure, and improve the accuracy of translation quality evaluation.

[0196] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling relevant hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0197] Therefore, an embodiment of the present application provides a computer-readable storage medium, in which multiple instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the translation quality evaluation methods provided by the embodiments of the present application. For example, the instructions can execute the following steps:

[0198] Obtain the source sentence and its corresponding candidate translation sentence; mark the hidden zero pronouns in the source sentence, and identify the target pronoun positions of the zero pronouns in the marked source sentence; determine the alignment information between the source sentence to be evaluated and the candidate translation sentence; according to the alignment information and the target pronoun positions, determine the number of correct translation pronouns corresponding to the zero pronouns in the candidate translation sentence; obtain the total number of zero pronouns corresponding to the source sentence to be evaluated, and calculate the quality evaluation score corresponding to the candidate translation sentence based on the number of correct translation pronouns and the total number of zero pronouns.

[0199] For the specific implementation of each of the above operations, reference may be made to the previous embodiments, which will not be elaborated here again.

[0200] Among them, the computer-readable storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

[0201] Since the instructions stored in the computer-readable storage medium can execute the steps in any of the translation quality assessment methods provided in the embodiments of the present application, the beneficial effects achievable by any of the translation quality assessment methods provided in the embodiments of the present application can be realized. For details, refer to the previous embodiments and will not be elaborated here.

[0202] The above has introduced in detail a translation quality assessment method, device, equipment and computer storage medium provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for evaluating translation quality, characterized in that, Comprising: Obtaining a source sentence and a corresponding candidate translation sentence; Marking zero pronouns hidden in the source sentence, and identifying the target pronoun positions of the zero pronouns in the source sentence to be evaluated after marking; Determining alignment information between the source sentence to be evaluated and the candidate translation sentence; Determining the number of correct translation pronouns corresponding to the zero pronoun in the candidate translation sentence according to the alignment information and the target pronoun positions; wherein, it includes: determining candidate translation pronouns corresponding to the zero pronoun in the candidate translation sentence according to the alignment information and the target pronoun positions; determining correct translation pronouns equivalent to the zero pronoun based on the candidate translation pronouns aligned with the zero pronoun; obtaining the number of the correct translation pronouns; The determining the candidate translation pronouns corresponding to the zero pronoun in the candidate translation sentence according to the alignment information and the target pronoun positions includes: Extracting the pronoun alignment information corresponding to the target pronoun positions from the alignment information; Searching for candidate translation words aligned with the zero pronoun in the candidate translation sentence according to the pronoun alignment information to obtain a search result; Determining the candidate translation pronouns aligned with the zero pronoun based on the search result; wherein, it includes: if it is detected that the search result does not include candidate translation words aligned with the zero pronoun, then according to the alignment information and the target pronoun positions, selecting a corresponding translation word region associated with the zero pronoun in the candidate translation sentence, where the translation word region is: the region aligned with the zero pronoun in the source sentence to be evaluated and other words adjacent to it in the candidate translation sentence; if it is recognized that the translation word region includes translation words, then determining the translation words as the candidate translation pronouns aligned with the zero pronoun; Obtaining the total number of pronouns corresponding to the source sentence to be evaluated, and calculating a quality evaluation score corresponding to the candidate translation sentence according to the number of the correct translation pronouns and the total number of pronouns; 2. The method according to claim 1, wherein The selecting the corresponding translation word region associated with the zero pronoun in the candidate translation sentence according to the alignment information and the target pronoun positions includes: Determining the word positions adjacent to the target pronoun positions in the source sentence to be evaluated; Determining the pronoun region adjacent to the zero pronoun in the source sentence to be evaluated according to the target pronoun positions and the word positions; Determining the corresponding translation word region of the pronoun region in the candidate translation sentence according to the alignment information; 3. The method according to claim 1 or 2, characterized in that, The determining the correct translation pronouns equivalent to the zero pronoun based on the candidate translation pronouns aligned with the zero pronoun includes: Matching the candidate translation pronouns aligned with the zero pronoun from a preset bilingual dictionary to obtain a matching result; If the matching result includes a preset pronoun matching the candidate translation pronoun, then determining the candidate translation pronoun as the correct translation pronoun equivalent to the zero pronoun; 4. The method according to claim 1, characterized in that, Also comprising: If it is detected that the search result includes candidate translation words aligned with the zero pronoun, then determining the candidate translation words as the candidate translation pronouns aligned with the zero pronoun; 5. The method according to claim 1, wherein The obtaining the source sentence and the corresponding candidate translation sentence includes: Obtain the source text to be translated, and extract multiple source sentences from the source text to be translated; Perform translation processing on the multiple source sentences through a machine translation model to obtain a candidate translation sentence corresponding to each source sentence.

6. A translation quality evaluation device, characterized in that, It includes: An obtaining unit, configured to obtain a source sentence and a corresponding candidate translation sentence; A marking unit, configured to mark the null pronouns hidden in the source sentence, and identify the target pronoun positions of the null pronouns in the source sentence to be evaluated after marking; A first determination unit, configured to determine the alignment information between the source sentence to be evaluated and the candidate translation sentence; A second determination unit, configured to determine the number of correct translation pronouns corresponding to the null pronoun in the candidate translation sentence according to the alignment information and the target pronoun position; wherein, it includes: determining the candidate translation pronouns aligned with the null pronoun in the candidate translation sentence according to the alignment information and the target pronoun position; based on the candidate translation pronouns aligned with the null pronoun, determining the correct translation pronouns equivalent to the null pronoun; obtaining the number of the correct translation pronouns; The determining the candidate translation pronouns aligned with the null pronoun in the candidate translation sentence according to the alignment information and the target pronoun position includes: Extracting the pronoun alignment information corresponding to the target pronoun position from the alignment information; According to the pronoun alignment information, searching for the candidate translation words aligned with the null pronoun in the candidate translation sentence to obtain a search result; Based on the search result, determining the candidate translation pronouns aligned with the null pronoun; wherein, it includes: if it is detected that the search result does not include the candidate translation words aligned with the null pronoun, then according to the alignment information and the target pronoun position, selecting the corresponding translation word region associated with the null pronoun in the candidate translation sentence, where the translation word region is: the null pronoun in the source sentence to be evaluated and other words adjacent to it, the region aligned in the candidate translation sentence; if it is recognized that the translation word region includes translation words, then determining the translation words as the candidate translation pronouns aligned with the null pronoun; A calculation unit, configured to obtain the total number of null pronouns corresponding to the source sentence to be evaluated, and calculate the quality evaluation score corresponding to the candidate translation sentence according to the number of correct translation pronouns and the total number of null pronouns.

7. A computer device, characterized in that, It includes a processor and a memory, the memory stores a computer program, and the processor is configured to run the computer program in the memory to implement the steps in the translation quality evaluation method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores multiple instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the translation quality evaluation method according to any one of claims 1 to 5.

9. A computer program product, characterized in that, It includes computer instructions, and when the computer instructions are executed, the steps in the translation quality evaluation method according to any one of claims 1 to 5 are implemented.

Images