Data processing method, data processing device, and computer storage medium

Abstract

Embodiments of the present specification provide a data processing method, a data processing device and a computer storage medium, wherein the data processing method comprises: obtaining an original text, a translation evaluation label sequence corresponding to the original text and a target text corresponding to the original text; based on the original text, a reference translation set same as the language category of the target text is obtained, the reference translation set comprising: a reference translation text; based on the original text, the reference translation set and the target text, an incorrect evaluation label in the translation evaluation label sequence is corrected. By using the above scheme, the accuracy of the translation evaluation result can be improved without changing the translation evaluation method.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of data processing, and particularly relate to a data processing method, a data processing device and a computer storage medium. BACKGROUND

[0002] With the progress of computer technology, text translation can translate original text of a language category into target text of a target language category through artificial translation, machine translation (MT) or a combination of both. Regardless of which way is used for translation, the accuracy of the target text is particularly important, and therefore translation needs to be evaluated.

[0003] Currently, translation evaluation mainly uses artificial evaluation or translation evaluation model, combines the original text and the target text, judges the correct and / or incorrect translation, and labels evaluation tags that can represent the judgment results at the corresponding positions of the original text or the target text, so as to reflect the translation evaluation results through the translation evaluation tag sequence.

[0004] However, the artificial evaluation method is too inefficient to meet the increasing demand for translation evaluation, and the artificial evaluation results vary from person to person, which may result in translation evaluation errors. Through the translation evaluation model, a certain amount of time needs to be spent in advance for model training to optimize the model parameters, and the performance of the translation evaluation model depends largely on the training method, which may also result in translation evaluation errors.

[0005] Therefore, whether it is artificial evaluation or translation evaluation model, there are translation evaluation errors, and how to improve the accuracy of the translation evaluation results is a problem that needs to be solved by those skilled in the art. SUMMARY

[0006] Therefore, the embodiments of the present specification provide a data processing method, a data processing device and a computer storage medium, which can improve the accuracy of the translation evaluation results without changing the translation evaluation method.

[0007] The embodiments of the present specification provide a data processing method applied to a data processing device, comprising:

[0008] obtaining an original text, a translation evaluation tag sequence corresponding to the original text and a target text corresponding to the original text, wherein the translation evaluation tag sequence comprises evaluation tags suitable for representing correct translation or incorrect translation;

[0009] based on the original text, obtaining a reference translation set with the same language category as the target text, the reference translation set comprising reference translation texts;

[0010] correct the error evaluation labels in the translation evaluation label sequence based on the original text, the reference translation set and the target text.

[0011] Optionally, the correcting the error evaluation labels in the translation evaluation label sequence based on the original text, the reference translation set and the target text comprises:

[0012] aligning the original text and the reference translation text to obtain first correspondence information, wherein the first correspondence information is adapted to represent the correspondence between each language element in the original text and each language element in the reference translation text;

[0013] aligning the original text and the target text to obtain second correspondence information, wherein the second correspondence information is adapted to represent the correspondence between each language element in the original text and each language element in the target text;

[0014] correcting the error evaluation labels in the translation evaluation label sequence based on the first correspondence information and the second correspondence information.

[0015] Optionally, the correcting the error evaluation labels in the translation evaluation label sequence based on the first correspondence information and the second correspondence information comprises:

[0016] based on the first correspondence information and the second correspondence information, when the language element in the original text has any of the following conditions, the corresponding evaluation label is corrected:

[0017] the language element is determined to be omitted, and the corresponding evaluation label represents a translation error;

[0018] the corresponding language element in the reference translation text and the corresponding language element in the target text are the same language element, and the corresponding evaluation label represents a translation error;

[0019] the corresponding language element in the reference translation text and the corresponding language element in the target text are different language elements, and the corresponding evaluation label represents a correct translation.

[0020] Optionally, the determination method of the omitted language element in the original text comprises:

[0021] based on the first correspondence information, determining the language element in the original text that has no correspondence with the reference translation text.

[0022] Optionally, the obtaining, based on the original text, a reference translation set same in language category as the target text comprises at least one of the following:

[0023] translating the original text based on the language category of the target text to obtain the reference translation text;

[0024] translating the original text based on the language category of the target text and performing synonym expansion of the same language category on the translation result to obtain the reference translation text.

[0025] Optionally, the translating the original text based on the language category of the target text and performing synonym expansion of the same language category on the translation result to obtain the reference translation text comprises:

[0026] translating the original text based on the language category of the target text to obtain a plurality of candidate translation texts;

[0027] performing synonym expansion of the same language category on each of the candidate translation texts to obtain a plurality of synonym translation texts;

[0028] performing deduplication processing based on the plurality of candidate translation texts and the plurality of synonym translation texts to determine the reference translation text.

[0029] Optionally, the data processing method further comprises:

[0030] outputting the corrected translation evaluation label sequence.

[0031] An embodiment of the present specification provides a data processing device, comprising:

[0032] a data acquisition module adapted to acquire an original text, a translation evaluation label sequence corresponding to the original text, and a target text corresponding to the original text, and obtain, based on the original text, a reference translation set same in language category as the target text; wherein the translation evaluation label sequence comprises an evaluation label adapted to represent correct translation or incorrect translation; and the reference translation set comprises a reference translation text;

[0033] a label correction module adapted to correct an incorrect evaluation label in the translation evaluation label sequence based on the original text, the reference translation set, and the target text.

[0034] An embodiment of the present specification provides a data processing device comprising a memory and a processor; wherein the memory is adapted to store one or more computer instructions, and the processor executes the computer instructions to perform the steps of the data processing method of any one of the above embodiments.

[0035] The embodiment of the present specification provides a computer readable storage medium, which stores computer instructions, and the computer instructions run the steps of the data processing method of any one of the above embodiments.

[0036] After obtaining the original text, the translation evaluation label sequence corresponding to the original text, and the target text corresponding to the original text, the data processing scheme provided by the embodiment of the present specification can be used to obtain a reference translation set with the same language category as the target text based on the original text, and then correct the incorrect evaluation labels in the translation evaluation label sequence based on the original text, the reference translation set, and the target text. As can be seen from the above, the reference translation set is determined according to the original text, and the reference translation texts contained in the reference translation set can change with the change of the original text, thereby improving the flexibility and reliability of the reference translation set. Moreover, the evaluation labels can be comprehensively evaluated and corrected based on the semantic correlation between the original text, the target text, and the reference translation set, thereby improving the accuracy of the translation evaluation result without changing the translation evaluation method. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following will briefly introduce the drawings needed to be used in the embodiments of the present specification or the prior art description. Obviously, the drawings described below are only some embodiments of the present specification, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0038] Figure 1 A translation evaluation schematic diagram for an English-French translation application scenario.

[0039] Figure 2 A flowchart of a data processing method provided by the embodiment of the present specification.

[0040] Figure 3 A flowchart of an evaluation label correction method provided by the embodiment of the present specification.

[0041] Figure 4 A structural block diagram of a data processing device provided by the embodiment of the present specification.

[0042] Figure 5 A structural block diagram of another data processing device provided by the embodiment of the present specification. DETAILED DESCRIPTION

[0043] As described in the background, for the existing translation evaluation method, whether it is a manual evaluation or a translation evaluation model method, there is a situation of translation evaluation error.

[0044] In order for those skilled in the art to clearly understand the problems existing in the current translation evaluation method, the following will be illustrated through specific application scenarios.

[0045] In an actual application, as shown in FIG. 1, it is a translation evaluation diagram of an English-French translation application scenario. In this example, the original text 1a in English is translated into the target text 1b in French. Figure 1

[0046] Combined with the original text 1a and the target text 1b, the translation of the original text 1a can be evaluated to determine whether each word in the original text 1a is correctly translated in the target text 1b, and according to the determination result, the evaluation label "OK" or the evaluation label "BAD" is marked at the corresponding position of each word in the original text 1a, to obtain the translation evaluation label sequence 1c of the original text 1a.

[0047] Combined with the original text 1a and the target text 1b, the translation of the original text 1a can be evaluated to determine whether each word in the original text 1a is correctly translated in the target text 1b, and according to the determination result, the evaluation label "OK" or the evaluation label "BAD" is marked at the corresponding position of each word in the original text 1a, to obtain the translation evaluation label sequence 1c of the original text 1a.

[0048] Among them, the evaluation label "OK" represents correct translation, and the evaluation label "BAD" represents translation error, and according to the corresponding relationship between the evaluation label "BAD" and the word, the evaluation label "BAD" can be further represented as a translation error of using a wrong word (such as the evaluation label "BAD" corresponding to the word "tool" in the translation evaluation label sequence 1c, the evaluation label "BAD" corresponding to the word "Der" in the translation evaluation label sequence 1d, etc.), or a translation error of missing translation (such as the evaluation label "BAD" before the word "Der" in the translation evaluation label sequence 1d).

[0049] However, when the original text 1a is evaluated, the word "sharpens" in the original text 1a is misjudged as being correctly translated in the target text 1b, so that the word "sharpens" in the original text 1a is incorrectly labeled with the evaluation label "OK", resulting in an incorrect evaluation label in the translation evaluation label sequence 1c.

[0050] ​Similarly, in the translation evaluation of the target text 1b, the word "areas" in the original text 1a is misjudged as being incorrectly translated in the target text 1b, so that the word "Bereiche" in the target text 1b is incorrectly labeled with the evaluation label "BAD", resulting in an incorrect evaluation label in the translation evaluation label sequence 1d.

[0051] To solve the above problems, the present specification provides a data processing scheme. After obtaining an original text, a translation evaluation label sequence corresponding to the original text, and a target text corresponding to the original text, a corresponding reference translation set can be obtained based on the original text. Then, the incorrect evaluation labels in the translation evaluation label sequence can be corrected based on the original text, the reference translation set, and the target text. In this way, the accuracy of the translation evaluation result can be improved without changing the translation evaluation method.

[0052] To make those skilled in the art more clearly understand and implement the concept, implementation scheme and advantages of the present specification, the following will be described by specific embodiments with reference to the accompanying drawings.

[0053] Reference Figure 2 A flowchart of a data processing method provided by the embodiments of the present specification is shown in the following. In the embodiments of the present specification, the data processing method can be applied to a data processing device, which can specifically include the following steps:

[0054] S11, obtaining an original text, a translation evaluation label sequence corresponding to the original text, and a target text corresponding to the original text.

[0055] The translation evaluation label sequence can include an evaluation label corresponding to the original text, which is suitable for indicating correct translation or incorrect translation.

[0056] It should be noted that the "translation error" described in the embodiments of the present specification can include at least one of the existence of a wrong character and the omission of translation, and the specific meaning of "translation error" can be determined according to the application scenario and the translation evaluation requirement, which is not limited in the present specification.

[0057] It can be understood that the target text can be obtained by at least one of manual translation and machine translation, and the present specification does not make a specific limitation on the translation method.

[0058] In specific implementation, the translation evaluation label sequence can be obtained by manual evaluation based on the original text and the target text, or by automatic evaluation through a translation evaluation model by inputting the original text and the target text into the translation evaluation model. The embodiments of the present specification do not make a specific limitation on the obtaining method of the translation evaluation label sequence.

[0059] The translation evaluation model can be any neural network model with translation evaluation function, for example, BERT (Bidirectional Encoder Representations from Transformers), XLM (Cross-lingual Language Model), XLM-R (a language model combining XLM and RoBERTa), etc.

[0060] S12, based on the original text, obtaining a reference translation set with the same language category as the target text.

[0061] The reference translation set can include one or more reference translation texts. The number of reference translation texts can be set according to specific application scenarios and requirements, and the present specification does not make specific limitations thereon.

[0062] In specific implementation, the language category of the reference translation text is the same as that of the target text. For example, the language category of the target text is English, and the language category of the reference translation text is also English.

[0063] In specific implementation, the way of obtaining the reference translation set can be determined according to specific application scenarios and requirements. For example, the original text can be translated based on the language category of the target text, and the translated text can be used as the reference translation text, thereby obtaining the reference translation set.

[0064] The way of obtaining the reference translation text is different from the way of obtaining the target text. For example, the target text is obtained by manually translating the original text, while the reference translation text is obtained by machine translation of the original text; for another example, the target text is obtained by inputting the original text into a machine translation system for machine translation, while the reference translation text is obtained by inputting the original text into one or more other machine translation systems for machine translation.

[0065] S13, based on the original text, the reference translation set and the target text, correcting the error evaluation label in the translation evaluation label sequence.

[0066] Specifically, since the original text and the target text are in a translation relationship with each other, the original text and the target text convey the same meaning through different languages and expressions, and thus the original text and the target text are semantically related. Similarly, the original text and the reference translation set are in a translation relationship with each other, the original text and the reference translation set also convey the same meaning through different languages and expressions, and thus the original text and the reference translation set are also semantically related.

[0067] Based on this, the original text, the reference translation set, and the target text can be matched based on the original text, so that it can be evaluated whether the evaluation labels in the translation evaluation label sequence are correct, and the incorrect evaluation labels can be corrected, so that each evaluation label in the translation evaluation label sequence is a correct evaluation label.

[0068] As can be seen from the above, the reference translation set is determined according to the original text, and the reference translation texts contained in the reference translation set can change with the original text, improving the flexibility and reliability of the reference translation set. Moreover, the evaluation labels can be comprehensively evaluated and the incorrect evaluation labels can be corrected from the semantic correlation between the original text, the target text, and the reference translation set, so that the accuracy of the translation evaluation result can be improved without changing the translation evaluation method.

[0069] In addition, the data processing method provided by the embodiments of the present specification is not related to which translation evaluation method is used, and thus the data processing method provided by the embodiments of the present specification can be applied to manual evaluation and translation evaluation models of different architectures, has strong universality, and has a wide range of applications.

[0070] In specific implementation, alignment can be performed between the original text, the reference translation set, and the target text, so as to obtain a corresponding relationship at the language element level, so as to facilitate matching and label correction among the three.

[0071] It should be noted that in the present specification, the name "language element" is used to refer to the smallest disassemblable part of a text. According to the language type and requirements, the smallest disassemblable part of the text can change. For example, for Chinese, the smallest disassemblable part can be a character and a Chinese punctuation mark, or a character, a word, and a Chinese punctuation mark, or a sentence and a Chinese punctuation mark, etc. For English, the smallest disassemblable part can be a word and an English punctuation mark, or a word, a phrase, and an English punctuation mark, or a sentence and an English punctuation mark, etc. In actual application, other names can also be used to refer to the smallest disassemblable part of the text. The present specification does not make specific limitations in this regard.

[0072] In a specific implementation, as shown in Figure 3 FIG. 1 is a flowchart of a method for correcting a test label according to an embodiment of the present specification. The method can include the following steps:

[0073] S21, aligning the original text and the reference translation text to obtain first correspondence information.

[0074] The first correspondence information is adapted to represent the correspondence between each language element in the original text and each language element in the reference translation text.

[0075] It can be understood that the first correspondence information can be obtained by a software program with a multilingual text alignment function (such as an alignment tool, an alignment model, or an alignment algorithm, etc.), and the present specification does not make specific limitations thereto.

[0076] In an optional example, the original text is {I like cats.}, and the reference translation text is {I like cats.}. Aligning the original text {I like cats.} and the reference translation text {I like cats.} can obtain the first correspondence information as follows: [I-I, like-like, cats-cats,.-.]. Wherein, “I”, “like”, “cats”, “.” are language elements of the original text, and “I”, “like”, “cats”, “.” are language elements of the reference translation text; “I-I” means that the language element “I” of the original text corresponds to the language element “I” of the reference translation text, and so on, which will not be repeated.

[0077] It should be noted that the curly braces and square brackets in the present specification are only used to limit the content range of the example text, and are not essential parts of the text content. Those skilled in the art can use other symbols that are not easily confused to limit the content range of the text. The curly braces and square brackets used in the following text are the same as described above.

[0078] It should also be noted that the above-mentioned representation of the first correspondence information is only used for illustrative purposes. In actual applications, the first correspondence information can be represented by other formats or reference symbols. For example, the position sequence number of each element in the corresponding text can be used to refer to the element, thereby obtaining the first correspondence information based on digital representation. The present specification does not make specific limitations on the representation of the first correspondence information.

[0079] It can be understood that, in order to facilitate description and understanding, only the case of one reference translation text is shown in the above examples, but in actual application, the reference translation set can include multiple reference translation texts, so that multiple reference translation texts are respectively aligned with the original text to obtain multiple first corresponding relationship information.

[0080] S22, aligning the original text and the target text to obtain second corresponding relationship information.

[0081] The second corresponding relationship information is adapted to represent the corresponding relationship between each language element in the original text and each language element in the target text.

[0082] It can be understood that the second corresponding relationship information can be obtained by a software program (such as an alignment tool, an alignment model, or an alignment algorithm) with a multilingual text alignment function, and the embodiments of the present specification do not make specific limitations thereto.

[0083] In an optional example, the original text is {I like cats.} and the target text is {I cats.}. Aligning the original text {I like cats.} and the target text {I cats.} can obtain the second corresponding relationship information as follows: [I-I, like-, cats-cats,.-.] wherein, “I”, “like”, “cats”, “.” are language elements of the original text, and “I”, “cats”, “.” are language elements of the target text; “I-I” means that the language element “I” of the original text corresponds to the language element “I” of the target text, and so on, which will not be repeated; and it can be obtained from the second corresponding relationship information that there is no language element in the target text corresponding to the language element “like” of the original text.

[0084] It should be noted that the representation of the above second corresponding relationship information is only for illustrative purposes, and in actual application, the second corresponding relationship information can be represented by other formats or symbols, for example, each element can be represented by the position sequence number of the element in the corresponding text, so as to obtain the second corresponding relationship information based on digital representation. The present specification does not make specific limitations on the representation of the second corresponding relationship information.

[0085] S23, correcting the error evaluation label in the translation evaluation label sequence based on the first corresponding relationship information and the second corresponding relationship information.

[0086] Specifically, since the first correspondence information can represent the correspondence between each language element in the original text and each language element in the reference translation text, and the second correspondence information can represent the correspondence between each language element in the original text and each language element in the target text, the language elements in the original text can be taken as a reference to match the corresponding language elements in the original text, the reference translation set and the target text, so as to determine and correct the incorrect evaluation labels in the translation evaluation label sequence.

[0087] As can be seen from the above, by the alignment manner, the correspondence between the original text and the reference translation text, and the correspondence between the original text and the target text about the language elements can be obtained, so that the matching of the language elements in the original text, the reference translation set and the target text can be performed, the matching accuracy is improved, the evaluation labels are accurately evaluated, and the correction of the evaluation labels is effectively realized.

[0088] It should be noted that the steps S21 and S22 in the above embodiments do not have a certain execution order, and the steps S21 and S22 can be executed according to specific circumstances, and the embodiments of the present specification do not make specific limitations thereto.

[0089] In a specific implementation, based on the correspondence among the original text, the reference translation set and the target text, one or more incorrect evaluation labels can be determined and corrected.

[0090] Specifically, based on the first correspondence information and the second correspondence information, when the language element in the original text has any of the following conditions, the corresponding evaluation label is corrected:

[0091] (1) The language element is determined to be omitted, and the corresponding evaluation label represents a translation error.

[0092] In an optional example, the original text A is {A1, A2, …, An}, where A1, A2, …, An are n language elements of the original text A. Based on the first correspondence information, the correspondence between each language element in the original text A and the corresponding reference translation text is determined. Thus, it can be obtained that the language element A2 in the original text A is an omitted language element.

[0093] If the evaluation label LA2 corresponding to the language element A2 of the original text A represents a translation error (such as the evaluation label "BAD"), the evaluation label LA2 corresponding to the language element A2 in the original text A is determined as an incorrect evaluation label, and the evaluation label LA2 is corrected to a correct evaluation label (such as the evaluation label "OK").

[0094] (2) it is the same language element as its corresponding language element in the reference translation text and its corresponding language element in the target text, and its corresponding evaluation label represents a translation error.

[0095] Specifically, the original text B is {B1 B2 …… Bm}, where B1, B2 to Bm are m language elements of the original text B. The reference translation text C is {C1 C2 …… Cp}, where C1, C2 to Cp are p language elements of the reference translation text C. The target text D is {D1 D2 …… Dq}, where D1, D2 to Dp are p language elements of the target text D.

[0096] Based on the first correspondence information and the second correspondence information, the correspondence of each language element in the original text B with the reference translation text C can be determined, and the correspondence of each language element in the original text B with the target text D can be determined. Thus, it can be obtained that the language element B1 in the original text B corresponds to the language element C2 in the reference translation text C, the language element B1 in the original text B corresponds to the language element D2 in the target text D, and the language element C2 in the reference translation text C and the language element D2 in the target text D are the same language element.

[0097] If the evaluation label LB1 of the language element B1 of the original text B represents a translation error (such as the evaluation label “BAD”), the evaluation label LB1 of the language element B1 of the original text B is determined as an incorrect evaluation label, and the evaluation label LB1 is corrected to a correct evaluation label (such as the evaluation label “OK”).

[0098] It can be understood that, in order to facilitate description and understanding, only one reference translation text is shown in the above example, but in actual application, the reference translation set can include multiple reference translation texts, so that multiple reference translation texts are respectively aligned with the original text to obtain multiple first correspondence information. At this time, if the language element in the original text is the same as its corresponding language element in at least one reference translation text and its corresponding language element in the target text, when the evaluation label of the language element in the original text represents a translation error, the evaluation label is determined as an incorrect evaluation label, and the evaluation label is corrected to a correct evaluation label.

[0099] (3) it is the same language element as its corresponding language element in the reference translation text and its corresponding language element in the target text, and its corresponding evaluation label represents a translation error.

[0100] Specifically, the original text E is: {E1 E2 …… Er}, where E1, E2 to Er are r language elements of the original text E. The reference translation text F is: {F1 F2 …… Fs}, where F1, F2 to Fs are s language elements of the reference translation text F. The target text G is: {G1 G2 …… Gt}, where G1, G2 to Gs are t language elements of the target text G. Based on the first correspondence information and the second correspondence information, the correspondence between each language element in the original text E and the reference translation text F, and the correspondence between each language element in the original text E and the target text G can be determined. Thus, it can be obtained that: the language element Er in the original text E corresponds to the language element F1 in the reference translation text F, the language element Er in the original text E corresponds to the language element G2 in the target text G, and the language element F1 in the reference translation text F and the language element G2 in the target text G are different language elements.

[0101] If the evaluation label LEr corresponding to the language element Er of the original text E indicates correct translation (such as the evaluation label "OK"), then the evaluation label LEr corresponding to the language element Er of the original text E is determined to be an incorrect evaluation label, and this evaluation label LEr is corrected to the correct evaluation label (such as corrected to the evaluation label "BAD").

[0102] It can be understood that, for the convenience of description and understanding, only the situation of one reference translation text is shown in the above example. However, in practical applications, the reference translation set may include multiple reference translation texts, so as to align multiple reference translation texts with the original text respectively to obtain multiple first correspondence information. At this time, if the language element in the original text, the language elements corresponding to it in all reference translation texts are all different from the language element corresponding to it in the target text, then when the evaluation label corresponding to this language element in the original text indicates correct translation, this evaluation label is determined to be an incorrect evaluation label, and this evaluation label is corrected to the correct evaluation label.

[0103] As can be seen from the above, for the first case, the language elements that can be reasonably omitted during translation in the original text (such as the English word "the", the Chinese character "了", etc.) can be identified. Thus, when performing translation evaluation on the original text and finding that such language elements are misjudged as translation errors, the corresponding evaluation label can be corrected according to the first correspondence information, thereby improving the accuracy of the translation evaluation result.

[0104] For the (2)th case, the language element in the original text that is misjudged as a translation error (e.g., misjudged as a translation error due to the limitations of manual evaluation or the translation evaluation model itself) can be identified, so that when the original text is evaluated for translation and it is found that there is a language element misjudged as a translation error, the corresponding evaluation label can be corrected according to the first corresponding relationship information and the second corresponding relationship information, thereby improving the accuracy of the translation evaluation result.

[0105] For the (3)th case, the language element in the original text that is misjudged as a translation correct (e.g., misjudged as a translation correct due to the limitations of manual evaluation or the translation evaluation model itself) can be identified, so that when the original text is evaluated for translation and it is found that there is a language element misjudged as a translation correct, the corresponding evaluation label can be corrected according to the first corresponding relationship information and the second corresponding relationship information, thereby improving the accuracy of the translation evaluation result.

[0106] In specific implementation, the determination manner of the language element that can be omitted in the original text can include: determining, based on the first corresponding relationship information, a language unit in the original text that does not have a corresponding relationship with the reference translation text. Specifically, the original text H is: {H1 H2…Hv}, where H1, H2 to Hv are v language elements of the original text H. The reference translation text J is: {J1 J2…Jw}, where J1, J2 to Jw are w language elements of the reference translation text J. If all the w language elements in the reference translation text J are traversed and no language element corresponding to the language element H2 in the original text H is found, the language element H2 in the original text H can be determined as a language element that can be omitted.

[0107] It can be understood that, in order to facilitate description and understanding, only one reference translation text is shown in the above examples, but in actual application, the reference translation set can include multiple reference translation texts, so that multiple reference translation texts are aligned with the original text respectively to obtain multiple first corresponding relationship information. At this time, if a language element in the original text does not have a corresponding relationship with at least one reference translation text, the language element of the original text can be determined as a language element that can be omitted.

[0108] In specific implementation, when the original text is translated to obtain the reference translation text based on the language category of the target text, multiple translation results can be obtained through manual translation or machine translation by a machine translation system with the function of outputting multiple translation results. At this time, these translation results can all be used as reference translation texts, or in order to reduce the amount of data, the translation results can be screened, and the screened translation results can be used as the reference translation text.

[0109] For example, the original text can be {I like cats because they are very elegant.} and the translation results of the original text can be {I like cats because they are very elegant.} and {I love cats because they are very graceful.}. The translation results can be all used as the reference translation texts, and several of them can be selected as the reference translation texts.

[0110] In specific implementations, the confidence of each translation result can be scored when the translation results of the original text are selected, so that the translation results can be selected according to the confidence.

[0111] In an optional example, the translation result with a confidence exceeding a confidence threshold is obtained as the reference translation text. The confidence threshold can be preset or obtained by calculation (such as average calculation or weighted average calculation) according to the confidence of the translation results.

[0112] In another optional example, the translation results can be sorted according to their confidence, and the translation results with the highest confidence are obtained as the reference translation texts. The number of the obtained translation results can be preset or obtained by calculation (such as average calculation) according to the number of the translation results.

[0113] In specific implementations, in order to reasonably expand the reference translation texts and increase the diversity of the reference translation set while ensuring the accuracy of the reference translation set, the original text can be translated based on the language category of the target text, and the translation results can be synonymically expanded in the same language category to obtain the reference translation texts.

[0114] The synonymic expansion refers to an operation of conveying the same semantics of the translation results of the original text by changing the expression of the translation results of the original text without changing the language category.

[0115] In actual applications, the translation results of the original text can be used as the reference translation texts, the synonymic expansion results of the original text can be used as the reference translation texts, or the translation results and the synonymic expansion results of the original text can be used as the reference translation texts according to specific requirements and conditions.

[0116] In a specific implementation, after the translation results are synonymously expanded in the same language category, there can be synonymously expanded results with repeated content, or there can be translation results and synonymously expanded results with repeated content. In order to reduce the amount of data and improve data processing efficiency, de-duplication processing can be performed between synonymously expanded results, and / or between synonymously expanded results and translation results.

[0117] For example, after translating the original text based on the language category of the target text to obtain a plurality of candidate translation texts, synonymously expanding each of the candidate translation texts in the same language category to obtain a plurality of synonymously expanded translation texts, and then performing de-duplication processing based on the plurality of candidate translation texts and the plurality of synonymously expanded translation texts, the reference translation text can be determined.

[0118] In an optional example of a Chinese-English translation application scenario, the original text can be {I like cats because they are very elegant.}. After translating the original text, a plurality of translation results can be obtained: {I like cats because they are very elegant.} and {I love cats because they are very graceful.}.

[0119] Synonymously expanding the translation result {I like cats because they are very elegant.} can obtain a plurality of synonymously expanded results: {I like cats because they are very graceful.}, {I love cats because they are very elegant.}, and {I like cats because of their elegance.}.

[0120] Synonymously expanding the translation result {I love cats because they are very graceful.} can obtain a plurality of synonymously expanded results: {I like cats because they are very graceful.}, {I love cats because they are very elegant.}, and {I love cats because of their grace.}.

[0121] In the obtained multiple synonym expansion results, there are repeated synonym expansion results {I like cats because they are very graceful.} and {I love cats because they are very elegant.}, and after the deduplication processing, it can be ensured that the multiple translation results and the multiple synonym expansion results are texts with different contents.

[0122] In specific implementations, after the deduplication processing based on the multiple candidate translation texts and the multiple synonym translation texts, the candidate translation texts and the synonym translation texts can be screened again. For details, refer to the related content described above, and no longer be repeated.

[0123] In specific implementations, after the incorrect evaluation labels in the translation evaluation label sequence are corrected, the corrected translation evaluation label sequence can be output. The output object of the corrected translation evaluation label sequence can be set according to specific application scenarios and needs.

[0124] For example, the corrected translation evaluation label sequence can be output to the corresponding storage medium for subsequent processor retrieval. For another example, the corrected translation evaluation label sequence can be output to the corresponding display through the interface, and the corrected translation evaluation label sequence is displayed in the display interface for the user to intuitively obtain the translation evaluation result.

[0125] In specific implementations, the data processing method provided by the embodiments of the present specification can be applied to any professional field with translation evaluation needs, and accordingly, the original text can be a text related to any field with translation evaluation needs.

[0126] For example, the data processing method provided by the embodiments of the present specification can be applied to the education professional field, the news professional field, etc. Accordingly, for the education professional field, the original text can be a translation question itself, an answer obtained by translating the translation question, etc.; for the news professional field, the original text can be a manuscript itself, a translated manuscript obtained by translating the manuscript, etc.

[0127] It can be understood that the above examples are only illustrative, and in the present specification, multiple embodiments are provided, and each embodiment can be combined with each other without conflict, cross-referenced, thereby extending multiple possible embodiments, which can be considered as the embodiments disclosed and disclosed by the embodiments of the present specification.

[0128] The specification also provides a data processing device corresponding to the above data processing method, which will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the data processing device described below can be considered as a functional module required to implement the data processing method provided in the specification; the content of the data processing device described below can be mutually corresponding to the content of the data processing method described above.

[0129] In specific implementation, as shown in Figure 4 , a structural block diagram of a data processing device provided by an embodiment of the specification. In Figure 4 , the data processing device M1 can include:

[0130] The data acquisition module M11 is adapted to acquire the original text, the translation evaluation label sequence corresponding to the original text, and the target text corresponding to the original text, and acquire a reference translation set of the same language category as the target text based on the original text; wherein the translation evaluation label sequence includes: an evaluation label adapted to represent correct translation or incorrect translation; the reference translation set includes: a reference translation text;

[0131] The label correction module M12 is adapted to correct the incorrect evaluation label in the translation evaluation label sequence based on the original text, the reference translation set, and the target text.

[0132] As can be seen from the above, the reference translation set is determined according to the original text, and the reference translation text contained in the reference translation set can change with the change of the original text, improving the flexibility and reliability of the reference translation set, and the association existing between the original text, the target text and the reference translation set in the semantic can be used to comprehensively evaluate the evaluation label and correct the incorrect evaluation label, thereby improving the accuracy of the translation evaluation result without changing the translation evaluation method.

[0133] In addition, since the data processing device provided by the embodiment of the specification is not related to which translation evaluation method is used, the data processing device provided by the embodiment of the specification can be applied to manual evaluation and translation evaluation models of different architectures, has strong universality, and has a wide range of applications.

[0134] In specific implementation, continuing to refer to Figure 4 , the data acquisition module M11 can align the original text, the reference translation set and the target text, thereby obtaining the corresponding relationship at the language element level, so as to facilitate the matching among the three, and facilitate the label correction of the label correction module M12.

[0135] Specifically, the data obtaining module M11 is adapted to align the original text and the reference translation text to obtain first correspondence information, and align the original text and the target text to obtain second correspondence information.

[0136] The first correspondence information is adapted to represent the correspondence between each language element in the original text and each language element in the reference translation text. The second correspondence information is adapted to represent the correspondence between each language element in the original text and each language element in the target text.

[0137] The label correction module M12 is adapted to correct the error evaluation label in the translation evaluation label sequence based on the first correspondence information and the second correspondence information.

[0138] It can be understood that the alignment operation of the data obtaining module M11 and the label correction operation of the label correction module M12 can refer to the description of the above related part, which will not be repeated here.

[0139] As can be seen from the above, by the alignment method, the correspondence between the original text and the reference translation text and the original text and the target text about the language elements can be obtained, so that the original text, the reference translation set and the target text can be matched at the language element level, the matching accuracy is improved, and the evaluation label is accurately evaluated, and the evaluation label correction is effectively realized.

[0140] In specific implementation, continuing to refer to Figure 4 , the data obtaining module M11 can obtain the reference translation set in at least one of the following ways:

[0141] (A) translating the original text based on the language category of the target text to obtain the reference translation text;

[0142] (B) translating the original text based on the language category of the target text, and performing synonymous expansion of the same language category on the translation result to obtain the reference translation text.

[0143] It can be understood that the specific process of obtaining the reference translation set by the data obtaining module M11 can refer to the description of the above related part, which will not be repeated here.

[0144] In specific implementation, continuing to refer to Figure 4 , the label correction module M12 is further adapted to output the corrected translation evaluation label sequence after correcting the error evaluation label.

[0145] In specific implementation, as Figure 5As shown in FIG. 1, the data processing device M2 provided by the embodiment of the present application includes a memory M21 and a processor M22. The memory M21 and the processor M22 can communicate with each other through a communication bus M23. The memory M21 stores computer instructions executable on the processor M22. When the processor M22 executes the computer instructions, the processor M22 can perform the steps of the data processing method according to any one of the above embodiments. Details can be referred to the above related content, and will not be described here. Figure 5 In some embodiments, the data processing device M2 can include a memory M21 and a processor M22. The memory M21 and the processor M22 can communicate with each other through a communication bus M23. The memory M21 stores computer instructions executable on the processor M22. When the processor M22 executes the computer instructions, the processor M22 can perform the steps of the data processing method according to any one of the above embodiments. Details can be referred to the above related content, and will not be described here.

[0146] In some embodiments, the processor can include a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), or the like. The memory can include a random access memory (RAM), a read-only memory (ROM), a non-volatile memory (NVM), or the like.

[0147] In some embodiments, the computer instructions can include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, and the like, implemented by using any suitable high-level, low-level, object-oriented, visual, compiled and / or interpreted programming language.

[0148] In some embodiments, as shown in FIG. 1, the data processing device M2 can further include a display interface M24 and a display M25 connected to the display interface M24. The display interface M24 can communicate with the memory M21 and the processor M22 through the communication bus M23. The display M25 can display the correction result obtained by the processor M22 executing the data processing method provided by the embodiment of the present application. Figure 5

[0149] The present application also provides a computer readable storage medium having computer instructions stored thereon. When the computer instructions are executed, the steps of the data processing method according to any one of the above embodiments can be performed. Details can be referred to the above related content, and will not be described here.

[0150] ​The computer-readable storage medium can include any suitable type of memory unit, memory device, memory article, memory medium, storage device, storage article, storage medium and / or storage unit. For example, memory, removable or non-removable media, erasable or non-erasable media, writeable or re-writeable media, digital or analog media, hard disk, floppy disk, Compact Disk Read Only Memory (CD-ROM), Compact Disk Recordable (CD-R), Compact Disk Rewriteable (CD-RW), optical disk, magnetic media, magneto-optical media, removable memory cards or disks, various types of Digital Versatile Disk (DVD), a tape, a cassette, or the like.

[0151] Also, the computer instructions can include any suitable type of codes, such as source codes, compiled codes, interpreted codes, executable codes, static codes, dynamic codes, encrypted codes, and the like implemented by using any suitable high-level, low-level, object-oriented, visual, compiled and / or interpreted programming language.

[0152] It can be understood that, in the description of the present specification, unless otherwise explicitly specified and limited, the terms in the present specification can be understood according to different application scenarios. In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can be explicitly or implicitly included one or more features. Moreover, the terms "first", "second" and the like are used to distinguish similar objects, and do not necessarily be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0153] Although the embodiments of the present specification are disclosed as above, the embodiments of the present specification are not limited thereto. Any person skilled in the art can make various modifications and changes without departing from the spirit and scope of the embodiments of the present specification, and therefore the protection scope of the embodiments of the present specification should be subject to the scope defined by the claims.

Claims

1. A data processing method, applied to a data processing device, characterized in that, include: Obtain the original text, the corresponding translation evaluation tag sequence of the original text, and the corresponding target text of the original text, wherein: the translation evaluation tag sequence includes: evaluation tags, suitable for characterizing whether the translation is correct or incorrect; Based on the original text, a reference translation set of the same language as the target text is obtained, the reference translation set including: reference translation text; Based on the original text, the reference translation set, and the target text, erroneous evaluation labels in the translation evaluation label sequence are corrected, including: Align the original text and the reference translated text to obtain first correspondence information, wherein the first correspondence information is suitable for characterizing the correspondence between each language element in the original text and each language element in the reference translated text; Align the original text and the target text to obtain second correspondence information, wherein the second correspondence information is suitable for characterizing the correspondence between each language element in the original text and each language element in the target text; Based on the first correspondence information and the second correspondence information, the erroneous evaluation labels in the translation evaluation label sequence are corrected.

2. The data processing method according to claim 1, characterized in that, The step of correcting erroneous evaluation tags in the translation evaluation tag sequence based on the first correspondence information and the second correspondence information includes: Based on the first and second correspondence information, when any of the following situations exist in the language elements of the original text, their corresponding evaluation labels are corrected: Language elements deemed optional are identified and their corresponding evaluation labels indicate translation errors. The language element corresponding to it in the reference translation text and the language element corresponding to it in the target text are the same language element, and its corresponding evaluation label indicates a translation error; The language element corresponding to it in the reference translation text and the language element corresponding to it in the target text are different language elements, and the corresponding evaluation label indicates that the translation is correct.

3. The data processing method according to claim 2, characterized in that, The methods for determining whether language elements in the original text can be omitted include: Based on the first correspondence information, language elements in the original text that do not have a corresponding relationship with the reference translated text are identified.

4. The data processing method according to any one of claims 1-3, characterized in that, The step of obtaining a reference translation set in the same language as the target text based on the original text includes at least one of the following: Based on the language of the target text, the original text is translated to obtain the reference translated text; Based on the language of the target text, the original text is translated, and the translation result is expanded with synonyms in the same language to obtain the reference translated text.

5. The data processing method according to claim 4, characterized in that, The process of translating the original text based on the language of the target text, and then expanding the translation result with synonyms in the same language to obtain the reference translated text, includes: Based on the language of the target text, the original text is translated to obtain multiple candidate translated texts; Each candidate translation text is expanded with synonyms in the same language to obtain multiple synonym translation texts; Based on multiple candidate translation texts and multiple synonym translation texts, deduplication is performed to determine the reference translation text.

6. The data processing method according to any one of claims 1-3, characterized in that, Also includes: Output the corrected translation evaluation label sequence.

7. A data processing device, characterized in that, include: The data acquisition module is adapted to acquire original text, a translation evaluation tag sequence corresponding to the original text, and a target text corresponding to the original text, and based on the original text, acquire a reference translation set of the same language as the target text; wherein, the translation evaluation tag sequence includes: evaluation tags, adapted to characterize whether the translation is correct or incorrect; the reference translation set includes: reference translation text; The label correction module is adapted to correct erroneous evaluation labels in the translation evaluation label sequence based on the original text, the reference translation set, and the target text, including: Align the original text and the reference translated text to obtain first correspondence information, wherein the first correspondence information is suitable for characterizing the correspondence between each language element in the original text and each language element in the reference translated text; Align the original text and the target text to obtain second correspondence information, wherein the second correspondence information is suitable for characterizing the correspondence between each language element in the original text and each language element in the target text; Based on the first correspondence information and the second correspondence information, the erroneous evaluation labels in the translation evaluation label sequence are corrected.

8. A data processing device, comprising a memory and a processor; wherein, The memory is adapted to store one or more computer instructions, characterized in that, when the processor executes the computer instructions, it performs the steps of the data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed, they perform the steps of the data processing method according to any one of claims 1 to 6.

Images