Estimation device, learning device, estimation method, learning method, and program

Abstract

Provided is an estimation device equipped with: an input unit for inputting an expanded word correspondence for original text and translated text, and a translation quality tag for said original text and said translated text; and an editing tag estimation unit for estimating an editing tag on the basis of the expanded word correspondence and the translation quality tag.

Description

[Technical Field]

[0001] This invention relates to translation quality estimation and support for editing (proofreading) translation results. [Background technology]

[0002] One prior art for estimating the word-level quality of machine translation is the technology disclosed in Non-Patent Document 1.

[0003] In the technology disclosed in Non-Patent Document 1, quality tags are assigned to each of the words in the original text, the words in the machine-translated text, and the gaps in the machine-translated text by binary classification of OK or BAD. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Hyun Kim, Joon-Ho Lim, Hyun-Ki Kim, and Seung-Hoon Na. Qe bert: Bilingual bert using multi-task learning for neural quality estimation. In Proceedings of the WMT-2019, pp. 85-89, 1999. [Overview of the project] [Problems that the invention aims to solve]

[0005] Conventional word-level quality estimation techniques can identify errors in machine-translated text using quality tags, but they cannot pinpoint the corresponding location of those errors in the source text. Therefore, when humans post-edit machine-translated text, they must determine what needs to be corrected and how. This challenge also arises when applying conventional word-level quality estimation techniques to translations other than machine-translated texts (e.g., human-translated texts).

[0006] The present invention has been made in view of the above points, and an object thereof is to provide a technique that enables estimation of editing operations to be performed during post-editing on a translated text.

Means for Solving the Problems

[0007] According to the disclosed technique, for the source text and the translated text This includes both correctly translated and incorrectly translated word pairs. expanded word correspondence, and the source text Each word in and the translated text Each word and each gap in for Indicates whether there is an error or not. translation quality tags are input, and an input unit, an editing tag estimation unit that estimates an editing tag based on the expanded word correspondence and the translation quality tags, are provided, The editing tag is at least one of the following: a replacement tag indicating that a word in the translated text should be replaced with the correct translation of the word in the source text; an insertion tag indicating that the translation of the word in the source text should be inserted into the translated text; and a deletion tag indicating that a word should be deleted from the translated text. The editing tag estimation unit estimates the editing tag using a program corresponding to a predetermined rule for estimating the editing tag from the extended word correspondence and the translation quality tag, or using a trained neural network for estimating the editing tag from the extended word correspondence and the translation quality tag. An estimation device is provided.

Advantages of the Invention

[0008] According to the disclosed technique, a technique that enables estimation of editing operations to be performed during post-editing on a translated text is provided.

Brief Description of the Drawings

[0009] [Figure 1] It is a diagram showing an example of the overall configuration of a system in an embodiment of the present invention. [Figure 2] It is a configuration example of the learning device 100. [Figure 3] It is a configuration example of the estimation device 200. [Figure 4] It is a diagram showing a procedure for learning an expanded word correspondence model and a translation quality model. [Figure 5] It is a diagram showing a procedure for estimating expanded word correspondence and an editing tag. [Figure 6] It is a diagram showing an output example of a word-level quality estimation result in the prior art. [Figure 7] It is a diagram showing an output example by the editing tag estimation unit 150. [Figure 8] This is a diagram showing the configuration of an extended word correspondence model. [Figure 9] This is a diagram showing an example of the procedure of a learning method for an extended word correspondence model. [Figure 10] This is a diagram showing the configuration of a translation quality tagging model. [Figure 11] This is a diagram showing an example of the hardware configuration of a device. [Figure 12] This is a diagram showing the accuracy of extended word correspondence. [Figure 13] This is a diagram showing the accuracy of translation quality tagging. [Figure 14] This is a diagram showing the accuracy of editing tagging (En-De). [Figure 15] This is a diagram showing the accuracy of editing tagging (En-Zh).

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention (these embodiments) will be described with reference to the drawings. The embodiments described below are merely examples, and the embodiments to which the present invention is applied are not limited to the following embodiments.

[0011] In the following description, machine-translated sentences are targeted, but this is an example. In these embodiments, it is also possible to target translated sentences other than machine-translated sentences (e.g., human-translated sentences). Also, in these embodiments, supervised word correspondence techniques are used, but unsupervised word correspondence techniques can also be used.

[0012] Moreover, all of the systems and devices described in these embodiments provide specific improvements over conventional techniques such as those of Non-Patent Document 1, and show improvements in the technical field related to translation quality estimation and support for editing (proofreading) of translation results.

[0013] Note that the technologies described in the references cited in the following explanation are publicly known technologies, but the descriptions of the problems related to those technologies are not publicly known. The reference numbers and titles are listed together at the end of the specification. The reference numbers cited in the following explanation are indicated as [1], etc.

[0014] (Summary of the embodiment) First, an overview of this embodiment will be described. The technology according to this embodiment relates to machine translation quality estimation. Below, an overview of the related technology (prior art), its problems, and the technology of this embodiment that solves those problems will be described.

[0015] Quality estimation of machine translation is a technique for evaluating the quality of the output of a machine translation system without referring to a reference translation. The quality estimation task, one of the shared tasks of the Conference on Machine Translation (WMT), a major international conference on machine translation,[6] has two subtasks: word-level and sentence-level.

[0016] Word-level quality estimation is a process that assigns one of two quality tags, OK or BAD, to every word in the source text (input), every word in the machine-translated text (output), and every gap in the machine-translated text. Here, a gap refers to the position between adjacent words in the machine-translated text.

[0017] A quality tag of BAD for a word in the source text indicates that the word has been mistranslated or not translated at all in the machine translation. A quality tag of BAD for a word in the machine translation indicates that the word has been mistranslated in the source text or that the word should be deleted. A quality tag of BAD for a gap indicates that one or more words should be inserted at the location of that gap in the machine translation.

[0018] Sentence-level translation quality estimation is the process of estimating the overall quality of the translated sentence. Measures used to assess overall sentence quality include the Human Translation Error Rate (HTER), which indicates how many edit operations are needed to correct translation errors, and direct assessment, which expresses the quality of the machine translation as a translation of the source text on a scale of 0 to 100, taking into account appropriateness and fluency.

[0019] If we have a set of three elements—the source sentence (SRC) which is the input to the machine translation system, the machine-translated sentence (MT) output from the machine translation system, and the correct post-edited sentence (PE) obtained by manually post-editing the machine-translated sentence—we can automatically determine the quality tags of the words from the word correspondences between the machine-translated sentence and the post-edited sentence, as well as from the word correspondences between the source sentence and the post-edited sentence.

[0020] The word correspondence between machine-translated text and post-edited text can be determined using monolingual word alignment tools included in tools that calculate the Translation Edit Rate (TER).

[0021] The word correspondence between the original text and the machine-translated text can be determined using bilingual word correspondence tools such as GIZA++ or FastAlign.

[0022] From the resulting pairs of source text and machine-translated text tagged with word-level quality, a model can be trained to automatically perform word-level quality estimation using machine learning such as neural networks [8, 9].

[0023] Conventional word-level quality estimation techniques, as described above, can identify where errors exist in machine-translated texts using quality tags, but they cannot pinpoint where those errors correspond in the original text. Therefore, when humans post-edit machine-translated texts, there is a problem in that humans must determine what needs to be corrected and how.

[0024] For example, if a word in a machine-translated text is marked "BAD," it's impossible to distinguish whether the word should be corrected because it's been mistranslated, or whether it should be deleted because it doesn't correspond to the original text. Similarly, if a gap in a machine-translated text is marked "BAD," it's unclear which word from the original text corresponds to which translation and insertion should be made.

[0025] In this embodiment, the estimation device 200, described later, can find not only word pairs that are correctly translated (translate each other) in the original text and the machine-translated text, but also word pairs that are incorrectly translated. This function of finding correctly translated and incorrectly translated word pairs is called extended word alignment. A word pair that includes both correctly translated and incorrectly translated word pairs may also be called an extended word alignment.

[0026] Furthermore, in this embodiment, the estimation device 200 uses this extended word correspondence to subdivide the word-level translation quality tag BAD into three categories: insert (INS), delete (DEL), and replace (REP). Here, tags that represent editing operations that post-editors should perform, such as insert (INS), delete (DEL), and replace (REP), are called edit tags, and the combination of edit tags and OK is called a refined quality tag. Alternatively, the combination of edit tags and OK may simply be called an "edit tag."

[0027] For words that are marked as BAD in the source text and for which there is no corresponding extended word in the machine translation, the INSE tag should be added so that the translated word should be inserted into the machine translation. For words for which a corresponding word exists in the machine translation, the REP tag should be added so that the word in the corresponding machine translation should be replaced with the correct translation for the word in the source text.

[0028] In machine translations, words that are marked as "BAD" and do not have corresponding extended word tags in the source text should be deleted, so a Delete (DEL) tag should be added. Words that do have corresponding tags in the source text should be replaced with the correct translation of the corresponding word in the source text, so a Replace (REP) tag should be added.

[0029] In this embodiment, in addition to assigning machine translation quality tags to each word in the source text and the machine-translated text in the target language, as in conventional word-level translation quality estimation, it is possible to provide quality tags that are subdivided down to the editing operations that the worker should perform during post-editing, such as insertion, deletion, and replacement, by determining the extended word correspondence between the source text and the machine-translated text.

[0030] In the following, in order to facilitate understanding of the technology according to this embodiment, various reference technologies related to machine translation quality estimation and their associated problems will be described. Following that, the configuration and operation of the system and apparatus according to this embodiment will be explained.

[0031] (Regarding reference technologies) <Word-level quality estimation based on predictor-estimator models> The predictor-estimator model is currently the standard translation quality estimation method used in machine translation quality estimation. Unbabel's system, which achieved the highest performance in the quality estimation task, one of the common tasks at WMT-2019, the most important international conference on machine translation, uses a predictor-estimator model based on Transformer

[14] , which has achieved the highest performance in machine translation[8].

[0032] Furthermore, OpenKiwi, which is designated as the baseline system for the word-level quality estimation task at the international conference WMT, also uses a predictor-estimator model.

[0033] A predictor is a model that predicts each token in a machine-translated text based on the context of the source text and the surrounding words in the machine-translated text. A predictor consists of a Transformer encoder that encodes the source text, a Transformer decoder that predicts a word in the machine-translated text using the words from the beginning of the sentence up to the word immediately preceding it, and a Transformer decoder that predicts a word using the words from the end of the sentence up to the word immediately following it.

[0034] The estimator is a model that uses the features generated by the predictor described above to estimate the quality tag (OK or BAD) for each word. Specifically, it uses information obtained from the encoder of the source text, the decoder from the beginning of the machine-translated text up to the word in question, and the decoder from the end of the machine-translated text up to the word in question to predict the quality tag of that word.

[0035] <Pre-trained language model BERT> BERT[3] is a language representation model that uses a Transformer-based encoder to output a word vector for each word in an input sequence, taking into account the surrounding context. Typically, the input sequence is a single sentence or two sentences concatenated with special characters in between.

[0036] BERT pre-trains a language representation model from large-scale language data using a task that trains a masked language model to predict masked words in an input sequence from both forward and backward directions, and a next sentence prediction task that determines whether two given sentences are adjacent. By using such pre-training tasks, BERT can output word vectors that capture features related to linguistic phenomena not only within a single sentence but also across two sentences. Language representation models like BERT are sometimes simply called language models.

[0037] It has been reported that by adding an appropriate output layer to a pre-trained BERT model and performing transfer learning (fine tuning) with training data for the target task, the highest accuracy can be achieved in various tasks such as semantic text similarity, natural language inference (text entailment recognition), question answering, and named entity recognition.

[0038] BERT was originally created for English, but now BERT models for various languages, including Japanese, have been developed and made publicly available. Furthermore, a general-purpose multilingual model, multilingual BERT, created using monolingual data from 104 languages ​​extracted from Wikipedia, is also publicly available.

[0039] Furthermore, the cross-language model XLM, pre-trained using a fill-in-the-blank language model with parallel texts, has been proposed and has been reported to be more accurate than multilingual BERT in applications such as cross-language text classification [2]. In addition, XLM-R, which is trained on large-scale monolingual texts in many languages ​​without using parallel texts, has been reported to be more accurate than XLM [1]. Pre-trained models of XLM and XLM-R are also publicly available.

[0040] <Word-level quality estimation using trained multilingual models> The South Korean public research institute ETRI has proposed a word-level quality estimation method using BERT [9] (Non-Patent Literature 1). First, a gapped machine translation is created by inserting a token called [GAP] corresponding to a gap between every two adjacent words in the machine translation. Next, the source text and the machine translation are concatenated to create an input sequence for BERT in the form of '[CLS]source text[SEP]gapped machine translation[SEP]'. Here, [CLS] is a special token used to create a vector that aggregates information from the two source texts, and [SEP] is a token that represents a sentence boundary.

[0041] A neural network is used that takes vectors output by BERT as input, corresponding to the words in the original text, the words in the machine-translated text, and the gaps in the machine-translated text. Quality tags are then assigned to each of these using a binary classification of OK or BAD.

[0042] The overall quality of the text is assessed by using a neural network to predict the HTER value, taking the vector output by BERT for [CLS] as input.

[0043] <Word correspondence using trained multilingual models> Traditionally, word mapping between parallel text pairs has relied on statistical machine translation models such as GIZA++

[12] and FastAlign[5]. However, in recent years, word mapping techniques that significantly surpass GIZA++ and FastAlign have been proposed by using pre-trained multilingual models.

[0044] Reference

[11] discloses a supervised word correspondence method that applies a question answering technique using BERT to obtain word correspondences through cross-language span prediction, where a word from one sentence of a pair of parallel sentences is used as the question, and the span (sequence of words) of the other sentence, which is the translation of that word, is obtained as the answer. This method has the advantage of achieving high accuracy with a small amount of training data, such as 300 sentences.

[0045] Reference [4] discloses AWESoME, a word mapping method that works robustly across various language pairs by fine-tuning a pre-trained multilingual model using parallel data based on an objective function that improves word mapping accuracy. This method is significantly more accurate than word mapping using statistical translation models such as GIZA++ and FastAlign, however, when training data is available, supervised word mapping disclosed in Reference

[11] is more accurate.

[0046] <Regarding the assignment> Conventional word-level quality estimation techniques can identify which words in the source text are not translated correctly, and which words or gaps (between words) in the machine-translated text are not translated correctly. However, they have the problem of not being able to provide information on where the errors in the machine-translated text correspond in the source text, or what editing operations (insertion, deletion, replacement) should be applied based on the correspondence between the source text and the machine-translated text.

[0047] Reference

[13] reports that displaying word correspondences is effective in post-editing machine translations. When the translation quality is low, displaying word correspondences statistically significantly improves the quality of post-editing. When the translation quality is high, displaying word correspondences shortens post-editing time, although this is not statistically significant. Reference

[13] uses the functionality of the statistical machine translation decoder Moses

[10] to display word correspondences between the source text and the machine-translated text.

[0048] Neural machine translation offers significantly higher translation accuracy than statistical machine translation, but it has the problem of not being able to easily present word correspondences between the source text and the machine-translated text, as statistical translation does. Reference [7] discloses a method for performing word correspondence and translation simultaneously in neural machine translation, but its word correspondence accuracy is about the same as GIZA++. The supervised word correspondence disclosed in Reference

[11] is significantly more accurate than GIZA++, but this method assumes parallel text pairs that are mutually translated as input, and there is a problem in whether it will work robustly with sentences that are not necessarily mutually translated due to translation errors, such as source text and machine-translated text.

[0049] (Specific description of the technology related to the embodiment) The technology relating to this embodiment will be described in detail below.

[0050] In this embodiment, an extended word correspondence between the source text and the machine-translated text is obtained using supervised word correspondence, and by combining this extended word correspondence with word-level translation quality estimation, quality tags that are subdivided to the level of editing operations that should be performed during post-editing such as insertion, deletion, and replacement are estimated.

[0051] <Example of device configuration> Figure 1 shows an example of the overall system configuration in this embodiment. The system consisting of the configuration shown in Figure 1, excluding the translation quality tag training data DB111 and the extended word correspondence training data DB121, is called the edit tag estimation system. The edit tag estimation system may also be called the edit tag estimation device. The translation quality tag training data DB111 and the extended word correspondence training data DB121 may be included within the edit tag estimation system.

[0052] As shown in Figure 1, the editing tag estimation system includes an extended word support learning unit 120, a trained multilingual model DB 122, an extended word support estimation unit 140, an extended word support model DB 141, a translation quality tag model learning unit 110, a trained multilingual model DB 112, a translation quality tag model estimation unit 130, a translation quality tag model DB 131, an editing tag estimation unit 150, input units 108, 118, 128, 138, 148, and output units 109, 119, 129, 139, 149.

[0053] Furthermore, the learning device 100 may be configured to include a translation quality tag model learning unit 110, a trained multilingual model DB 112, an extended word support learning unit 120, and a trained multilingual model DB 122. Alternatively, the learning device 100 may be configured to include the "translation quality tag model learning unit 110 and trained multilingual model DB 112" but not the "extended word support learning unit 120 and trained multilingual model DB 122". Alternatively, the learning device 100 may be configured to include the "extended word support learning unit 120 and trained multilingual model DB 122" but not the "translation quality tag model learning unit 110 and trained multilingual model DB 112".

[0054] Figure 2 shows an example of a learning device 100 comprising an extended word support learning unit 120, a trained multilingual model DB 122, an input unit 118, and an output unit 119. The example shown in Figure 2 uses extended word support pseudo-training data. An example of its operation will be described later.

[0055] Alternatively, an estimation device 200 may be configured having a translation quality tag model estimation unit 130, a translation quality tag model DB 131, an extended word correspondence estimation unit 140, an extended word correspondence model DB 141, an edit tag estimation unit 150, input units 128, 138, 148, and output units 129, 139, 149.

[0056] The estimation device 200 may include a "translation quality tag model estimation unit 130, translation quality tag model DB 131, extended word correspondence estimation unit 140, and extended word correspondence model DB 141," but may not include an "edit tag estimation unit 150."

[0057] The estimation device 200 may include an "edit tag estimation unit 150" but may not include a "translation quality tag model estimation unit 130, translation quality tag model DB 131, extended word correspondence estimation unit 140, and extended word correspondence model DB 141".

[0058] The estimation device 200 may include a "translation quality tag model estimation unit 130 and translation quality tag model DB 131" but may not include an "extended word support estimation unit 140, extended word support model DB 141, and editing tag estimation unit 150".

[0059] The estimation device 200 may be equipped with an "extended word-compatible estimation unit 140 and an extended word-compatible model DB 141" but may not be equipped with a "translation quality tag model estimation unit 130, a translation quality tag model DB 131, and an edit tag estimation unit 150".

[0060] The estimation device 200 may include an "extended word-response estimation unit 140, an extended word-response model DB 141, and an edit tag estimation unit 150," but may not include a "translation quality tag model estimation unit 130 and a translation quality tag model DB 131."

[0061] Figure 3 shows an example of an estimation device 200 comprising a translation quality tag model estimation unit 130, a translation quality tag model DB 131, an extended word correspondence estimation unit 140, an extended word correspondence model DB 141, and an edit tag estimation unit 150.

[0062] <Overview and Operation Example> The operation of each part of the editing tag estimation system will be explained with reference to the flowcharts in Figures 4 and 5.

[0063] Figure 4 shows the procedure for training the extended word recognition model and the translation quality model. Here, it is assumed that the extended word recognition training data is stored in Extended Word Recognition Training Data DB121, and the translation quality tag training data is stored in Translation Quality Tag Training Data DB111. The trained multilingual models are stored in Trained Multilingual Model DB112 and 122, respectively.

[0064] In this embodiment, the "model" refers to a neural network model, specifically data consisting of weight parameters, functions, etc.

[0065] In S101, the Extended Word Support Learning Unit 120 receives training data for extended word support from the Extended Word Support Training Data DB 121 via the Input Unit 118, and uses this training data to fine-tune the trained multilingual model read from the Trained Multilingual Model DB 122 to create an Extended Word Support model. The created Extended Word Support model is output by the Output Unit 119 and stored in the Extended Word Support Model DB 141.

[0066] Fine-tuning involves using the parameters of a pre-trained model as initial values ​​for the target model (a model created by adding an output layer to the pre-trained multilingual model) to train the target model.

[0067] In S102, the translation quality tag model learning unit 110 receives training data for translation quality tags from the translation quality tag training data DB 111 via the input unit 108, and uses this training data to fine-tune the trained multilingual model read from the trained multilingual model DB 112 to create a translation quality tag model. The created translation quality tag model is output from the output unit 109 and stored in the translation quality tag model DB 131.

[0068] Note that the order in which S101 and S102 are processed can be reversed, or S101 and S102 can be executed simultaneously.

[0069] Next, referring to the flowchart in Figure 5, we will explain the procedure for estimating extended word correspondences and edit tags.

[0070] In S201, pairs of source text and machine-translated text are input to the extended word correspondence estimation unit 140 and the translation quality tag estimation unit 130, respectively, via the input units 138 and 128.

[0071] In S202, the extended word correspondence estimation unit 140 uses the extended word correspondence model read from the extended word correspondence model DB141 to estimate the extended word correspondence for the input source text and machine translation pairs, and outputs it via the output unit 139.

[0072] In S203, the translation quality tag estimation unit 130 uses the translation quality tag model read from the translation quality tag model DB131 to estimate the translation quality tag for each input source text and machine-translated text pair, and outputs it via the output unit 129.

[0073] In S204, the editing tag estimation unit 150 receives the extended word correspondence and translation quality tag from the input unit 148, estimates the editing tag, and outputs the editing tag from the output unit 149. The order of S202 and S203 may be reversed, or S202 and S203 may be executed simultaneously.

[0074] The output format for editing tags is not limited to a specific format, but for example, as shown in Figure 7 below, the editing tags are output along with the original text, the machine-translated text, and the extended word correspondence between the original text and the machine-translated text.

[0075] Alternatively, the output unit 149 may output translation quality tags and extended word correspondences without including the editing tag estimation unit 150.

[0076] The following provides a more detailed explanation of the processing operations involved in the editing tag estimation system.

[0077] (Support for extended words and editing tags: Example of operation of the editing tag estimation unit 150) Figure 6 shows an example of the output of word-level quality estimation results in quality estimation, one of the common tasks of the international conference WMT. Figure 7 shows an example of output by the editing tag estimation unit 150 of this embodiment. Figure 6 is shown for comparison with Figure 7. The original text and machine-translated text are the same in both Figure 6 and Figure 7.

[0078] As shown in Figure 6, in the conventional word-level quality estimation, the word "white" in the original text is tagged with BAD, and the word "black" in the machine-translated text is also tagged with BAD. However, with the conventional technique, the mistranslated word pairs are unknown, so it is not possible to determine, for example, whether the word "black" in the machine-translated text is mistranslated or whether it expresses something that does not correspond to the original text.

[0079] On the other hand, in the output example of the editing tag estimation unit 150 in this embodiment, shown in Figure 7, extended word correspondence is shown between the original text and the machine-translated text. Extended word correspondence not only assigns correspondences to word pairs that are correctly translated, such as "I" in the original text and "watashi" and "wa" in the machine-translated text, but also to word pairs that are incorrectly translated, such as "white" in the original text and "kuroi" in the machine-translated text.

[0080] Furthermore, in the conventional word-level quality estimation shown in Figure 6, there are only two types of translation quality tags for each word: OK and BAD. However, as shown in Figure 7, the editing tag estimation unit 150 of this embodiment subdivides BAD tags into editing tags such as REP tags, INS tags, and DEL tags based on the combination of translation quality tags and extended word correspondences.

[0081] In other words, the editing tag estimation unit 150 assigns a replacement (REP) tag to word pairs where the translation quality tag is BAD in both the source text and the machine-translated text, and where extended word correspondence has been assigned, indicating that the word in the machine-translated text should be replaced with the correct translation of the word in the source text. The word pair "white" and "black" in Figure 7 is an example of a word pair to which a replacement (REP) tag has been assigned.

[0082] The editing tag estimation unit 150 assigns an insert (INS) tag to words that have a BAD tag in the original text and do not have an extended word correspondence (words for which there is no corresponding word in the machine translation), indicating that the translation of that word should be inserted into the machine translation. Figure 7 shows examples of "and" and "dogs" with the insert (INS) tag assigned to them.

[0083] Furthermore, the editing tag estimation unit 150 assigns a delete (DEL) tag to words that have been assigned a BAD tag in the machine-translated text and for which no extended word correspondence has been assigned (words that do not have a corresponding word in the original text), indicating that they should be deleted from the machine-translated text. The word "が" in Figure 7 is an example of a word that has been assigned a delete (DEL) tag.

[0084] The edit tag estimation unit 150 can perform the above-described edit tag assignment operation using a program corresponding to the rules described above. Furthermore, the edit tag estimation unit 150 can also implement the above-described edit tag assignment operation using a trained neural network.

[0085] (Estimation of extended word support, and model training) Figure 8 shows the configuration of the extended word support model. Figure 8 may also be considered as the configuration of the extended word support estimation unit 140. Alternatively, Figure 8 may be considered as the extended word support model being trained (in progress) in the extended word support learning unit 120. As shown in Figure 8, the extended word support model has a configuration in which an extended word support output layer is added to a trained multilingual model.

[0086] The following sections will specifically explain the method for estimating extended word correspondence using an extended word correspondence model, and the method for training the extended word correspondence model.

[0087] <About the training data> In this embodiment, training data for augmented word correspondence is created manually or by the automated method described below, and the augmented word correspondence model is trained using the training data and the augmented word correspondence is estimated using the augmented word correspondence model, using the supervised word correspondence method disclosed in reference

[11] .

[0088] Reference

[11] discloses a supervised word correspondence method that applies a question answering technique using BERT to determine word correspondences by cross-language span prediction, where a word from one sentence of a pair of parallel sentences is used as the question, and the span (sequence of words) of the other sentence, which is the translation of that word, is used as the answer. This method has the advantage of achieving high accuracy with a small amount of training data, such as 300 sentences.

[0089] When manually creating training data (ground truth data) for extended word support, for example, as shown in Figure 7, you can prepare the source text and the machine-translated text, manually create the extended word correspondences between them, and store the source text, the machine-translated text, and information indicating the extended word correspondences in the extended word support training data DB121. Note that the manually created extended word support training data may also be called ground truth data.

[0090] It is also possible to automatically create training data for extended word support by having a computer run a program (software). One method (the first method) is to use existing word support software with relatively high accuracy, such as the word support method AWESoME disclosed in reference [4], to obtain word correspondences between the source text and the machine-translated text, and use this as pseudo-correct answer data for extended word support. In this case, the pseudo-correct answer data for extended word support is stored in the extended word support training data DB121, and the pseudo-correct answer data is used as training data to train the extended word support model by the extended word support learning unit 120.

[0091] Furthermore, when using pseudo-correct answer data as described above, it is also possible to create an extended word recognition model using both the pseudo-correct answer data and manually created training data (correct answer data). In this case, as shown in the example in Figure 2, an extended word recognition training data DB121 is provided to store the manually created training data, and an extended word recognition pseudo-training data DB123 is provided to store the pseudo-correct answer data. The procedure for training the extended word recognition model in this case will be explained with reference to Figure 9.

[0092] In S301, word correspondences between the source text and the machine-translated text, including mistranslations, are determined and stored in the Extended Word Correspondence Pseudo-Training Data DB123 as pseudo-training data for extended word correspondence. This pseudo-training data may be created using AWESoME as described above, or it may be created using other software.

[0093] In S302, the extended word support learning unit 120 learns by fine-tuning the trained multilingual model using pseudo-correct data read from the extended word support pseudo-training data DB123, and the learned model is designated as the language model M.

[0094] In S303, the extended word support learning unit 120 further fine-tunes the language model M with training data (created manually) read from the extended word support training data DB121, generating and outputting a (trained) extended word support model. The extended word support model trained through this flow corresponds to ft-mBERT (fine-tuned mBERT), which will be described later.

[0095] A second method for automatically creating training data for extended vocabulary support by having a computer run a program (software) is to determine the extended vocabulary support between the original text and the machine translation by using the words in the edited text as an intermediary, based on the word correspondence between the original text and the edited text (correct translation), and the word correspondence between the machine translation and the edited text.

[0096] The word correspondence between the source text and the edited text, and between the machine-translated text and the edited text, may be created manually or obtained using existing word correspondence software. Since the source text and the edited text are parallel translations of each other, relatively high-accuracy word correspondence can be obtained using existing word correspondence software such as AWESoME. For word correspondence between the edited text and the machine-translated text, monolingual word correspondence software included in tools for calculating TER can be used. Alternatively, a reference translation from parallel data may be used instead of the edited text, and extended word correspondence may be obtained in a similar manner. Any sentence that is considered to be a correct translation of the source text can be used instead of the edited text.

[0097] <Regarding the estimation of extended word support> This section describes how the extended word correspondence estimation unit 140 in this embodiment performs extended word correspondence estimation using an extended word correspondence model. First, the extended word correspondence estimation unit 140 creates an "annotated source text" in which the words for which word correspondence is to be sought in the source text are enclosed with special delimiters.

[0098] In this embodiment, the paragraph mark '¶' is used as the delimiter, as in reference

[11] . However, using the paragraph mark '¶' as the delimiter is just one example, and it is not necessarily required. Any symbol that is included in the vocabulary of the trained multilingual model and does not significantly affect the probability estimation based on the language model may be used.

[0099] The extended word correspondence estimation unit 140 adds the special character '[CLS]' to the beginning of the annotated source text, inserts a special delimiter '[SEP]' between the annotated source text and the machine-translated text, and adds the special delimiter '[SEP]' to the end of the machine-translated text.

[0100] The extended word correspondence estimation unit 140 then takes the sequence '[CLS] Annotated source text [SEP] Machine translation [SEP]' as input to the trained multilingual model. Figure 8 shows the input (and output) when determining the word in the machine translation that corresponds to 'cats' in the source text.

[0101] The trained multilingual model outputs a vector corresponding to each input word, and each vector is input to an output layer attached to the trained multilingual model. The output layer predicts the start and end positions of words in the machine-translated text that correspond to words in the source text enclosed by special delimiters.

[0102] Figure 8 shows that the extended word support output layer predicts the start and end positions of the word "cat" in the machine-translated text, corresponding to the word 'cats' in the original text.

[0103] <About training an extended word recognition model> Basically, the same processing as described above is performed during the learning of the extended word correspondence model in the extended word correspondence learning unit 120. However, the input is extended word correspondence training data (or pseudo-correct answer data), and the parameters are adjusted so that the error between the output and the correct answer is minimized.

[0104] The following will explain in more detail.

[0105] Let the source text be S = [s1, s2, ..., MK, s i ,MK, ..., s m , and the machine translation text be T = [t1, t2, ..., t n . Here, MK is a special symbol used to enclose the word for which word correspondence is to be determined. m is the "length of the source text" (e.g., the number of words, etc.). n is the "length of the machine translation text" (e.g., the number of words, etc.).

[0106] Based on the training data of extended word correspondence, the extended word correspondence learning unit 120 trains the extended word correspondence model to identify the word sequence (span) T i = [t (j,k) , ..., t j , ..., t k (1≦j≦k≦n) in the machine translation text corresponding to the source text word s

[0107]

Equation

[0108] <Bidirectional Estimation and Learning> Since span predictions may differ depending on the direction, the extended word correspondence estimation unit 140 may determine the word sequence (span) of the source text corresponding to each word in the machine-translated text using the same procedure as in the source text -> machine-translated text case, and determine the corresponding points if the sum of the probabilities of the bidirectional span predictions is greater than or equal to a certain threshold. When predictions are made bidirectionally in this manner, the extended word correspondence learning unit 120 may also make bidirectional predictions in this manner when training the extended word correspondence model, and train it in such a way that the error between the prediction result and the correct answer is minimized.

[0109] (Estimation of translation quality tags and training of the model) Figure 10 shows the configuration of the translation quality tag model. Figure 10 may also be considered as the configuration of the translation quality tag estimation unit 130. Alternatively, Figure 10 may be considered as the translation quality tag model being trained (in progress) in the translation quality tag model learning unit 110. As shown in Figure 10, the translation quality tag model has a configuration in which a translation quality tag output layer is added to a trained multilingual model.

[0110] The following sections will specifically explain how to estimate translation quality tags using a translation quality tag model, and how to train the translation quality tag model.

[0111] <About the training data> In this embodiment, training data for translation quality tags is created manually or by the automated method described below.

[0112] When manually creating training data (ground truth data) for translation quality tags, for example, as shown in Figure 6, you can prepare the source text and the machine-translated text, manually create translation quality tags for these, and store the source text, the machine-translated text, and information indicating the translation quality tags in the translation quality tag training data DB111.

[0113] It is also possible to automatically create training data for translation quality tags by having a computer run a program (software). One method is to create training data for translation quality tags from word correspondences between the source text and the edited text, and between the edited text and the machine-translated text, using a procedure similar to the method for creating ground truth data for word-level translation quality estimation at the international conference WMT, as disclosed in reference [6]. Alternatively, a reference translation from parallel data may be used instead of the edited text.

[0114] <Regarding the estimation of translation quality tags> The translation quality tag estimation unit 130 can estimate translation quality tags using a tagging method similar to named entity recognition using a trained multilingual model (reference [3]). That is, the sequence '[CLS]source text[SEP] machine translation[SEP]' is taken as input to the trained multilingual model, and the output layer attached to the trained multilingual model predicts the probability that each word is OK or BAD. The output layer determines that if this probability exceeds a preset threshold, it is OK, and if it is below the threshold, it is BAD. Figure 10 shows that translation quality tags in the machine translation corresponding to each word in the source text are output.

[0115] <About learning translation quality tags> The same processing as described above is basically performed when training the translation quality tag model in the translation quality tag learning unit 110. However, the input is the translation quality tag training data, and the parameters are adjusted so that the error between the output and the correct answer is minimized.

[0116] The following provides a more detailed explanation.

[0117] The original text is S = [s1, s2, ..., s i , ..., s m Let ] be the machine translation text, and T = [t1, t2, ..., t j , ..., t nThe output layer calculates the probability of each word being an OK / BAD tag as follows. During estimation, the output layer determines that if this probability exceeds a predetermined threshold, it is OK, and if it is below the threshold, it is BAD.

[0118]

number

[0119] Thus, translation quality tags may not be determined by simple binary classification, but rather by using the output of a trained multilingual model on the words of the source text and the words of the machine-translated text, and setting separate criteria for the source text and the machine-translated text. Specifically, for example, the formula in [Equation 2] can be used to set separate thresholds for the source text and the machine-translated text to determine OK / BAD.

[0120] The translation quality tag model is trained to minimize the average of the following binary entropy loss over all words.

[0121]

number

[0122] In other words, the device can be realized by using hardware resources such as the CPU and memory built into a computer to execute a program corresponding to the processing performed by the device. The program can be recorded on a computer-readable recording medium (such as portable memory), saved, and distributed. It can also be provided via a network, such as the Internet or email.

[0123] Figure 11 shows an example of the hardware configuration of the computer described above. The computer in Figure 11 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., all of which are interconnected by a bus BS.

[0124] The program that enables processing on the computer is provided, for example, on a recording medium 1001 such as a CD-ROM or memory card. When the recording medium 1001 containing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001; it may also be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files and data.

[0125] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when a program startup command is received. The CPU 1004 implements the functions related to the memory device 1003 according to the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) etc. generated by a program. The input device 1007 consists of a keyboard and mouse, buttons, or a touch panel etc., and is used to input various operation commands. The output device 1008 outputs the calculation results.

[0126] (Effects of the embodiment) An evaluation experiment was conducted using the technology according to this embodiment, and its details are described below.

[0127] <Data used in the evaluation experiment> To confirm the effectiveness of the technology according to this embodiment, the following experiment was conducted using word-level quality estimation data from the international conference WMT-2020

[15] . This data consists of English-to-German (En-De) and English-to-Chinese (En-Zh) translations.

[0128] For each language pair, 7,000 sentences are provided for training, 1,000 for development, and 1,000 for test data. Each dataset consists of the source text, the machine-translated text, and the post-edited text, with translation quality tags assigned to each word in the source text, each word in the machine-translated text, and each gap in the machine-translated text.

[0129] Furthermore, the results of automated word matching using FastAlign are provided as word matching data. To evaluate the extended word matching, manual correction of extended word matching was performed on 200 English-German sentences and 200 English-Chinese sentences in the development data. In addition, to evaluate the effectiveness of supervised extended word matching, manual correction of extended word matching was performed on the remaining 800 sentences in the English-German development data.

[0130] For the pre-trained multilingual models, we used multilingual BERT (mBERT) and XLM-R.

[0131] <Accuracy of extended word support> Figure 12 shows the accuracy of the extended word mapping for 200 sentences in the development data. The accuracy of word mapping is evaluated by precision (P), recall (R), and their harmonic mean, F1.

[0132] The baseline is the accuracy of word mapping obtained by FastAlign, a word mapping tool based on a statistical translation model. The accuracy of AWESoME, a word mapping tool using a pre-trained multilingual model, is considerably higher than that of FastAlign.

[0133] In Figure 12, mBERT represents the accuracy when the extended word mapping model of this embodiment is trained using AWESoME's word mapping as training data (pseudo-ground truth data). ft-mBERT (fine-tuned mBERT) shows the accuracy when the extended word mapping model of this embodiment is trained using 800 manually created ground truth sentences.

[0134] Training the extended word response model of this embodiment using pseudo-correct answer data created with AWESoME results in higher accuracy than using AWESoME as is. If manual correct answer data is available, using it for fine-tuning will further improve accuracy.

[0135] <Translation quality tagging accuracy> Figure 13 shows the accuracy of the translation quality tag (OK / BAD). The accuracy of the translation quality tag is evaluated using MCC, the official evaluation metric for word-level quality estimation in WMT-2020.

[0136] The OpenKiwi system shown in Figure 13 is the system designated as the baseline for word-level quality estimation in WMT-2020. mBERT and XLM-R represent the accuracy when the translation quality tag model of this embodiment is trained using mBERT and XLM-R, respectively, as pre-trained multilingual models.

[0137] As shown in Figure 13, the translation quality tagging in this embodiment is more accurate than that of OpenKiwi. There is no clear trend as to whether mBERT or XLM is better as a trained multilingual model. It differs between the source text and the machine-translated text, and also differs between language pairs.

[0138] <Accuracy of editing and tagging> Figure 14 shows the accuracy of editing tagging for English-German translations. Figure 15 shows the accuracy of editing tagging for English-Chinese translations. Editing tagging is evaluated using the F1 value for tags for all words. In Figures 14 and 15, the F1 values ​​for OK, Insert (INS), Delete (DEL), and Replace (REP) tags are shown in parentheses. The accuracy of editing tagging was evaluated for each combination of the method for determining extended word support and the method for determining translation quality tags.

[0139] Compared to the baseline combination of FastAlign and OpenKiwi, the combination of extended word support and translation quality tagging in this embodiment shows significantly higher accuracy. There is no clear trend regarding whether mBERT or XLM-R is better as a pre-trained multilingual model. When manual ground truth data for extended word support is available, using manual ground truth data for extended word support results in higher accuracy for editing and tagging than using pseudo-ground truth data.

[0140] (Summary of the embodiments) The technology according to this embodiment not only assigns machine translation quality tags to each word in the source text and the machine-translated text in the target language, as in conventional word-level translation quality estimation, but also, by determining extended word correspondences between the source text and the machine-translated text, it is possible to present quality tags that are subdivided down to the editing operations that the worker should perform during post-editing, such as insertion, deletion, and replacement.

[0141] Furthermore, the technology according to this embodiment is expected to reduce the psychological burden on workers performing post-editing by presenting them with extended word support and translation quality tags, or extended word support and editing tags.

[0142] Furthermore, the technology according to this embodiment can be applied to any language pair included in a trained multilingual model.

[0143] Furthermore, while the extended word handling model and translation quality tag model of this embodiment are based on supervised learning, they can be trained using automatically generated pseudo-correct data. Even higher accuracy can be obtained by training with manually created correct data.

[0144] (Note) The following additional information is disclosed regarding the embodiments described above. (Additional note 1) Memory and At least one processor connected to the memory, Includes, The aforementioned processor, Enter extended word correspondences for the source text and the translated text, and translation quality tags for the source text and the translated text. Based on the aforementioned extended word correspondence and the translation quality tag, the editing tag is estimated. Estimation device. (Additional note 2) The processor estimates the extended word correspondence between the source text and the translated text using a trained extended word correspondence model. The estimation device described in Appendix 1. (Additional note 3) Based on the extended word support and the translation quality tag, the processor estimates, as the editing tag, at least one of the following: a replacement tag indicating that a word in the translated text should be replaced with the correct translation of the word in the source text; an insertion tag indicating that the translation of the word in the source text should be inserted into the translated text; and a deletion tag indicating that a word should be deleted from the translated text. Estimation device as described in Appendix 1 or 2. (Additional note 4) Memory and At least one processor connected to the memory, Includes, The aforementioned processor, The software automatically generates pseudo-correct data for extended word mapping to the source text and machine-translated text, and manually generates correct data for extended word mapping to the source text and machine-translated text. A language model is created by fine-tuning a pre-trained multilingual model using the aforementioned pseudo-ground data, and an extended word-response model is created by fine-tuning the language model using the aforementioned ground data. Learning device.

[0145] Regarding the above statement, "Input pseudo-correct data for extended word correspondences to the source text and machine translation automatically generated by software, and correct data for extended word correspondences to the source text and machine translation manually," "automatically generated by software" refers to "pseudo-correct data," "for the source text and machine translation" refers to "extended word correspondences," and "manually generated" refers to "correct data." (Additional note 5) An input step for assigning extended vocabulary to the source text and the translated text, and for inputting translation quality tags to the source text and the translated text, An edit tag estimation step is performed to estimate edit tags based on the extended word correspondence and the translation quality tag, An estimation method performed by a computer processor, comprising the following: (Additional note 6) The first learning step involves creating a language model by fine-tuning a pre-trained multilingual model using pseudo-correct data of extended word correspondences for source text and machine-translated text automatically generated by software. The second learning step involves creating an extended word mapping model by fine-tuning the language model using the correct data for extended word mapping to the source text created by humans and the machine-translated text. A learning method performed by a computer processor, which includes [a specific feature / feature]. (Additional note 7) A non-temporary storage medium that stores a program executable by a computer to perform an estimation process, The aforementioned estimation process is, Enter extended word correspondences for the source text and the translated text, and translation quality tags for the source text and the translated text. Based on the aforementioned extended word correspondence and the translation quality tag, the editing tag is estimated. Non-transitory storage medium. (Additional note 8) A non-temporary storage medium that stores a program executable by a computer to perform a learning process, The aforementioned learning process is, The software automatically generates pseudo-correct data for extended word mapping to the source text and machine-translated text, and manually generates correct data for extended word mapping to the source text and machine-translated text. A language model is created by fine-tuning a pre-trained multilingual model using the aforementioned pseudo-ground data, and an extended word-response model is created by fine-tuning the language model using the aforementioned ground data. Non-transitory storage medium. (References) [1] Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm´an, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross-lingual representation learning at scale. In Proceedings of ACL-2020, pp. 8440-8451, 2020. [2] Alexis Conneau and Guillaume Lample. Cross-lingual Language Model Pretraining. In Proceedings of NeurIPS-2019, pp. 7059-7069, 2019. [3] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the NAACL-2019, pp. 4171-4186, 2019. [4] Zi-Yi Dou and Graham Neubig. Word alignment by fine-tuning embeddings on parallel corpora. In Proceedings of EACL-2021, pp. 2112-2128, 2021. [5] Chris Dyer, Victor Chahuneau, and Noah A. Smith. A simple, fast, and effective reparameterization of ibm model 2. In Proceedings of the NAACL-HLT-2013, pp. 644-648, 2013. [6] Erick Fonseca, Lisa Yankovskaya, Andr´e F. T. Martins, Mark Fishel, and Christian Federmann. Findings of the wmt 2019 shared tasks on quality estimation. In Proceedings of the WMT-2019, pp. 1-10, 2019. [7] Sarthak Garg, Stephan Peitz, Udhyakumar Nallasamy, and Matthias Paulik. Jointly learning to align and translate with transformer models. In Proceedings of the EMNLP-IJCNLP-2019, pp. 4452-4461, 2019. [8] Fabio Kepler, Jonay Tr´enous, Marcos Treviso, Miguel Vera, Ant´onio G´ois, M. Amin Farajian, Ant´onio V. Lopes, and Andr´e F. T. Martins. Unbabel's participation in the wmt19 translation quality estimation shared task. In Proceedings of the WMT-2019, pp. 78-84, 1999. [9] Hyun Kim, Joon-Ho Lim, Hyun-Ki Kim, and Seung-Hoon Na. Qe bert: Bilingual bert using multi-task learning for neural quality estimation. In Proceedings of the WMT-2019, pp. 85-89, 1999.

[10] Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. Moses: Open Source Toolkit for Statistical Machine Translation. In Proceedings of the ACL-2007, pp. 177-180, 2007.

[11] Masaaki Nagata, Katsuki Chousa, and Masaaki Nishino. A supervised word alignment method based on cross-language span prediction using multilingual bert. In Proceedings of EMNLP-2020, pp. 555-565, 2020.

[12] Franz Josef Och and Hermann Ney. A systematic comparison of various statistical alignment models. Computational Linguistics, Vol. 29, No. 1, pp. 19-51, 2003. 11

[13] Lane Schwartz, Isabel Lacruz, and Tatyana Bystrova. Effects of word alignment visualization on post-editing quality & speed. In Proceedings of Machine Translation Summit XV, pp. 186-199, 2015.

[14] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the NIPS 2017, pp. 5998-6008, 2017.

[15] Findings of the WMT 2020 Shared Task on Quality Estimation Lucia Specia, Fr´ed´eric Blain, Marina Fomicheva, Erick Fonseca, Vishrav Chaudhary, Francisco Guzm´an, Andr´e FT Martins (https: / / aclanthology.org / 2020.wmt-1.79 / )

[0146] Although this embodiment has been described above, the present invention is not limited to this specific embodiment, and various modifications and changes are possible within the scope of the gist of the invention as described in the claims. [Explanation of symbols]

[0147] 100 Learning Devices 120 Extended Word Support Learning Unit 122 Pre-trained multilingual model databases 111 Translation Quality Tag Training Data DB 121 Extended Word Support Training Data Database 140 Extended word correspondence estimation unit 141 Extended Word Support Model DB 110 Translation Quality Tag Model Learning Unit 112 Pre-trained multilingual model database 130 Translation Quality Tag Model Estimation Unit 131 Translation Quality Tag Model DB 150 Edit Tag Estimation Section Input sections 108, 118, 128, 138, 148 Output sections 109, 119, 129, 139, 149 200 Estimation device 1000 drive unit 1001 Recording media 1002 Auxiliary storage 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input device 1008 Output device

Claims

1. An input unit for inputting extended word correspondences for the source text and the translated text, including correctly translated and incorrectly translated word pairs, and translation quality tags indicating whether there are errors or no errors for each word in the source text and each word and gap in the translated text. The system includes an editing tag estimation unit that estimates editing tags based on the extended word correspondence and the translation quality tags, The editing tag is at least one of the following: a replacement tag indicating that a word in the translated text should be replaced with the correct translation of the word in the source text; an insertion tag indicating that the translation of the word in the source text should be inserted into the translated text; and a deletion tag indicating that a word should be deleted from the translated text. The editing tag estimation unit estimates the editing tag using a program corresponding to a predetermined rule for estimating the editing tag from the extended word correspondence and the translation quality tag, or using a trained neural network for estimating the editing tag from the extended word correspondence and the translation quality tag. Estimation device.

2. Extended word correspondence estimation unit estimates the extended word correspondence between the original text and the translated text using a trained extended word correspondence model. The estimation device according to claim 1, further comprising:

3. An input unit for inputting pseudo-correct data with extended word support, which includes correctly translated and incorrectly translated word pairs for source text and machine-translated text automatically generated by software, and correct data with extended word support, which includes correctly translated and incorrectly translated word pairs for source text and machine-translated text created manually. A learning device comprising: an extended word correspondence learning unit that creates a language model by fine-tuning a trained multilingual model using the aforementioned pseudo-correct data, and an extended word correspondence model by fine-tuning the language model using the aforementioned correct data, The aforementioned pseudo-correct data is, Extended word correspondence data created from the original text and machine-translated text using word correspondence software, or This is extended word correspondence data created by mediating words in the post-edited text, based on the word correspondence between the original text and the correct post-edited text, and the word correspondence between the machine-translated text and the post-edited text. Learning device.

4. An input step involves inputting an extended word correspondence for the source text and the translated text, including correctly translated and incorrectly translated word pairs, and a translation quality tag indicating whether there is an error or not for each word in the source text and each word and gap in the translated text. An estimation method performed by a computer, comprising: an editing tag estimation step of estimating editing tags based on the extended word correspondence and the translation quality tags, The editing tag is at least one of the following: a replacement tag indicating that a word in the translated text should be replaced with the correct translation of the word in the source text; an insertion tag indicating that the translation of the word in the source text should be inserted into the translated text; and a deletion tag indicating that a word should be deleted from the translated text. In the editing tag estimation step, the computer estimates the editing tag using a program corresponding to a predetermined rule for estimating the editing tag from the extended word correspondence and the translation quality tag, or using a trained neural network for estimating the editing tag from the extended word correspondence and the translation quality tag. Estimation method.

5. The first learning step involves creating a language model by fine-tuning a trained multilingual model using augmented word-matching pseudo-correct data, which includes correctly translated and incorrectly translated word pairs, for source text and machine-translated text automatically generated by software. A computer learning method comprising: a second learning step of creating an extended word correspondence model by fine-tuning the language model using extended word correspondence ground data, which includes correctly translated word pairs and incorrectly translated word pairs, for a source text created by a human and a machine-translated text, the method being executed by a computer, The aforementioned pseudo-correct data is, Extended word correspondence data created from the original text and machine-translated text using word correspondence software, or This is extended word correspondence data created by mediating words in the post-edited text, based on the word correspondence between the original text and the correct post-edited text, and the word correspondence between the machine-translated text and the post-edited text. Learning methods.

6. A program for causing a computer to function as a component in the estimation device described in claim 1 or 2.

7. A program for causing a computer to function as a component of the learning device described in claim 3.

Images