Generation apparatus, generation method, and program

Abstract

A generation device for generating a second sequence from constraint information and a first sequence which is a sequence of information, said second sequence being a separate sequence of information, and wherein the generation device comprises: an input generation unit that uses a constraint information list as input and outputs each element of one or more constraint information subsets included in the constraint information list as a vocabulary constraint; a sequence generation unit that generates one or more candidates pertaining to the second sequence using the first sequence and the vocabulary constraints; and a re-ranking unit that calculates a score indicating the suitability as the second sequence for each of the one or more candidates.

Description

Technical Field

[0001] The present invention relates to the technical field of machine translation.

Background Art

[0002] When converting a sentence in a certain domain to another domain (e.g., another language), a constraint is imposed with the aim of including all specified words and phrases (constrained words and phrases), which is called machine translation with vocabulary constraints. Since machine translation with vocabulary constraints can unify the translations for specific words, it is a particularly important technology in the translation of patent / legal / technical documents that require consistency.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

[0004] In vocabulary-constrained machine translation methods, the translated text is generated to include the given constraint words. However, whether constraint words are extracted automatically or manually, the selection of constraint words may include inappropriate constraint words (noise).

[0005] If a lexically constrained machine translation method is applied using all noisy constraint phrases as lexical constraints, it is expected that incorrect phrases will be included in the translated text, leading to a decrease in translation accuracy. This issue is not limited to machine translation; it can occur in any field that uses constraint information for sequence transformation.

[0006] This invention has been made in view of the above points, and aims to provide a technique for performing sequence transformations using constraint information with high accuracy. [Means for solving the problem]

[0007] According to the disclosed technology, a generation device for generating a second sequence of information from constraint information and a first sequence of information, An input generation unit that takes a constraint information list as input and outputs each element of a subset of multiple constraint information included in the constraint information list as a lexical constraint, For each of the multiple lexical constraints output from the input generation unit, A sequence generation unit that generates one or more candidates for the second sequence using the first sequence and the lexical constraints, The score indicating the appropriateness of the second series is, For each of the above multiple lexical constraints obtained A reranking unit that calculates for each of the one or more candidates mentioned above. A generating apparatus equipped with the following is provided. [Effects of the Invention]

[0008] The disclosed technology provides a technique for performing sequence transformations with high accuracy using constraint information. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of machine translation with vocabulary constraints. [Figure 2] This figure shows an example configuration of the generating device 100. [Figure 3] This is a flowchart illustrating the operation of the generating device 100. [Figure 4] This figure shows an example of the configuration of the extraction unit 120. [Figure 5] This figure shows an example of the configuration of the extraction unit 120. [Figure 6] This figure shows an example configuration of the generating device 100. [Figure 7] This figure shows an example configuration of the sequence generation unit 140. [Figure 8] This diagram shows an example of a machine translation model configuration. [Figure 9] This figure shows an example configuration of the sequence generation unit 140. [Figure 10] This figure shows the display image provided by the display unit 500. [Figure 11]It is a diagram showing a configuration example of the generation device 100. [Figure 12] It is a diagram showing the detailed settings and hyperparameters that are the basis in each setting used in the experiment. [Figure 13] It is a diagram showing the evaluation results. [Figure 14] It is a diagram showing a hardware configuration example of the device.

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments (these embodiments) of the present invention will be described with reference to the drawings. The embodiments described below are merely examples, and the embodiments to which the present invention is applied are not limited to the following embodiments.

[0011] In the embodiments described below, an example of applying the present invention to machine translation is shown. However, the present invention is applicable to sequence conversion in any field as long as it is sequence conversion using constraint information. For example, the present invention can also be used for tasks such as summarization tasks, utterance generation tasks, tasks of attaching explanatory texts to images, etc.

[0012] Also, in the embodiments described below, the unit of translation is a sentence, but the unit of translation may be arbitrary.

[0013] The generation device 100 described below provides specific improvements over the prior art that performs sequence conversion with constraints, and shows an improvement in the technical field related to sequence conversion with constraints. Also, the extraction device described below provides specific improvements over the prior art in the extraction of constraint information, and shows an improvement in the technical field related to the extraction of constraint information.

[0014] (Regarding the problems) Before explaining the configuration and operation according to these embodiments in detail, first, the prior art and the problems therewith will be explained. Note that the following explanation of the problems is not a known technique. Also, the problems described below are problems related to the technology of the embodiments.

[0015] As already explained, machine translation with vocabulary constraints is a method that imposes constraints on the translation of a sentence from one domain to another (e.g., another language) with the aim of including all specified words and phrases. For reference, Figure 1 shows an example of input and output in machine translation with vocabulary constraints.

[0016] In the example in Figure 1, the machine translation (MT Output), constraints, and lexical-constrained machine translation (Constrained MT Output) are shown for the original sentence, "We developed a geometric optical theory of standing waves based on ray coincidence." The underlined parts indicate the constraints.

[0017] As a prior art technique for lexically constrained machine translation, Non-Patent Document 1, "Chen, G., Chen, Y., and Li, VO (2021). "Lexically Constrained Neural Machine Translation with Explicit Alignment Guidance." Proceedings of the AAAI Conference on Artificial Intelligence," discloses a lexically constrained machine translation method for manually created constraint phrases. The method disclosed in Non-Patent Document 1 is also called the soft method. The method disclosed in Non-Patent Document 1 does not guarantee that the constraint phrases will always be included in the translated text.

[0018] Non-patent document 2, "Matt Post and David Vilar. 2018. Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1314-1324, New Orleans, Louisiana. Association for Computational Linguistics," and reference 1, "Chousa, K. and Morishita, M. (2021). "Input Augmentation Improves Constrained Beam Search for Neural Machine Translation: NTT at WAT 2021." In Proceedings of the 8th Workshop on Asian Translation (WAT), pp. 53-61, Online. Association for Computational Linguistics," also disclose a lexical-constrained machine translation method for manually created constraint phrases. This method guarantees that the constraint phrase will always be included in the translated text. This method is also known as the hard method.

[0019] There are also use cases where constraint phrases are created automatically rather than manually. For example, in the translation of domain documents containing many proper nouns, such as patents and scientific and technical papers, translation memories and bilingual dictionaries created from past translation results are often used. Therefore, a use case can be conceivable where machine translation with vocabulary constraints is performed using constraint phrases automatically extracted from the bilingual dictionary.

[0020] On the other hand, when constraint terms are extracted automatically, it is possible that the extracted constraint terms may contain noisy terms. Furthermore, even when constraint terms are extracted manually, noise may be present.

[0021] Conventional lexical-constrained machine translation methods disclosed in Non-Patent Documents 1 and 2, etc., assume that the given constraint phrases are included in the reference translation. Therefore, when applying a lexical-constrained machine translation method using extracted constraint phrases as lexical constraints, it is expected that incorrect phrases may be included in the translated text, leading to a decrease in translation accuracy.

[0022] Based on the above, the following describes techniques for reducing noise and appropriately extracting constraint phrases, as well as techniques for performing accurate lexical-constrained machine translation even when using sets of constraint phrases that may contain noise.

[0023] (Example of device configuration, overall operation) Figure 2 shows an example of the configuration of the generation device 100 in this embodiment. As shown in Figure 2, the generation device 100 has an input unit 110, an extraction unit 120, an input generation unit 130, a series generation unit 140, a reranking unit 150, and an output unit 160.

[0024] Furthermore, a bilingual dictionary DB200 and a model DB300 are provided. The bilingual dictionary DB200 stores bilingual dictionaries, and the model DB300 stores trained machine translation models. The bilingual dictionary DB200 and the model DB300 may be located outside the generation device 100 (as in the example in Figure 2) or inside the generation device 100.

[0025] Referring to the flowchart in Figure 3, the overall operation flow of the generation device 100 will be explained. In S101, the input unit 110 inputs the source language text. In S102, the extraction unit 120 automatically extracts constraint phrases based on the source language text (input text) input by the input unit 110 and the translation dictionary read from the translation dictionary DB 200.

[0026] In S103, the input generation unit 130 generates multiple inputs (lexical constraints) from any combination of constraint phrases. In S104, the sequence generation unit 140 uses the multiple inputs generated in S103 and the machine translation model read from the model DB 300 to perform translation on the input sentence. Here, a translation result is obtained for each of the multiple inputs generated in S103. That is, the sequence generation unit 140 uses a given sequence and lexical constraints to generate one or more candidates for another sequence based on a pre-trained sequence transformation model.

[0027] In S105, the reranking unit 150 predicts a reranking score for each translation result using the input sentence. In S106, the output unit 160 outputs the translation result (target language sentence) with the highest score. The configuration and operation of the main functional units are described in detail below.

[0028] (Extraction part 120) First, let's explain the extraction unit 120. The extraction unit 120 receives the source language text and the translation dictionary as input and outputs the source language text and the constraint vocabulary list. Note that the source language text may not be output.

[0029] Figure 4 is a diagram showing the configuration of the extraction unit 120. As shown in Figure 4, the extraction unit 120 includes a filtering unit 121, a division unit 122, and a constraint phrase extraction unit 123. The extraction unit 120 also refers to the bilingual dictionary 200. Note that the extraction unit 120 may also be configured without the filtering unit 121.

[0030] The bilingual dictionary DB200 stores a set of pairs of two words that are used to convert sequences. Specifically, in this embodiment which targets translation, the bilingual dictionary DB200 stores a set of <source language phrase, target language phrase> pairs. The source language phrase and target language phrase may each consist of multiple words. In this embodiment, one <source language phrase, target language phrase> pair is referred to as a "translation". The source language phrase and target language phrase may also be referred to as the source language translation and the target language translation, respectively.

[0031] Furthermore, when using the translation dictionary DB200 for tasks other than translation, its contents are not limited to sets of <source language phrase, target language phrase> pairs.

[0032] The filtering unit 121 filters out noisy translations from the translation dictionary. The filtered translation dictionary is stored in the translation dictionary DB 200, and the splitting unit 122 and the constraint phrase extraction unit 123 refer to the filtered translation dictionary.

[0033] The splitting unit 122 performs morphological analysis on the source language sentence and the source language phrases in the translation dictionary. In other words, the splitting unit 122 divides the source language sentence and the source language phrases in the translation dictionary into unit information. The constraint phrase extraction unit 123 extracts translations corresponding to the phrases contained in the source language sentence (examples of unit information obtained by splitting) and creates a constraint phrase list. The processing of each unit will be explained in more detail below.

[0034] <Extraction unit 120: Filtering unit 121> The filtering unit 121 removes translations that fall under (A) to (C) below, or words or phrases included in the translations, from the translation dictionary. However, it is not mandatory for the filtering unit 121 to perform all of (A) to (C), and it may perform at least one of (A) to (C). It may also perform filtering other than (A) to (C). In particular, in the modified examples 1 and 2 described later, the process of (C) below may be skipped.

[0035] (A) Translations that include words other than nouns / noun phrases (excluding verbs due to their conjugation) (B) Translations consisting of phrases with a length of 1 Examples of (B) include single-letter terms such as units. For example, the translation of "Target language: C, Source language: degrees" falls under (B).

[0036] (C) Cases where there is no unique correspondence between the source language and the target language (for example, cases where there are multiple translations for a single word or phrase in the source language). For translations that fall under (C), the translation shall be deleted. Alternatively, one translation shall be kept from multiple translations, and the others shall be deleted, so that there is a one-to-one correspondence between the source language phrase and the target language phrase. Any method may be used to keep one translation from multiple translations and delete the others; for example, one may keep the translation that is listed first, or one may keep the translation that appears most frequently.

[0037] For example, the translation of "source language: computer, target language: calculator, computer" falls under (C). In this case, for example, this translation should be deleted, or changed to something like "target language: computer, source language: calculator" so that there is a one-to-one correspondence between the source language phrase and the target language phrase.

[0038] <Extraction section 120: Division section 122> The splitting unit 122 divides (tokenizes) the source language text and the source language translation from the bilingual dictionary into morphemes and inserts a predetermined symbol (e.g., a space, " / ") at the morpheme boundaries. This splitting unit may differ from the splitting unit used in the subsequent translation process.

[0039] For example, if the original language sentence is "not limited to that," the original language sentence after processing by the splitting unit 122 becomes "not limited to that."

[0040] <Extraction unit 120: Constraint phrase extraction unit 123> The constraint phrase extraction unit 123 extracts translations corresponding to words and phrases contained in the source language sentence and creates a constraint phrase list using the extracted translations. An example of a specific constraint phrase extraction method is described below. Note that the dictionary format and search method are not limited to the method described below, and other methods may be used as long as they can extract constraint phrases corresponding to words and phrases contained in the source language sentence.

[0041] In this example, we will use a bilingual dictionary that represents the original language translations character by character using a data structure called a Trie tree.

[0042] The constraint phrase extraction unit 123 performs a prefix matching search on the set of source language translations in the translation dictionary, starting from the beginning of the source language sentence. When a translation (pair) containing a source language translation that matches a phrase in the source language sentence is found, the object translation is extracted as a constraint phrase. During the prefix matching search, the translation with the longest length of the source language translation phrase is selected.

[0043] For example, suppose that in the source language text, morphological analysis by the segmentation unit 122 results in three phrases, namely "ABC / GHI / XYZ". Here, A, B, C, etc., are letters. When the constraint phrase extraction unit 123 searches the source language translation in the translation dictionary using "ABC / GHI / XYZ", it matches the beginning (or front) of "ABC / GHI / XYZ".

[0044] As a result of the above search, even if four phrases such as "AB," "ABC," "ABCG," and "ABC / GHI" match, as will be explained later, "AB" and "ABCG" can be excluded from the match because their morphemes do not match. In this case, of the remaining "ABC" and "ABC / GHI," the object translation paired with "ABC / GHI," which has the longest phrase length in the source language translation, is extracted as a constraint phrase. Then, the same process is performed using "XYZ," which is the part after "ABC / GHI."

[0045] As in this embodiment, the division unit 122 pre-divides the source language text and the source language translation in the bilingual dictionary into morphemes (examples of unit information), and by performing a search while considering the morpheme boundaries, it is possible to prevent the mis-extraction of words that do not match in the division units. This is particularly effective when the source language is a language that is not separated by spaces, such as Japanese. For example, it is possible to prevent the source language translation "hana" (flower) from matching "sono / kagiri / de / wa / nai" in the source language text. In other words, "hana" will no longer be able to match "ha / na".

[0046] The prefix matching, longest match, and word segmentation methods described here are just examples of techniques for achieving less noisy and less ambiguous restricted phrase extraction. Other methods that can resolve ambiguity may also be used.

[0047] For example, when morphological analysis is performed in the splitting unit 122, information necessary for resolving ambiguity, such as part of speech, base form, stem, conjugation form, and reading (pronunciation), is attached to the split words, and matching is performed using this attached information. In other words, by using not only the string but also its accompanying information such as part of speech during matching, ambiguity can be resolved in situations where, for example, the word "in" in the original language sentence matches both the preposition "in" and the noun "inn" (inn), which are the original language translations. Resolving ambiguity during matching is an important factor in improving translation accuracy.

[0048] <Other examples of the configuration of the extraction unit 120> The extraction unit 120 may be configured as shown in Figure 5 instead of the configuration shown in Figure 4. In the configuration shown in Figure 5, instead of filtering the translation dictionary, the filtering unit 121 filters the constraint phrases extracted by the constraint phrase extraction unit 121.

[0049] The filtering process is the same as that performed by the filtering unit 121 described above. However, "translation" is read as "constraining phrase". Specifically, the filtering unit 121 removes constraint phrases that fall under (A) to (C) below from the extraction results of the constraint phrase extraction unit 123. However, it is not mandatory for the filtering unit 121 to perform all of (A) to (C), and it may perform at least one of (A) to (C). In addition, rules other than (A) to (C) may be used. In particular, when performing the modified examples 1 and 2 described later, the process of (C) below may be skipped.

[0050] (A) Restrictive phrases that include words other than nouns / noun phrases (verbs are removed because they have conjugations) (B) Constraint phrases consisting of a word or phrase of length 1 (C) Cases where the correspondence between the source language and the target language is not unique (for example, cases where there are multiple constraint terms for a single word in the source language). When implementing (C), if there are multiple constraint clauses for a single word in the source language, for example, the multiple constraint clauses may be deleted, or one constraint clause may be kept and the others deleted, so that there is a one-to-one correspondence between the source language word and the target language word.

[0051] (Regarding other configuration examples of the extraction unit 120 and the generation device 100) The extraction unit 120 may be a standalone device independent of the generation device 100. This standalone device may be called an extraction device. Furthermore, an extraction unit 120 included in the generation device 100 may also be called an extraction device. Additionally, the generation device 100 having an extraction unit 120 may be called an extraction device. Both the extraction unit 120 and the extraction device may also include either or both of the display information generation unit 170 and the modification unit 180 as described in the embodiment later.

[0052] When the extraction unit 120 is configured as a separate device independent of the generation device 100, the generation device 100 may not include the extraction unit 120. The configuration of the generation device 100 in this case is shown in Figure 6. In the configuration of Figure 6, the constraint phrase list generated by the extraction device is input to the generation device 100. However, in the configuration of Figure 6, a constraint phrase list other than the constraint phrase list generated by the extraction device (e.g., a constraint phrase list containing a lot of noise) may be input to the generation device 100.

[0053] The operation of the input generation unit 130, the sequence generation unit 140, and the reranking unit 150 in Figure 6 is the same as the operation of the input generation unit 130, the sequence generation unit 140, and the reranking unit 150 in Figure 2.

[0054] (Input generation unit 130) Next, the input generation unit 130 will be described. The input generation unit 130 receives a constraint phrase list as input and sets all elements of a subset of the words included in the constraint phrase list as lexical constraints. However, it is also possible to set only some of the elements out of all as lexical constraints.

[0055] Finally, the input generation unit 130 outputs the above lexical constraints as lexical constraints corresponding to the source language sentence input to the extraction unit 120. A specific example is shown below.

[0056] Assume that {A, B, C} is input to the input generation unit 130 as a constraint phrase list. Here, A, B, and C are constraint phrases.

[0057] The input generation unit 130 extracts {},{A},{B},{C},{A,B},{A,C},{B,C},{A,B,C} as elements of a subset of {A,B,C}, and outputs each of these as a lexical constraint.

[0058] Note that {{},{A},{B},{C},{A,B},{A,C},{B,C},{A,B,C}} are constraint lexical sets, and each {...} represents one lexical constraint.

[0059] The lexical constraints created from a subset of constraint words in ListC are 2 |C| These become individual vocabulary constraints, and as will be described later, multiple candidate translations can be obtained from each of these constraints.

[0060] (Sequence generation unit 140) Next, the sequence generation unit 140 will be explained. The sequence generation unit 140 holds a trained machine translation model read from the model DB 300. The sequence generation unit 140 repeats the following process a number of times equal to the number of lexical constraints (the number of elements in the lexical constraint set). For example, if the lexical constraint set is {{},{A},{B},{C},{A,B},{A,C},{B,C},{A,B,C}}, the process will be repeated 8 times.

[0061] The sequence generation unit 140 receives an input sentence (source language sentence) and lexical constraints as input. The sequence generation unit 140 generates a translated sentence (target language sentence) using a machine translation model by applying existing methods for machine translation with lexical constraints. Here, multiple translated sentences are generated as candidate translations (target language sentence candidates). The candidate translations are also assigned a score as a translated sentence.

[0062] Any existing method can be used for vocabulary-constrained machine translation, but for example, LeCA or LeCA+LCD can be used. LeCA is disclosed in Non-Patent Document 1 and is also called the soft method. LeCA+LCD is disclosed in the aforementioned Reference Document 1 and is also called the hard method.

[0063] The sequence generation unit 140 outputs multiple generated translation candidates. For example, the sequence generation unit 140 outputs a predetermined number of translation candidates in descending order of score. The "predetermined number" may be 1. In other words, only the translation with the highest score may be output. Here, for example, 30 translation candidates are output for each vocabulary constraint.

[0064] <Example of the configuration of the sequence generation unit 140> Figure 7 shows an example of the configuration of the sequence generation unit 140. As shown in Figure 7, the sequence generation unit 140 has a sequence conversion unit 141 and a search unit 142.

[0065] When using the soft method for generating the translated text, the sequence conversion unit 141 uses the vocabulary constraint information. When using the hard method, the sequence conversion unit 141 may or may not use the vocabulary constraint, depending on the type of hard method. The dotted line indicates the input of the vocabulary constraint to the sequence conversion unit 142. Among the hard methods, the LeCA+LCD mentioned above uses the vocabulary constraint information in the sequence conversion unit 141. The following describes the configuration / operation assuming LeCA+LCD.

[0066] In the sequence conversion unit 141, as shown in Figure 8, a model based on a general encoder-decoder model (e.g., Transformer) having an encoder and a decoder can be used as the machine translation model. However, the present invention can also be implemented using a model other than the encoder-decoder model.

[0067] The sequence conversion unit 141 receives the source language sentence and lexical constraints as input, and first creates an input sequence with added lexical constraint information by extending the source language sentence using the lexical constraints, and uses this as input to the machine translation model.

[0068] More specifically, in the above extension, the sequence conversion unit 141 converts the input sequence, which is the original language sentence X, to each constraint phrase C. i And, as follows <sep>A vocabulary-constrained input sequence is created by concatenating (linking) words together via a special delimiter string. <eos>This string indicates the end of a sentence.

[0069] [X, <sep> ,C1, <sep>,C2,…,C N , <eos>] The sequence conversion unit 141 takes the extended input sequence as input to a machine translation model and generates a sentence. More specifically, it outputs the probability of each word in the set of words that can constitute the output sequence.

[0070] The search unit 142 uses the output probabilities of the decoder in the machine translation model to search for an output sequence (or an approximate solution thereof) that maximizes the generation probability given an input sequence. The search unit 142 uses a grid beam search method based on beam search, which ensures that the output sequence satisfies all constraint vocabulary.

[0071] Note that the search unit 142 using grid beam search is just one example. Any processing method that performs a lexical-constrained search to include the constraint phrases may be used.

[0072] (Re-ranking section 150) Next, the reranking unit 150 will be explained. The reranking unit 150 receives one or more translation candidate sentences generated by the sequence generation unit 140 as input. For example, if the sequence generation unit 140 generates 30 translation candidate sentences for each vocabulary constraint, and there are 8 vocabulary constraints, the reranking unit 150 receives 30 translation candidate sentences as input for each vocabulary constraint in those 8 vocabulary constraints.

[0073] Next, the re-ranking unit 150 calculates a score for each translation candidate using the input sentence (source language sentence), and outputs the translation candidate with the best score as the final translation. At this point, it is also possible to output all (or some) translations and their scores, rather than narrowing it down to the one with the highest score. This allows the output unit 160 to present the translations to the user in a ranked format using the scores.

[0074] Regarding the method for calculating the score by the reranking unit 150, any method that can calculate the score of the translated text may be used, but for example, the methods in Example 1 and Example 2 below can be used.

[0075] Example 1: The reranking unit 150 uses the likelihood of the translated sentence candidates output by the machine translation model used for translation in the sequence generation unit 140 as a score.

[0076] Example 2: The reranking unit 150 uses a machine translation model trained on an Encoder-Decoder model called a Transformer for a Right-to-Left translation task that generates translated sentences from the end of a sentence to the beginning of a sentence as its reranking model, and uses the likelihood of forcing the output of translated sentence candidates with this reranking model as its score. Forcing the output of translated sentence candidates can also be rephrased as performing forced decoding using translated sentence candidates.

[0077] In other words, the source language sentence is input to the encoder of the reranking model, and the words of the translation candidate for which you want to evaluate the score (likelihood) are sequentially input to the decoder of the reranking model.

[0078] In Example 1 and Example 2, the likelihood output by the machine translation model can be any value that indicates plausibility. The likelihood output by the machine translation model can be a probability or a value other than a probability.

[0079] Furthermore, the reranking unit 150 may calculate the reranking score using both the likelihood of Example 1 and the likelihood of Example 2. For example, the average of the likelihoods of Example 1 and Example 2 may be used as the reranking score.

[0080] (Variation 1) Next, Modification 1 will be described. In Modification 1, the constraint phrase list generated by the extraction unit 120 can be one in which multiple target language phrases correspond to one source language phrase. Such a constraint phrase list may be called a constraint phrase list that allows multiple translations. For example, such a constraint phrase list may be generated if the filtering unit of the extraction unit 120 does not perform procedure (C).

[0081] For example, suppose there are multiple target language phrases A and A' for a given source language phrase, and the extraction unit 120 generates "A, A', B, C" which includes these two and B and C, as multiple elements in the constraint phrase list. For example, if the source language phrase is computer and the target language phrases are calculator and computer, then A and A' correspond to calculator and computer.

[0082] Here, we represent a list of constraint phrases with multiple elements as {{A,A'},{B},{C}}.

[0083] The input generation unit 130, having received {{A,A´},{B},{C}} from the extraction unit 120, generates {},{A},{B},{C},{A,B},{A,C},{B,C},{A,B,C} as well as {A´},{A´,B},{A´,C},{A´,B,C} as lexical constraints.

[0084] The input generation unit 130 inputs each of the generated lexical constraints to the sequence generation unit 140.

[0085] The sequence generation unit 140 uses each of the 12 lexical constraints {},{A},{B},{C},{A,B},{A,C},{B,C},{A,B,C},{A´},{A´,B},{A´,C},{A´,B,C} as a lexical constraint to perform 12 lexical-constrained machine translations and obtain candidate translations. For example, if one candidate translation is generated for each lexical constraint, 12 candidate translations will be obtained.

[0086] After performing vocabulary-constrained machine translation, the re-ranking unit 150 performs a re-ranking process using the method described above, and outputs, for example, the translation candidate with the highest score as the final translation.

[0087] (Modification 2) Next, let's describe a second modification. In this second modification as well, the constraint phrase list generated by the extraction unit 120 can be one in which multiple target language phrases correspond to one source language phrase.

[0088] In the second modification, the search process for the translated text in the search unit 142 of the sequence generation unit 140 may be modified to allow for multiple phenotypes of a single constraint phrase. In other words, the search may be performed so that each candidate constraint phrase satisfies at least one element. Specifically, this is as follows.

[0089] In the second variation, we assume that there are multiple target language phrases A and A' for a given source language phrase, and that "A, A', B, C", including B and C, is generated as multiple elements of the constraint phrase list by the extraction unit 120. Here, the constraint phrase list {A, B, C} is generated, and information indicating that A can also be A' is input from the extraction unit 120 to the input generation unit 130. Alternatively, the constraint phrase list {A, A', B, C} may be generated, and information indicating that A and A' can be either may be input from the extraction unit 120 to the input generation unit 130. Note that the above example shows a format that allows two ambiguities for one constraint phrase, but a format that allows three or more ambiguities for one constraint phrase may also be used.

[0090] For example, if three ambiguities are allowed for A, the constraint phrase list {A, B, C} is generated, and information indicating that A can be A' or A'' is input from the extraction unit 120 to the input generation unit 130. Alternatively, the constraint phrase list {A, A', A'', B, C} may be generated, and information indicating that A, A', and A'' can be any of them may be input from the extraction unit 120 to the input generation unit 130.

[0091] For a constraint phrase list {A, B, C}, if A can also be A', the input generation unit 130 generates seven lexical candidate constraints: {}, {{A, A'}}, {{B}}, {{C}}, {{A, A'}, {B}}, {{A, A'}, {C}}, {{A, A'}, {B}, {C}}. In the modified example 2, since a given source language phrase may correspond to multiple target language phrases (e.g., A, A'), there is ambiguity in the translation, and the vocabulary used as constraints is not fixed, so it is called a "lexical candidate constraint" instead of a lexical constraint. In other words, a "lexical candidate constraint" is a lexical constraint that retains ambiguity. Note that the above expression format for lexical candidate constraints is just one example. Any expression format other than the above format may be used as long as it can express that it can be either A or A'.

[0092] As shown in Figure 9, the sequence generation unit 140 receives the source language sentence along with the lexical candidate constraints as input. The sequence generation unit 140 uses each of the seven lexical candidate constraints {},{{A,A´}},{{B}},{{C}},{{A,A´},{B}},{{A,A´},{C}},{{A,A´},{B},{C}} as a lexical candidate constraint and performs seven lexical-constrained machine translations to obtain translated sentence candidates. For example, if one translated sentence candidate is generated for each lexical candidate constraint, seven translated sentence candidates will be obtained.

[0093] After performing vocabulary-constrained machine translation, the re-ranking unit 150 performs a re-ranking process using the method described above, and outputs, for example, the translation candidate with the highest score as the final translation.

[0094] When using a lexical candidate constraint that includes {A,A'}, the search unit 142 of the sequence generation unit 140 performs the search assuming that the word A may also be A'. In other words, it performs a search that takes ambiguity into account. For the search, for example, the method described in reference 2, "Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2017. Guided Open Vocabulary Image Captioning with Constrained Beam Search. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 936-945, Copenhagen, Denmark. Association for Computational Linguistics," disclosed at https: / / aclanthology.org / D17-1098 / , can be used. This method is an example of a "search that takes ambiguity into account."

[0095] The method disclosed in Reference 2 performs a lexical-constrained beam search that takes into account the ambiguity of the translation, where either A or A' is acceptable. In other words, the ambiguity between A and A' is resolved during the beam search.

[0096] It should be noted that the method disclosed in Reference 2 is a language generation method, but not a translation technique. There are no prior techniques that apply this method to search during the decoding process of translation.

[0097] In the descriptions of the embodiments described so far, and in the descriptions of modifications 1 and 2, the multiple target language terms (e.g., A, A') for a given source language term may not only be synonyms such as "computer" and "calculator," but also terms other than synonyms, such as "trunk" for "trunk of a car," "elephant's trunk," "trunk," and "main road." Since the meaning of words is not considered during the search in the search unit 142, A and A' may be completely unrelated terms. Furthermore, when using the technology according to the present invention for tasks other than translation, the criteria for determining which multiple words in the converted sequence correspond to the words in the original sequence can be set according to any arbitrary criteria.

[0098] Furthermore, in the descriptions of the embodiments described above, and in the descriptions of Modifications 1 and 2, it is also possible to convert non-base forms of words into their base forms during morphological analysis, taking into account word variations (plural forms, tense changes, etc.).

[0099] As an example, suppose the translation dictionary is English-Japanese and has the dictionary entry "corn - corn, fish eye". In this case, assume that the source language sentence "We roasted corns over the charcoal." is input to the generation device 100. In this case, when the extraction unit 120 performs morphological matching, "corns" matches the translation dictionary because "corn" is included as a morpheme. However, if, for example, the entry in the translation dictionary is "feet" and the morpheme in the input sentence is "foot", then it will not match. This problem can be solved by changing "foot" back to its base form and then performing the matching.

[0100] (Examples) Next, as a more specific example, we will describe an embodiment using the technology described above. In this embodiment, it is possible to edit (modify, add) constraint phrases in the display unit 500 (a device capable of display and input operations), which will be described later, and to check the target language sentence (translated sentence) for each constraint phrase each time.

[0101] <Display Image> First, the display image on the display unit 500 will be explained with reference to Figure 10. In the example shown in Figure 10, the user enters "We have prototyped a superconducting single-phase autotransformer with auxiliary windings only on the shunt winding" as the source language text and presses "Send".

[0102] The display unit 500 displays multiple constraint phrases (constraint phrase list) for the input source language sentence. The phrases displayed here as constraint phrases are the phrases that have been filtered by the filtering unit 121. To the right of these, the filtered constraint phrases are displayed in the form of "Do you want to add them?".

[0103] Users can select constraint terms they wish to modify (or delete) or add by checking checkboxes, and then modify (or delete) / add the selected constraint terms by pressing the corresponding buttons. Users can also add constraint terms they have created themselves.

[0104] By pressing "Update" in the displayed image, you can view the target language sentence using the constraint phrases at that time.

[0105] <Device configuration and operation> Figure 11 shows an example configuration of the generation device 100 for realizing the above-described display. As shown in Figure 11, the generation device 100 in this embodiment includes an extraction unit 120, a display information generation unit 170, a modification unit 180, a generation unit 190, a bilingual dictionary DB 200, and a constraint word list DB 400. Note that the modification unit 180 may be included in the display information generation unit 170.

[0106] The bilingual dictionary DB200 and the constraint term list DB400 may be located outside the generation device 100. The generation unit 190 may also be located outside the generation device 100 (for example, on a separate server). Furthermore, the generation device 100 may be used to display the list of constraint terms on the display unit 500. In that case, the generation device 100 may consist only of the extraction unit 120 and the display information generation unit 170 from the functional units shown in Figure 11. The generation device 100 may also be called the extraction device. The functions of each unit are as follows.

[0107] The extraction unit 120 is the extraction unit 120 shown in Figure 4 or Figure 5. It takes the source language sentence as input and outputs a list of constraint phrases. The output list of constraint phrases is stored in the constraint phrase list DB 400 and is also input to the display information generation unit 170. The extraction unit 120 may also output the filtered constraint phrases as a list of filtered phrases. The output list of filtered phrases is also input to the display information generation unit 170.

[0108] The display information generation unit 170 generates information (referred to as information for presenting the constraint phrase list) for displaying the constraint phrase list on the display unit 500. The information for presenting the constraint phrase list includes the constraint phrase list. The information for presenting the constraint phrase list may also include information from the filter phrase list, such as deleted information, filter candidate phrases, or additional candidate phrases. The information for presenting the constraint phrase list is transmitted from the display information generation unit 170 to the display unit 500 and input to the display unit 500. The display information generation unit 170 may also generate display information for displaying the constraint phrases in a format that allows for modification, along with the target language sentence (translated text) generated using the constraint phrases. Furthermore, when the generation device 100 receives added or modified constraint phrases from the display unit 500, the display information generation unit 170 may acquire the target language sentence (translated text) generated based on the received constraint phrases and generate display information for displaying the target language sentence (translated text).

[0109] Furthermore, the display information generation unit 170 may generate "correction support information" for the user to use when reviewing the constraint word list and transmit it to the display unit 500. The correction support information includes at least one of the source language sentence entered by the user, the extracted constraint word list, and the target language sentence generated based on the extracted constraint word list.

[0110] The modification unit 180 receives from the display unit 500 at least one of the additional constraint phrases or modified constraint phrases as information that the user has modified the presented list of constraint phrases.

[0111] The modification unit 180 modifies the information stored in the constraint word list DB 400 based on the received information. Alternatively, when the constraint word list is modified, the target language sentence may be generated again by machine translation with vocabulary constraints based on the modified constraint word list, the display information generation unit 170 may generate modification support information containing the target language sentence, and transmit this to the display unit 500 for display on the display unit 500.

[0112] The generation unit 190 includes an input generation unit 130, a sequence generation unit 140, and a reranking unit 150. As described above, the generation unit 190 uses these functional units to generate a target language sentence (translated sentence) that takes vocabulary constraints into account, based on the constraint phrase list read from the constraint phrase list DB400 and the source language sentence received from the display unit 500, and inputs the generated target language sentence to the display information generation unit 170.

[0113] The display unit 500 is, for example, a computer (terminal) having a display. The display unit 500 is connected to the generation device 100 via a network. As explained with reference to Figure 10, the display unit 500 receives the source language text from the user and displays a list of constraint words, etc. The display unit 500 also receives instructions for adding or modifying constraint words and the source language text. The display unit 500 can also output the source language text, the final target language text, and the final list of constraint words as a set.

[0114] With the generation device 100 in the above embodiment, by interactively and repeatedly modifying the list of constraint phrases while checking the results of vocabulary-constrained machine translation, it is possible to generate a target language sentence (translated sentence) that is closer to the user's image.

[0115] (Experimental results) In the following explanation of the experimental results, "the generation apparatus 100 in this embodiment" will be referred to as the proposed method or proposed system.

[0116] To confirm the effectiveness of the proposed machine translation method with vocabulary constraints, which involves re-ranking translation candidates against automatically extracted vocabulary constraints, we evaluated the accuracy of machine translation with vocabulary constraints using vocabulary constraints automatically extracted from a parallel dictionary, focusing on Japanese-English translation.

[0117] <About the bilingual dictionary> To extract vocabulary constraints, we used the general-purpose EDR Japanese-English Bilingual Dictionary (EDR-JE) and the Japanese-English translation system ALT-J / E as bilingual dictionaries.

[0118] <Model> The following translation models were used for evaluation.

[0119] Transformer LeCA + {EDR-JE, ALT-J / E} ·LeCA+LCD + {EDR-JE, ALT-J / E} ASPEC was used as the parallel corpus for training and evaluating the translation models. The detailed settings and hyperparameters for each model are shown in Figure 12.

[0120] For constraints extracted from the dictionary, 2 |C| For each lexical constraint, we used the top 30 generated sentences. For reranking the translation candidates, we used a score calculated by a reranking model using the source language sentences and translation candidates via Reranker.

[0121] The reranking model used was a model trained on a Right-to-Left translation task that generates translated sentences from the end of a sentence to the beginning of a sentence, using a Transformer (big) model. The reranking score was calculated using the likelihood obtained when the input translation candidates were forced-decoded. BLEU, an automated evaluation metric for translation accuracy, was used to evaluate each method.

[0122] <About the experimental results> Figure 13 shows the translation accuracy of each method when using vocabulary constraints automatically extracted from a bilingual dictionary. It can be seen that in the Reranker, which uses a score from a reranking model, LeCA and LeCA+LCD improve translation accuracy compared to the baseline (Transformer). Furthermore, Figure 13 shows that translation accuracy is high regardless of the type of dictionary used.

[0123] (Example hardware configuration) Any of the devices described in this embodiment (generation device 100, extraction device) can be realized, for example, by having a computer run a program. This computer may be a physical computer or a virtual machine on the cloud.

[0124] In other words, the device can be realized by using hardware resources such as the CPU and memory built into a computer to execute a program corresponding to the processing performed by the device. The program can be recorded on a computer-readable recording medium (such as portable memory), saved, and distributed. It can also be provided via a network, such as the Internet or email.

[0125] Figure 14 shows an example of the hardware configuration of the computer described above. The computer in Figure 14 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., all of which are interconnected by a bus BS. The computer may also be equipped with a GPU.

[0126] The program that enables processing on the computer is provided, for example, on a recording medium 1001 such as a CD-ROM or memory card. When the recording medium 1001 containing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001; it may also be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files and data.

[0127] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when a program startup command is received. The CPU 1004 implements the functions related to the light touch maintenance device 100 according to the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) etc. generated by the program. The input device 1007 consists of a keyboard and mouse, buttons, or a touch panel etc., and is used to input various operation commands. The output device 1008 outputs the calculation results.

[0128] (Summary of the embodiments, effects, etc.) As described above, the technology described in this embodiment makes it possible to automatically extract constraint phrases used in vocabulary-constrained machine translation with low noise. Furthermore, the technology described in this embodiment enables accurate translation in vocabulary-constrained machine translation.

[0129] The embodiments described above are further disclosed in the following appendices 1 and 2.

[0130] <Note 1> (Additional note 1) Memory and At least one processor connected to the memory, Includes, The aforementioned processor, The dictionary is a collection of pairs of first and second pieces of information. The first piece of information and each of the first sequence are divided into unit pieces of information. The second information corresponding to the first information that matches the unit information of the first series is extracted from the dictionary as constraint information used to generate the second series based on the first series. Extraction device. (Additional note 2) The processor removes pairs from the dictionary that match predetermined rules and uses the dictionary after the removal process has been performed. The extraction apparatus described in Appendix 1. (Additional note 3) A pair that conforms to the predetermined rules is at least one of the following: a pair containing a word or phrase other than a noun, a pair consisting of a word or phrase of length 1, or a pair in which the correspondence between the first piece of information and the second piece of information is not unique. The extraction apparatus described in Appendix 2. (Additional note 4) The processor performs matching between the unit information of the first sequence and the first information in order to resolve ambiguity. An extraction device as described in any one of the appendices 1 to 3. (Additional note 5) The processor generates display information for transmitting the constraint information to the display unit, and receives the constraint information that has been added to or modified from the constraint information displayed on the display unit. An extraction device as described in any one of the appendices 1 through 4. (Additional note 6) Memory and At least one processor connected to the memory, Includes, The aforementioned processor, Taking the first sequence as input, constraint information is extracted based on the first sequence and a dictionary which is a set of pairs of first and second pieces of information. A second sequence is generated based on the aforementioned constraint information and the first sequence. The constraint information is generated in a format that allows for modification, and display information is generated to display it together with the second series. generator. (Additional note 7) When the processor receives constraint information that has been added or modified, it obtains a series generated based on the received constraint information and generates display information for displaying that series. The generating apparatus described in Appendix 6. (Additional note 8) The processor generates display information to show the constraint information filtered based on predetermined rules as additional candidates. The generating apparatus described in Appendix 6 or 7. (Additional note 9) A computer-based extraction method, The dictionary is a collection of pairs of first and second pieces of information, and the first piece of information is divided into unit pieces in a division step. A constraint information extraction step of extracting from the dictionary the second information corresponding to the first information that matches the unit information of the first series, as constraint information used to generate the second series based on the first series. An extraction method comprising the following features. (Additional note 10) A generation method performed by a computer, An extraction step that takes a first sequence as input and extracts constraint information based on the first sequence and a dictionary which is a set of pairs of first and second pieces of information, A generation step of generating a second sequence based on the constraint information and the first sequence, A display information generation step that generates display information for displaying the aforementioned constraint information together with the second series in a modifiable format, A generation method comprising the following: (Additional note 11) A non-temporary storage medium storing a program for causing a computer to function as an extraction device as described in any one of the appendices 1 through 5.

[0131] <Note 2> (Additional note 1) A generation device for generating a second sequence of information from constraint information and a first sequence of information, which is another sequence of information, Memory and At least one processor connected to the memory, Includes, The aforementioned processor, A constraint information list is input, and each element of one or more subsets of constraint information included in the constraint information list is output as a lexical constraint. Using the first sequence and the lexical constraints, one or more candidates for the second sequence are generated. A score indicating the suitability of the second series is calculated for each of the one or more candidates. generator. (Additional note 2) The processor calculates the score based on at least one of the likelihood output by the model used to generate the candidates in the sequence generation unit and the likelihood obtained from the candidates by the reranking model. The generating apparatus described in Appendix 1. (Additional note 3) If the constraint information list includes constraint information with ambiguity, the processor generates one or more candidates by performing a beam search with vocabulary constraints that take ambiguity into account. The generating apparatus described in Appendix 1 or 2. (Additional note 4) At least one constraint information is input to the processor in a format that allows for two or more ambiguities, and the processor generates a lexical constraint while retaining those ambiguities. A generating apparatus as described in any one of the appendices 1 to 3. (Additional note 5) A computer generation method for generating a second sequence of information from constraint information and a first sequence of information, wherein An input generation step that inputs a constraint information list and outputs each element of one or more subsets of constraint information included in the constraint information list as a lexical constraint, A sequence generation step that generates one or more candidates for the second sequence using the first sequence and the lexical constraints, A reranking step is performed to calculate a score indicating the suitability of each of the one or more candidates as the second series. A generation method comprising the following: (Additional note 6) A non-temporary storage medium storing a program for causing a computer to function as a component in any one of the generating devices described in any one of the appendices 1 through 4.

[0132] Although this embodiment has been described above, the present invention is not limited to this specific embodiment, and various modifications and changes are possible within the scope of the gist of the invention as described in the claims. [Explanation of Symbols]

[0133] 100 generator 110 Input Section 120 Extraction part 121 Filtering section 122 Division 123 Constraint phrase extraction part 130 Input generation unit 140 Sequence generator 141 Sequence Conversion Unit 142 Search Department 150 Re-ranking Department 160 Output section 170 Display information generation section 180 Correction Department 190 Generation part 200 Bilingual Dictionary Database 300 Model DB 400 Constraint Phrase List DB 500 Display 1000 drive unit 1001 Recording media 1002 Auxiliary storage 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input device 1008 Output device< / eos> < / sep> < / sep> < / eos> < / sep>

Claims

1. A generation device for generating a second sequence of information from constraint information and a first sequence of information, which is another sequence of information, An input generation unit that takes a constraint information list as input and outputs each element of a subset of multiple constraint information included in the constraint information list as a lexical constraint, A sequence generation unit generates one or more candidates for the second sequence using the first sequence and the lexical constraints for each of the multiple lexical constraints output from the input generation unit, A reranking unit calculates a score indicating the appropriateness of the second series for each of the one or more candidates obtained for each of the multiple lexical constraints. A generating device equipped with the following features.

2. The reranking unit calculates the score based on at least one of the likelihood output by the model used to generate the candidates in the sequence generation unit and the likelihood obtained from the candidates by the reranking model. The generating apparatus according to claim 1.

3. If the constraint information list contains ambiguous constraint information, the sequence generation unit generates one or more candidates by performing a beam search with vocabulary constraints that take ambiguity into account. The generating apparatus according to claim 1.

4. At least one constraint information is input to the input generation unit in a format that allows for two or more ambiguities, and the input generation unit generates lexical constraints while retaining those ambiguities. The generating apparatus according to claim 1.

5. The constraint information is a constraint phrase, and the constraint information list is a constraint phrase list. The generating apparatus according to claim 1.

6. The constraint information list includes a constraint phrase list in which multiple target language phrases correspond to a source language phrase. The generating apparatus according to claim 5.

7. A generation method performed by a generation device for generating a second sequence of information from constraint information and a first sequence of information, the second sequence of information being another sequence of information, An input generation step that takes a constraint information list as input and outputs each element of a subset of multiple constraint information included in the constraint information list as a lexical constraint, A sequence generation step that generates one or more candidates for the second sequence using the first sequence and the lexical constraints for each of the multiple lexical constraints output by the input generation step, A reranking step is performed to calculate a score indicating the appropriateness of the second series for each of the one or more candidates obtained for each of the multiple lexical constraints. A generation method comprising the following:

8. A program for causing a computer to function as a component in the generating apparatus described in any one of claims 1 to 6.

Images